Module org.apache.lucene.core
Package org.apache.lucene.util.hnsw
Class HnswConcurrentMergeBuilder.ConcurrentMergeWorker
- java.lang.Object
-
- org.apache.lucene.util.hnsw.HnswGraphBuilder
-
- org.apache.lucene.util.hnsw.HnswConcurrentMergeBuilder.ConcurrentMergeWorker
-
- All Implemented Interfaces:
HnswBuilder
- Enclosing class:
- HnswConcurrentMergeBuilder
private static final class HnswConcurrentMergeBuilder.ConcurrentMergeWorker extends HnswGraphBuilder
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
HnswGraphBuilder.GraphBuilderKnnCollector
-
-
Field Summary
Fields Modifier and Type Field Description private int
batchSize
private BitSet
initializedNodes
private java.util.concurrent.atomic.AtomicInteger
workProgress
A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph.-
Fields inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
DEFAULT_BEAM_WIDTH, DEFAULT_MAX_CONN, hnsw, HNSW_COMPONENT, randSeed
-
-
Constructor Summary
Constructors Modifier Constructor Description private
ConcurrentMergeWorker(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, BitSet initializedNodes, java.util.concurrent.atomic.AtomicInteger workProgress)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addGraphNode(int node)
Inserts a doc with vector value to the graphprivate int
getStartPos(int maxOrd)
Reserve the work by atomically increment theworkProgress
private void
run(int maxOrd)
This method first try to "reserve" part of work by callinggetStartPos(int)
and then callingHnswGraphBuilder.addVectors(int, int)
to actually add the nodes to the graph.-
Methods inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
addVectors, build, create, create, getGraph, setInfoStream
-
-
-
-
Field Detail
-
workProgress
private final java.util.concurrent.atomic.AtomicInteger workProgress
A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph.
-
initializedNodes
private final BitSet initializedNodes
-
batchSize
private int batchSize
-
-
Constructor Detail
-
ConcurrentMergeWorker
private ConcurrentMergeWorker(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, BitSet initializedNodes, java.util.concurrent.atomic.AtomicInteger workProgress) throws java.io.IOException
- Throws:
java.io.IOException
-
-
Method Detail
-
run
private void run(int maxOrd) throws java.io.IOException
This method first try to "reserve" part of work by callinggetStartPos(int)
and then callingHnswGraphBuilder.addVectors(int, int)
to actually add the nodes to the graph. By doing this we are able to dynamically allocate the work to multiple workers and try to make all of them finishing around the same time.- Throws:
java.io.IOException
-
getStartPos
private int getStartPos(int maxOrd)
Reserve the work by atomically increment theworkProgress
-
addGraphNode
public void addGraphNode(int node) throws java.io.IOException
Description copied from interface:HnswBuilder
Inserts a doc with vector value to the graph- Specified by:
addGraphNode
in interfaceHnswBuilder
- Overrides:
addGraphNode
in classHnswGraphBuilder
- Throws:
java.io.IOException
-
-