Class Lucene99HnswVectorsWriter

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, Accountable

    public final class Lucene99HnswVectorsWriter
    extends KnnVectorsWriter
    Writes vector values and knn graphs to index segments.
    • Field Detail

      • SHALLOW_RAM_BYTES_USED

        private static final long SHALLOW_RAM_BYTES_USED
      • M

        private final int M
      • beamWidth

        private final int beamWidth
      • numMergeWorkers

        private final int numMergeWorkers
      • finished

        private boolean finished
    • Constructor Detail

      • Lucene99HnswVectorsWriter

        Lucene99HnswVectorsWriter​(SegmentWriteState state,
                                  int M,
                                  int beamWidth,
                                  FlatVectorsWriter flatVectorWriter,
                                  int numMergeWorkers,
                                  TaskExecutor mergeExec)
                           throws java.io.IOException
        Throws:
        java.io.IOException
    • Method Detail

      • flush

        public void flush​(int maxDoc,
                          Sorter.DocMap sortMap)
                   throws java.io.IOException
        Description copied from class: KnnVectorsWriter
        Flush all buffered data on disk *
        Specified by:
        flush in class KnnVectorsWriter
        Throws:
        java.io.IOException
      • finish

        public void finish()
                    throws java.io.IOException
        Description copied from class: KnnVectorsWriter
        Called once at the end before close
        Specified by:
        finish in class KnnVectorsWriter
        Throws:
        java.io.IOException
      • ramBytesUsed

        public long ramBytesUsed()
        Description copied from interface: Accountable
        Return the memory usage of this object in bytes. Negative values are illegal.
      • reconstructAndWriteGraph

        private HnswGraph reconstructAndWriteGraph​(OnHeapHnswGraph graph,
                                                   int[] newToOldMap,
                                                   int[] oldToNewMap,
                                                   int[][] levelNodeOffsets)
                                            throws java.io.IOException
        Reconstructs the graph given the old and new node ids.

        Additionally, the graph node connections are written to the vectorIndex.

        Parameters:
        graph - The current on heap graph
        newToOldMap - the new node ids indexed to the old node ids
        oldToNewMap - the old node ids indexed to the new node ids
        levelNodeOffsets - where to place the new offsets for the nodes in the vector index.
        Returns:
        The graph
        Throws:
        java.io.IOException - if writing to vectorIndex fails
      • reconstructAndWriteNeighbours

        private void reconstructAndWriteNeighbours​(NeighborArray neighbors,
                                                   int[] oldToNewMap,
                                                   int maxOrd)
                                            throws java.io.IOException
        Throws:
        java.io.IOException
      • writeGraph

        private int[][] writeGraph​(OnHeapHnswGraph graph)
                            throws java.io.IOException
        Parameters:
        graph - Write the graph in a compressed format
        Returns:
        The non-cumulative offsets for the nodes. Should be used to create cumulative offsets.
        Throws:
        java.io.IOException - if writing to vectorIndex fails
      • writeMeta

        private void writeMeta​(FieldInfo field,
                               long vectorIndexOffset,
                               long vectorIndexLength,
                               int count,
                               HnswGraph graph,
                               int[][] graphLevelNodeOffsets)
                        throws java.io.IOException
        Throws:
        java.io.IOException
      • close

        public void close()
                   throws java.io.IOException
        Throws:
        java.io.IOException