Module org.apache.lucene.core
Class Lucene99FlatVectorsWriter
- java.lang.Object
-
- org.apache.lucene.codecs.FlatVectorsWriter
-
- org.apache.lucene.codecs.lucene99.Lucene99FlatVectorsWriter
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,Accountable
public final class Lucene99FlatVectorsWriter extends FlatVectorsWriter
Writes vector values to index segments.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
Lucene99FlatVectorsWriter.FieldWriter<T>
(package private) static class
Lucene99FlatVectorsWriter.FlatCloseableRandomVectorScorerSupplier
-
Field Summary
Fields Modifier and Type Field Description private java.util.List<Lucene99FlatVectorsWriter.FieldWriter<?>>
fields
private boolean
finished
private IndexOutput
meta
private SegmentWriteState
segmentWriteState
private static long
SHALLLOW_RAM_BYTES_USED
private IndexOutput
vectorData
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description Lucene99FlatVectorsWriter(SegmentWriteState state)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description FlatFieldVectorsWriter<?>
addField(FieldInfo fieldInfo, KnnFieldVectorsWriter<?> indexWriter)
Add a new field for indexing, allowing the user to provide a writer that the flat vectors writer can delegate to if additional indexing logic is required.void
close()
void
finish()
Called once at the end before closevoid
flush(int maxDoc, Sorter.DocMap sortMap)
Flush all buffered data on disk *void
mergeOneField(FieldInfo fieldInfo, MergeState mergeState)
Write field for mergingCloseableRandomVectorScorerSupplier
mergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState)
Write the field for merging, providing a scorer over the newly merged flat vectors.long
ramBytesUsed()
Return the memory usage of this object in bytes.private static DocsWithFieldSet
writeByteVectorData(IndexOutput output, ByteVectorValues byteVectorValues)
Writes the byte vector values to the output and returns a set of documents that contains vectors.private void
writeByteVectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData)
private void
writeField(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int maxDoc)
private void
writeFloat32Vectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData)
private void
writeMeta(FieldInfo field, int maxDoc, long vectorDataOffset, long vectorDataLength, DocsWithFieldSet docsWithField)
private long
writeSortedByteVectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int[] ordMap)
private long
writeSortedFloat32Vectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int[] ordMap)
private void
writeSortingField(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int maxDoc, Sorter.DocMap sortMap)
private static DocsWithFieldSet
writeVectorData(IndexOutput output, FloatVectorValues floatVectorValues)
Writes the vector values to the output and returns a set of documents that contains vectors.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
-
-
-
Field Detail
-
SHALLLOW_RAM_BYTES_USED
private static final long SHALLLOW_RAM_BYTES_USED
-
segmentWriteState
private final SegmentWriteState segmentWriteState
-
meta
private final IndexOutput meta
-
vectorData
private final IndexOutput vectorData
-
fields
private final java.util.List<Lucene99FlatVectorsWriter.FieldWriter<?>> fields
-
finished
private boolean finished
-
-
Constructor Detail
-
Lucene99FlatVectorsWriter
Lucene99FlatVectorsWriter(SegmentWriteState state) throws java.io.IOException
- Throws:
java.io.IOException
-
-
Method Detail
-
addField
public FlatFieldVectorsWriter<?> addField(FieldInfo fieldInfo, KnnFieldVectorsWriter<?> indexWriter) throws java.io.IOException
Description copied from class:FlatVectorsWriter
Add a new field for indexing, allowing the user to provide a writer that the flat vectors writer can delegate to if additional indexing logic is required.- Specified by:
addField
in classFlatVectorsWriter
- Parameters:
fieldInfo
- fieldInfo of the field to addindexWriter
- the writer to delegate to, can be null- Returns:
- a writer for the field
- Throws:
java.io.IOException
- if an I/O error occurs when adding the field
-
flush
public void flush(int maxDoc, Sorter.DocMap sortMap) throws java.io.IOException
Description copied from class:FlatVectorsWriter
Flush all buffered data on disk *- Specified by:
flush
in classFlatVectorsWriter
- Throws:
java.io.IOException
-
finish
public void finish() throws java.io.IOException
Description copied from class:FlatVectorsWriter
Called once at the end before close- Specified by:
finish
in classFlatVectorsWriter
- Throws:
java.io.IOException
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.
-
writeField
private void writeField(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int maxDoc) throws java.io.IOException
- Throws:
java.io.IOException
-
writeFloat32Vectors
private void writeFloat32Vectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData) throws java.io.IOException
- Throws:
java.io.IOException
-
writeByteVectors
private void writeByteVectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData) throws java.io.IOException
- Throws:
java.io.IOException
-
writeSortingField
private void writeSortingField(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int maxDoc, Sorter.DocMap sortMap) throws java.io.IOException
- Throws:
java.io.IOException
-
writeSortedFloat32Vectors
private long writeSortedFloat32Vectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int[] ordMap) throws java.io.IOException
- Throws:
java.io.IOException
-
writeSortedByteVectors
private long writeSortedByteVectors(Lucene99FlatVectorsWriter.FieldWriter<?> fieldData, int[] ordMap) throws java.io.IOException
- Throws:
java.io.IOException
-
mergeOneField
public void mergeOneField(FieldInfo fieldInfo, MergeState mergeState) throws java.io.IOException
Description copied from class:FlatVectorsWriter
Write field for merging- Overrides:
mergeOneField
in classFlatVectorsWriter
- Throws:
java.io.IOException
-
mergeOneFieldToIndex
public CloseableRandomVectorScorerSupplier mergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState) throws java.io.IOException
Description copied from class:FlatVectorsWriter
Write the field for merging, providing a scorer over the newly merged flat vectors. This way any additional merging logic can be implemented by the user of this class.- Specified by:
mergeOneFieldToIndex
in classFlatVectorsWriter
- Parameters:
fieldInfo
- fieldInfo of the field to mergemergeState
- mergeState of the segments to merge- Returns:
- a scorer over the newly merged flat vectors, which should be closed as it holds a temporary file handle to read over the newly merged vectors
- Throws:
java.io.IOException
- if an I/O error occurs when merging
-
writeMeta
private void writeMeta(FieldInfo field, int maxDoc, long vectorDataOffset, long vectorDataLength, DocsWithFieldSet docsWithField) throws java.io.IOException
- Throws:
java.io.IOException
-
writeByteVectorData
private static DocsWithFieldSet writeByteVectorData(IndexOutput output, ByteVectorValues byteVectorValues) throws java.io.IOException
Writes the byte vector values to the output and returns a set of documents that contains vectors.- Throws:
java.io.IOException
-
writeVectorData
private static DocsWithFieldSet writeVectorData(IndexOutput output, FloatVectorValues floatVectorValues) throws java.io.IOException
Writes the vector values to the output and returns a set of documents that contains vectors.- Throws:
java.io.IOException
-
close
public void close() throws java.io.IOException
- Throws:
java.io.IOException
-
-