Class Lucene90CompressingStoredFieldsWriter

    • Field Detail

      • FIELDS_EXTENSION

        public static final java.lang.String FIELDS_EXTENSION
        Extension of stored fields file
        See Also:
        Constant Field Values
      • INDEX_EXTENSION

        public static final java.lang.String INDEX_EXTENSION
        Extension of stored fields index
        See Also:
        Constant Field Values
      • META_EXTENSION

        public static final java.lang.String META_EXTENSION
        Extension of stored fields meta
        See Also:
        Constant Field Values
      • INDEX_CODEC_NAME

        public static final java.lang.String INDEX_CODEC_NAME
        Codec name for the index.
        See Also:
        Constant Field Values
      • TYPE_BITS

        static final int TYPE_BITS
      • TYPE_MASK

        static final int TYPE_MASK
      • segment

        private final java.lang.String segment
      • chunkSize

        private final int chunkSize
      • maxDocsPerChunk

        private final int maxDocsPerChunk
      • numStoredFields

        private int[] numStoredFields
      • endOffsets

        private int[] endOffsets
      • docBase

        private int docBase
      • numBufferedDocs

        private int numBufferedDocs
      • numChunks

        private long numChunks
      • numDirtyChunks

        private long numDirtyChunks
      • numDirtyDocs

        private long numDirtyDocs
      • numStoredFieldsInDoc

        private int numStoredFieldsInDoc
      • NEGATIVE_ZERO_FLOAT

        static final int NEGATIVE_ZERO_FLOAT
      • NEGATIVE_ZERO_DOUBLE

        static final long NEGATIVE_ZERO_DOUBLE
      • BULK_MERGE_ENABLED_SYSPROP

        static final java.lang.String BULK_MERGE_ENABLED_SYSPROP
      • BULK_MERGE_ENABLED

        static final boolean BULK_MERGE_ENABLED
    • Constructor Detail

      • Lucene90CompressingStoredFieldsWriter

        Lucene90CompressingStoredFieldsWriter​(Directory directory,
                                              SegmentInfo si,
                                              java.lang.String segmentSuffix,
                                              IOContext context,
                                              java.lang.String formatName,
                                              CompressionMode compressionMode,
                                              int chunkSize,
                                              int maxDocsPerChunk,
                                              int blockShift)
                                       throws java.io.IOException
        Sole constructor.
        Throws:
        java.io.IOException
    • Method Detail

      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • startDocument

        public void startDocument()
                           throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called before writing the stored fields of the document. writeField will be called for each stored field. Note that this is called even if the document has no stored fields.
        Specified by:
        startDocument in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • finishDocument

        public void finishDocument()
                            throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called when a document and all its fields have been added.
        Overrides:
        finishDocument in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • saveInts

        private static void saveInts​(int[] values,
                                     int length,
                                     DataOutput out)
                              throws java.io.IOException
        Throws:
        java.io.IOException
      • writeHeader

        private void writeHeader​(int docBase,
                                 int numBufferedDocs,
                                 int[] numStoredFields,
                                 int[] lengths,
                                 boolean sliced,
                                 boolean dirtyChunk)
                          throws java.io.IOException
        Throws:
        java.io.IOException
      • triggerFlush

        private boolean triggerFlush()
      • flush

        private void flush​(boolean force)
                    throws java.io.IOException
        Throws:
        java.io.IOException
      • writeZFloat

        static void writeZFloat​(DataOutput out,
                                float f)
                         throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 3 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • writeZDouble

        static void writeZDouble​(DataOutput out,
                                 double d)
                          throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 8 bytes. When it is equal to 0xFE then the value is stored as a float in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 7 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • writeTLong

        static void writeTLong​(DataOutput out,
                               long l)
                        throws java.io.IOException
        Writes a long in a variable-length format. Writes between one and ten bytes. Small values or values representing timestamps with day, hour or second precision typically require fewer bytes.

        ZLong --> Header, Bytes*?

        • Header --> The first two bits indicate the compression scheme:
          • 00 - uncompressed
          • 01 - multiple of 1000 (second)
          • 10 - multiple of 3600000 (hour)
          • 11 - multiple of 86400000 (day)
          Then the next bit is a continuation bit, indicating whether more bytes need to be read, and the last 5 bits are the lower bits of the encoded value. In order to reconstruct the value, you need to combine the 5 lower bits of the header with a vLong in the next bytes (if the continuation bit is set to 1). Then zigzag-decode it and finally multiply by the multiple corresponding to the compression scheme.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • finish

        public void finish​(int numDocs)
                    throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called before StoredFieldsWriter.close(), passing in the number of documents that were written. Note that this is intentionally redundant (equivalent to the number of calls to StoredFieldsWriter.startDocument(), but a Codec should check that this is the case to detect the JRE bug described in LUCENE-1282.
        Specified by:
        finish in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • tooDirty

        boolean tooDirty​(Lucene90CompressingStoredFieldsReader candidate)
        Returns true if we should recompress this reader, even though we could bulk merge compressed data

        The last chunk written for a segment is typically incomplete, so without recompressing, in some worst-case situations (e.g. frequent reopen with tiny flushes), over time the compression ratio can degrade. This is a safety switch.

      • ramBytesUsed

        public long ramBytesUsed()
        Description copied from interface: Accountable
        Return the memory usage of this object in bytes. Negative values are illegal.