Class CountValueContainer

java.lang.Object
edu.berkeley.nlp.lm.values.CountValueContainer
All Implemented Interfaces:
CompressibleValueContainer<LongRef>, ValueContainer<LongRef>, Serializable

public final class CountValueContainer extends Object
See Also:
  • Field Details

    • valueRanks

      protected final CustomWidthArray[] valueRanks
    • storeSuffixIndexes

      protected final boolean storeSuffixIndexes
    • valueCoder

      protected final VariableLengthBitCompressor valueCoder
    • valueRadix

      protected final int valueRadix
    • valueWidth

      protected int valueWidth
    • defaultValRank

      protected final int defaultValRank
      See Also:
    • numNgramsForEachOrder

      protected final long[] numNgramsForEachOrder
    • suffixBitsForOrder

      protected final int[] suffixBitsForOrder
    • useMapValueArray

      protected boolean useMapValueArray
  • Constructor Details

    • CountValueContainer

      public CountValueContainer(LongToIntHashMap countCounter, int valueRadix, boolean storePrefixes, long[] numNgramsForEachOrder)
  • Method Details

    • createFreshValues

      public CountValueContainer createFreshValues(long[] numNgramsForEachOrder_)
      Description copied from interface: ValueContainer
      Creates a fresh value container for copying purposes.
      Returns:
    • getFromOffset

      public void getFromOffset(long index, int ngramOrder, LongRef outputVal)
      Description copied from interface: ValueContainer
      Gets the value living at a particular offset.
      Parameters:
      index -
      ngramOrder -
    • getFromRank

      protected void getFromRank(long rank, LongRef outputVal)
    • getCount

      public final long getCount(int ngramOrder, long index)
    • getDefaultVal

      protected LongRef getDefaultVal()
    • trimAfterNgram

      public void trimAfterNgram(int ngramOrder, long size)
      Description copied from interface: ValueContainer
      Clear storage after an n-gram order is complete
      Specified by:
      trimAfterNgram in interface ValueContainer<LongRef>
      Parameters:
      ngramOrder -
      size -
    • getUnigramSum

      public long getUnigramSum()
    • getScratchValue

      public LongRef getScratchValue()
      Description copied from interface: ValueContainer
      Creates a fresh value of object (useful for passing as an output parameter)
      Returns:
    • setFromOtherValues

      public void setFromOtherValues(ValueContainer<LongRef> o)
      Description copied from interface: ValueContainer
      Destructively sets internal storage from another object.
      Specified by:
      setFromOtherValues in interface ValueContainer<LongRef>
      Parameters:
      o -
    • trim

      public void trim()
      Description copied from interface: ValueContainer
      Final clean up of storage.
      Specified by:
      trim in interface ValueContainer<LongRef>
    • getCountRank

      protected long getCountRank(long val)
    • setMap

      public void setMap(NgramMap<LongRef> map)
      Description copied from interface: ValueContainer
      Initializes a value container with the map that contains it
      Specified by:
      setMap in interface ValueContainer<V extends LongRepresentable<V>>
    • useValueStoringArray

      protected boolean useValueStoringArray()
    • swap

      public void swap(long a, long b, int ngramOrder)
      Description copied from interface: CompressibleValueContainer
      Swaps values at offsets a and b.
      Specified by:
      swap in interface CompressibleValueContainer<V extends LongRepresentable<V>>
      Parameters:
      a -
      b -
      ngramOrder -
    • add

      public boolean add(int[] ngram, int startPos, int endPos, int ngramOrder, long offset, long prefixOffset, int word, LongRef val_, long suffixOffset, boolean ngramIsNew)
      Description copied from interface: ValueContainer
      Adds a new value at the specified offset.
      Specified by:
      add in interface ValueContainer<V extends LongRepresentable<V>>
      Parameters:
      ngramOrder - As always, ngramOrder is 0-based (0=unigram)
      offset -
      prefixOffset -
      word -
      val_ -
      suffixOffset -
      Returns:
      Whether or not the add was successful
    • getCompressed

      public BitList getCompressed(long offset, int ngramOrder)
      Description copied from interface: CompressibleValueContainer
      Compresses the value at the given offset into a list of bits.
      Specified by:
      getCompressed in interface CompressibleValueContainer<V extends LongRepresentable<V>>
      Parameters:
      offset -
      ngramOrder -
      Returns:
    • decompress

      public void decompress(BitStream bits, int ngramOrder, boolean justConsume, LongRef outputVal)
      Description copied from interface: CompressibleValueContainer
      Reads and decompresses from the bit stream bits.
      Specified by:
      decompress in interface CompressibleValueContainer<V extends LongRepresentable<V>>
      Parameters:
      bits -
      ngramOrder -
      justConsume - If true, nothing is returned, and the function simply consumes the appropriate number of bits from the BitStream.
    • setSizeAtLeast

      public void setSizeAtLeast(long size, int ngramOrder)
      Description copied from interface: ValueContainer
      Sets internal storage for size for a particular n-gram order
      Specified by:
      setSizeAtLeast in interface ValueContainer<V extends LongRepresentable<V>>
      Parameters:
      size -
      ngramOrder -
    • getSuffixOffset

      public long getSuffixOffset(long index, int ngramOrder)
    • getSuffixOffsetFromInternalVal

      protected int getSuffixOffsetFromInternalVal(long internalVal)
      Parameters:
      internalVal -
      Returns:
    • getRank

      protected long getRank(int ngramOrder, long offset)
    • clearStorageAfterCompression

      public void clearStorageAfterCompression(int ngramOrder)
      Specified by:
      clearStorageAfterCompression in interface CompressibleValueContainer<V extends LongRepresentable<V>>
    • clearStorageForOrder

      public void clearStorageForOrder(int ngramOrder)
      Specified by:
      clearStorageForOrder in interface ValueContainer<V extends LongRepresentable<V>>
    • storeSuffixoffsets

      public boolean storeSuffixoffsets()
      Specified by:
      storeSuffixoffsets in interface ValueContainer<V extends LongRepresentable<V>>
    • numValueBits

      public int numValueBits(int ngramOrder)
      Specified by:
      numValueBits in interface ValueContainer<V extends LongRepresentable<V>>