- java.lang.Object
-
- org.apache.lucene.index.MergePolicy
-
- org.apache.lucene.index.FilterMergePolicy
-
- org.apache.lucene.misc.index.BPReorderingMergePolicy
-
- All Implemented Interfaces:
Unwrappable<MergePolicy>
public final class BPReorderingMergePolicy extends FilterMergePolicy
A merge policy that reorders merged segments according to aBPIndexReorderer
. When reordering doesn't have enough RAM, it simply skips reordering in order not to fail the merge. So make sure to give enough RAM to yourBPIndexReorderer
viaBPIndexReorderer.setRAMBudgetMB(double)
.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy
MergePolicy.MergeAbortedException, MergePolicy.MergeContext, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge, MergePolicy.OneMergeProgress
-
-
Field Summary
Fields Modifier and Type Field Description private int
minNaturalMergeNumDocs
private float
minNaturalMergeRatioFromBiggestSegment
(package private) static java.lang.String
REORDERED
Whether a segment has been reordered.private BPIndexReorderer
reorderer
-
Fields inherited from class org.apache.lucene.index.FilterMergePolicy
in
-
Fields inherited from class org.apache.lucene.index.MergePolicy
DEFAULT_MAX_CFS_SEGMENT_SIZE, DEFAULT_NO_CFS_RATIO, maxCFSSegmentSize, noCFSRatio
-
-
Constructor Summary
Constructors Constructor Description BPReorderingMergePolicy(MergePolicy in, BPIndexReorderer reorderer)
Sole constructor.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description MergePolicy.MergeSpecification
findForcedDeletesMerges(SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)
Determine what set of merge operations is necessary in order to expunge all deletes from the index.MergePolicy.MergeSpecification
findForcedMerges(SegmentInfos segmentInfos, int maxSegmentCount, java.util.Map<SegmentCommitInfo,java.lang.Boolean> segmentsToMerge, MergePolicy.MergeContext mergeContext)
Determine what set of merge operations is necessary in order to merge to<=
the specified segment count.MergePolicy.MergeSpecification
findFullFlushMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)
Identifies merges that we want to execute (synchronously) on commit.MergePolicy.MergeSpecification
findMerges(CodecReader... readers)
Define the set of merge operations to perform on provided codec readers inIndexWriter.addIndexes(CodecReader...)
.MergePolicy.MergeSpecification
findMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)
Determine what set of merge operations are now necessary on the index.private MergePolicy.MergeSpecification
maybeReorder(MergePolicy.MergeSpecification spec, boolean forced, SegmentInfos infos)
void
setMinNaturalMergeNumDocs(int minNaturalMergeNumDocs)
Set the minimum number of docs that a merge must have for the resulting segment to be reordered.void
setMinNaturalMergeRatioFromBiggestSegment(float minNaturalMergeRatioFromBiggestSegment)
Set the minimum number of docs that a merge must have for the resulting segment to be reordered, as a ratio of the total number of documents of the current biggest segment in the index.-
Methods inherited from class org.apache.lucene.index.FilterMergePolicy
getMaxCFSSegmentSizeMB, getNoCFSRatio, keepFullyDeletedSegment, maxFullFlushMergeSize, numDeletesToMerge, setMaxCFSSegmentSizeMB, setNoCFSRatio, size, toString, unwrap, useCompoundFile
-
Methods inherited from class org.apache.lucene.index.MergePolicy
assertDelCount, isMerged, message, segString, verbose
-
-
-
-
Field Detail
-
REORDERED
static final java.lang.String REORDERED
Whether a segment has been reordered.- See Also:
- Constant Field Values
-
reorderer
private final BPIndexReorderer reorderer
-
minNaturalMergeNumDocs
private int minNaturalMergeNumDocs
-
minNaturalMergeRatioFromBiggestSegment
private float minNaturalMergeRatioFromBiggestSegment
-
-
Constructor Detail
-
BPReorderingMergePolicy
public BPReorderingMergePolicy(MergePolicy in, BPIndexReorderer reorderer)
Sole constructor. It takes the merge policy that should be used to compute merges, and will then reorder doc IDs from all merges above the configured minimum doc count, as well as all forced merges.If you wish to only run reordering upon forced merges, pass
Integer.MAX_VALUE
as aminNaturalMergeNumDocs
. Otherwise a default value of2^18 = 262,144
is suggested. This should help retain merging optimizations on small merges while reordering the larger segments that are important for good search performance.- Parameters:
in
- the merge policy to use to compute mergesreorderer
- theBPIndexReorderer
to use to renumber doc IDs
-
-
Method Detail
-
setMinNaturalMergeNumDocs
public void setMinNaturalMergeNumDocs(int minNaturalMergeNumDocs)
Set the minimum number of docs that a merge must have for the resulting segment to be reordered.
-
setMinNaturalMergeRatioFromBiggestSegment
public void setMinNaturalMergeRatioFromBiggestSegment(float minNaturalMergeRatioFromBiggestSegment)
Set the minimum number of docs that a merge must have for the resulting segment to be reordered, as a ratio of the total number of documents of the current biggest segment in the index. This parameter helps only enable reordering on segments that are large enough that they will significantly contribute to overall search performance.
-
maybeReorder
private MergePolicy.MergeSpecification maybeReorder(MergePolicy.MergeSpecification spec, boolean forced, SegmentInfos infos)
-
findMerges
public MergePolicy.MergeSpecification findMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) throws java.io.IOException
Description copied from class:MergePolicy
Determine what set of merge operations are now necessary on the index.IndexWriter
calls this whenever there is a change to the segments. This call is always synchronized on theIndexWriter
instance so only one thread at a time will call this method.- Overrides:
findMerges
in classFilterMergePolicy
- Parameters:
mergeTrigger
- the event that triggered the mergesegmentInfos
- the total set of segments in the indexmergeContext
- the IndexWriter to find the merges on- Throws:
java.io.IOException
-
findForcedMerges
public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos segmentInfos, int maxSegmentCount, java.util.Map<SegmentCommitInfo,java.lang.Boolean> segmentsToMerge, MergePolicy.MergeContext mergeContext) throws java.io.IOException
Description copied from class:MergePolicy
Determine what set of merge operations is necessary in order to merge to<=
the specified segment count.IndexWriter
calls this when itsIndexWriter.forceMerge(int)
method is called. This call is always synchronized on theIndexWriter
instance so only one thread at a time will call this method.- Overrides:
findForcedMerges
in classFilterMergePolicy
- Parameters:
segmentInfos
- the total set of segments in the indexmaxSegmentCount
- requested maximum number of segments in the indexsegmentsToMerge
- contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos. If the value is True for a given SegmentInfo, that means this segment was an original segment present in the to-be-merged index; else, it was a segment produced by a cascaded merge.mergeContext
- the MergeContext to find the merges on- Throws:
java.io.IOException
-
findForcedDeletesMerges
public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) throws java.io.IOException
Description copied from class:MergePolicy
Determine what set of merge operations is necessary in order to expunge all deletes from the index.- Overrides:
findForcedDeletesMerges
in classFilterMergePolicy
- Parameters:
segmentInfos
- the total set of segments in the indexmergeContext
- the MergeContext to find the merges on- Throws:
java.io.IOException
-
findFullFlushMerges
public MergePolicy.MergeSpecification findFullFlushMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) throws java.io.IOException
Description copied from class:MergePolicy
Identifies merges that we want to execute (synchronously) on commit. By default, this will returnnatural merges
whose segments are all less than themax segment size for full flushes
.Any merges returned here will make
IndexWriter.commit()
,IndexWriter.prepareCommit()
orIndexWriter.getReader(boolean, boolean)
block until the merges complete or untilLiveIndexWriterConfig.getMaxFullFlushMergeWaitMillis()
has elapsed. This may be used to merge small segments that have just been flushed, reducing the number of segments in the point in time snapshot. If a merge does not complete in the allotted time, it will continue to execute, and eventually finish and apply to future point in time snapshot, but will not be reflected in the current one.If a
MergePolicy.OneMerge
in the returnedMergePolicy.MergeSpecification
includes a segment already included in a registered merge, thenIndexWriter.commit()
orIndexWriter.prepareCommit()
will throw aIllegalStateException
. UseMergePolicy.MergeContext.getMergingSegments()
to determine which segments are currently registered to merge.- Overrides:
findFullFlushMerges
in classFilterMergePolicy
- Parameters:
mergeTrigger
- the event that triggered the merge (COMMIT or GET_READER).segmentInfos
- the total set of segments in the index (while preparing the commit)mergeContext
- the MergeContext to find the merges on, which should be used to determine which segments are already in a registered merge (seeMergePolicy.MergeContext.getMergingSegments()
).- Throws:
java.io.IOException
-
findMerges
public MergePolicy.MergeSpecification findMerges(CodecReader... readers) throws java.io.IOException
Description copied from class:MergePolicy
Define the set of merge operations to perform on provided codec readers inIndexWriter.addIndexes(CodecReader...)
.The merge operation is required to convert provided readers into segments that can be added to the writer. This API can be overridden in custom merge policies to control the concurrency for addIndexes. Default implementation creates a single merge operation for all provided readers (lowest concurrency). Creating a merge for each reader, would provide the highest level of concurrency possible with the configured merge scheduler.
- Overrides:
findMerges
in classFilterMergePolicy
- Parameters:
readers
- CodecReader(s) to merge into the main index- Throws:
java.io.IOException
-
-