org.apache.lucene.analysis.ngram
Class EdgeNGramTokenizer
public class EdgeNGramTokenizer
Tokenizes the input into n-grams of the given size.
Token | next() - Returns the next token in the stream, or null at EOS.
|
EdgeNGramTokenizer
public EdgeNGramTokenizer(Reader input,
String side,
int gramSize)
EdgeNGramTokenizer
public EdgeNGramTokenizer(Reader input,
EdgeNGramTokenizer.Side side,
int gramSize)
Creates EdgeNGramTokenizer that can generate an n-gram of the given size.
input
- Reader holding the input to be tokenizedside
- the EdgeNGramTokenizer.Side
from which to chop off an n-gramgramSize
- the size of the n-gram to generate
next
public final Token next()
throws IOException
Returns the next token in the stream, or null at EOS.
- next in interface TokenStream
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.