org.freecompany.util.text

Class Tokenizer

public class Tokenizer extends Scanner

Provides methods to tokenize character sequence given either absolute indexes or sequence for which to scan. TODO: Make the whitespace boolean a ctor argument and refactor out a trim method.
Method Summary
static CharSequenceafter(TokenizerState state, CharSequence sequence)
Retrieves the remaining characters after the provided sequence, starting at the current position.
static CharSequenceafter(CharSequence target, CharSequence sequence)
Retrieves the remaining characters after the provided sequence.
static CharSequenceafter(CharSequence target, int index)
Retrieves the remaining characters after the provided index.
static CharSequenceafter(CharSequence target, CharSequence sequence, boolean trim)
Retrieves the remaining characters after the provided sequence, optionally trimming the results.
static CharSequenceafter(CharSequence target, int start, boolean trim)
Retrieves the remaining characters after the provided index, optionally trimming the results.
static CharSequencebefore(TokenizerState state, CharSequence sequence)
Retrieves the characters preceeding the provided sequence, starting at the current position.
static CharSequencebefore(CharSequence target, CharSequence sequence)
Retrieves the characters preceeding the provided sequence.
static CharSequencebefore(CharSequence target, CharSequence sequence, boolean trim)
Retrieves the characters preceeding the provided sequence, optionally trimming the resulting sequence.
static CharSequencebefore(CharSequence target, int start)
Convenience method for returning the untrimmed portion of a sequence up to the provided index.
static CharSequencebefore(CharSequence target, int start, boolean trim)
Retrieves the characters preceding the provided index, optionally trimming the results.
static CharSequencebetween(TokenizerState state, CharSequence one, CharSequence two)
Retrieves the next token that falls between the two provided sequences, starting at the current position.
static CharSequencebetween(CharSequence target, CharSequence one, CharSequence two)
Retrieves the next token that falls between the two provided sequences.
static CharSequencebetween(CharSequence target, CharSequence one, CharSequence two, boolean trim)
Retrieves the next token that falls between the two provided sequences, optionally trimming the results.
static CharSequencerest(TokenizerState state)
Retrieves the remaining characters from this tokenizer.
protected static CharSequencesub(CharSequence target, int start, int end, boolean trim)

Method Detail

after

public static final CharSequence after(TokenizerState state, CharSequence sequence)
Retrieves the remaining characters after the provided sequence, starting at the current position. The position is moved to the end of the target only if a match is found.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

after

public static final CharSequence after(CharSequence target, CharSequence sequence)
Retrieves the remaining characters after the provided sequence.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

after

public static final CharSequence after(CharSequence target, int index)
Retrieves the remaining characters after the provided index.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

after

public static final CharSequence after(CharSequence target, CharSequence sequence, boolean trim)
Retrieves the remaining characters after the provided sequence, optionally trimming the results.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

after

public static final CharSequence after(CharSequence target, int start, boolean trim)
Retrieves the remaining characters after the provided index, optionally trimming the results.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

before

public static final CharSequence before(TokenizerState state, CharSequence sequence)
Retrieves the characters preceeding the provided sequence, starting at the current position. The position is reset to the character following the match only if the match is found.

Returns: the characters preceeding the sequence, or null if the sequence is not found. Retrieves the next token delimited by the provided sequence.

before

public static final CharSequence before(CharSequence target, CharSequence sequence)
Retrieves the characters preceeding the provided sequence.

Returns: the characters preceeding the sequence, or null if the sequence is not found.

before

public static final CharSequence before(CharSequence target, CharSequence sequence, boolean trim)
Retrieves the characters preceeding the provided sequence, optionally trimming the resulting sequence.

Returns: the characters preceeding the sequence, or null if the sequence is not found.

before

public static final CharSequence before(CharSequence target, int start)
Convenience method for returning the untrimmed portion of a sequence up to the provided index. Equivalent to calling before with false as the last parameter.

before

public static final CharSequence before(CharSequence target, int start, boolean trim)
Retrieves the characters preceding the provided index, optionally trimming the results.

Returns: the characters remaining after the sequence, or null if the sequence is not found.

between

public static final CharSequence between(TokenizerState state, CharSequence one, CharSequence two)
Retrieves the next token that falls between the two provided sequences, starting at the current position. The position is moved to the character following the match only if the match is found.

Returns: the characters between the two sequences, or null if either sequence is not found.

between

public static final CharSequence between(CharSequence target, CharSequence one, CharSequence two)
Retrieves the next token that falls between the two provided sequences.

Returns: the characters between the two sequences, or null if either sequence is not found.

between

public static final CharSequence between(CharSequence target, CharSequence one, CharSequence two, boolean trim)
Retrieves the next token that falls between the two provided sequences, optionally trimming the results.

Returns: the characters between the two sequences, or null if either sequence is not found.

rest

public static final CharSequence rest(TokenizerState state)
Retrieves the remaining characters from this tokenizer.

sub

protected static final CharSequence sub(CharSequence target, int start, int end, boolean trim)