Prev Class | Next Class | Frames | No Frames |
Summary: Nested | Field | Method | Constr | Detail: Nested | Field | Method | Constr |
java.lang.Object
com.ibm.icu.text.Transliterator
public abstract class Transliterator
extends Object
Transliterator
is an abstract class that
transliterates text from one format to another. The most common
kind of transliterator is a script, or alphabet, transliterator.
For example, a Russian to Latin transliterator changes Russian text
written in Cyrillic characters to phonetically equivalent Latin
characters. It does not translate Russian to English!
Transliteration, unlike translation, operates on characters, without
reference to the meanings of words and sentences.
Although script conversion is its most common use, a
transliterator can actually perform a more general class of tasks.
In fact, Transliterator
defines a very general API
which specifies only that a segment of the input text is replaced
by new text. The particulars of this conversion are determined
entirely by subclasses of Transliterator
.
Transliterators are stateless
Transliterator
objects are stateless; they
retain no information between calls to
transliterate()
. As a result, threads may share
transliterators without synchronizing them. This might seem to
limit the complexity of the transliteration operation. In
practice, subclasses perform complex transliterations by delaying
the replacement of text until it is known that no other
replacements are possible. In other words, although the
Transliterator
objects are stateless, the source text
itself embodies all the needed information, and delayed operation
allows arbitrary complexity.
Batch transliteration
The simplest way to perform transliteration is all at once, on a
string of existing text. This is referred to as batch
transliteration. For example, given a string input
and a transliterator t
, the call
String result = t.transliterate(input);
will transliterate it and return the result. Other methods allow
the client to specify a substring to be transliterated and to use
Replaceable
objects instead of strings, in order to
preserve out-of-band information (such as text styles).
Keyboard transliteration
Somewhat more involved is keyboard, or incremental
transliteration. This is the transliteration of text that is
arriving from some source (typically the user's keyboard) one
character at a time, or in some other piecemeal fashion.
In keyboard transliteration, a Replaceable
buffer
stores the text. As text is inserted, as much as possible is
transliterated on the fly. This means a GUI that displays the
contents of the buffer may show text being modified as each new
character arrives.
Consider the simple RuleBasedTransliterator
:
th>{theta}
t>{tau}
When the user types 't', nothing will happen, since the
transliterator is waiting to see if the next character is 'h'. To
remedy this, we introduce the notion of a cursor, marked by a '|'
in the output string:
t>|{tau}
{tau}h>{theta}
Now when the user types 't', tau appears, and if the next character
is 'h', the tau changes to a theta. This is accomplished by
maintaining a cursor position (independent of the insertion point,
and invisible in the GUI) across calls to
transliterate()
. Typically, the cursor will
be coincident with the insertion point, but in a case like the one
above, it will precede the insertion point.
Keyboard transliteration methods maintain a set of three indices
that are updated with each call to
transliterate()
, including the cursor, start,
and limit. These indices are changed by the method, and they are
passed in and out via a Position object. The start
index
marks the beginning of the substring that the transliterator will
look at. It is advanced as text becomes committed (but it is not
the committed index; that's the cursor
). The
cursor
index, described above, marks the point at
which the transliterator last stopped, either because it reached
the end, or because it required more characters to disambiguate
between possible inputs. The cursor
can also be
explicitly set by rules in a RuleBasedTransliterator
.
Any characters before the cursor
index are frozen;
future keyboard transliteration calls within this input sequence
will not change them. New text is inserted at the
limit
index, which marks the end of the substring that
the transliterator looks at.
Because keyboard transliteration assumes that more characters
are to arrive, it is conservative in its operation. It only
transliterates when it can do so unambiguously. Otherwise it waits
for more characters to arrive. When the client code knows that no
more characters are forthcoming, perhaps because the user has
performed some input termination operation, then it should call
finishTransliteration()
to complete any
pending transliterations.
Inverses
Pairs of transliterators may be inverses of one another. For
example, if transliterator A transliterates characters by
incrementing their Unicode value (so "abc" -> "def"), and
transliterator B decrements character values, then A
is an inverse of B and vice versa. If we compose A
with B in a compound transliterator, the result is the
indentity transliterator, that is, a transliterator that does not
change its input text.
The Transliterator
method getInverse()
returns a transliterator's inverse, if one exists, or
null
otherwise. However, the result of
getInverse()
usually will not be a true
mathematical inverse. This is because true inverse transliterators
are difficult to formulate. For example, consider two
transliterators: AB, which transliterates the character 'A'
to 'B', and BA, which transliterates 'B' to 'A'. It might
seem that these are exact inverses, since
"A" x AB -> "B"where 'x' represents transliteration. However,
"B" x BA -> "A"
"ABCD" x AB -> "BBCD"so AB composed with BA is not the identity. Nonetheless, BA may be usefully considered to be AB's inverse, and it is on this basis that AB
"BBCD" x BA -> "AACD"
.getInverse()
could legitimately return
BA.
IDs and display names
A transliterator is designated by a short identifier string or
ID. IDs follow the format source-destination,
where source describes the entity being replaced, and
destination describes the entity replacing
source. The entities may be the names of scripts,
particular sequences of characters, or whatever else it is that the
transliterator converts to or from. For example, a transliterator
from Russian to Latin might be named "Russian-Latin". A
transliterator from keyboard escape sequences to Latin-1 characters
might be named "KeyboardEscape-Latin1". By convention, system
entity names are in English, with the initial letters of words
capitalized; user entity names may follow any format so long as
they do not contain dashes.
In addition to programmatic IDs, transliterator objects have
display names for presentation in user interfaces, returned by
getDisplayName
.
Factory methods and registration
In general, client code should use the factory method
getInstance()
to obtain an instance of a
transliterator given its ID. Valid IDs may be enumerated using
getAvailableIDs()
. Since transliterators are
stateless, multiple calls to getInstance()
with the
same ID will return the same object.
In addition to the system transliterators registered at startup,
user transliterators may be registered by calling
registerInstance()
at run time. To register a
transliterator subclass without instantiating it (until it is
needed), users may call registerClass()
.
Composed transliterators
In addition to built-in system transliterators like
"Latin-Greek", there are also built-in composed
transliterators. These are implemented by composing two or more
component transliterators. For example, if we have scripts "A",
"B", "C", and "D", and we want to transliterate between all pairs
of them, then we need to write 12 transliterators: "A-B", "A-C",
"A-D", "B-A",..., "D-A", "D-B", "D-C". If it is possible to
convert all scripts to an intermediate script "M", then instead of
writing 12 rule sets, we only need to write 8: "A~M", "B~M", "C~M",
"D~M", "M~A", "M~B", "M~C", "M~D". (This might not seem like a big
win, but it's really 2n vs. n2 -
n, so as n gets larger the gain becomes
significant. With 9 scripts, it's 18 vs. 72 rule sets, a big
difference.) Note the use of "~" rather than "-" for the script
separator here; this indicates that the given transliterator is
intended to be composed with others, rather than be used as is.
Composed transliterators can be instantiated as usual. For
example, the system transliterator "Devanagari-Gujarati" is a
composed transliterator built internally as
"Devanagari~InterIndic;InterIndic~Gujarati". When this
transliterator is instantiated, it appears externally to be a
standard transliterator (e.g., getID() returns
"Devanagari-Gujarati").
Subclassing
Subclasses must implement the abstract method
handleTransliterate()
. Subclasses should override
the transliterate()
method taking a
Replaceable
and the transliterate()
method taking a String
and StringBuffer
if the performance of these methods can be improved over the
performance obtained by the default implementations in this class.
Copyright © IBM Corporation 1999. All rights reserved.
Nested Class Summary | |
static interface |
|
static class |
|
Field Summary | |
static int |
|
static int |
|
Constructor Summary | |
|
Method Summary | |
protected String |
|
static Transliterator |
|
void |
|
void |
|
static Enumeration |
|
static Enumeration |
|
static Enumeration |
|
static Enumeration |
|
static String |
|
static String |
|
static String |
|
Transliterator[] |
|
UnicodeFilter |
|
String |
|
static Transliterator |
|
static Transliterator |
|
Transliterator |
|
int |
|
UnicodeSet |
|
UnicodeSet |
|
protected UnicodeSet |
|
protected abstract void |
|
static void |
|
static void |
|
static void |
|
static void |
|
void |
|
protected void |
|
protected void |
|
String |
|
String |
|
void |
|
void |
|
void |
|
void |
|
int |
|
static void |
|
public static final int FORWARD
Direction constant indicating the forward direction in a transliterator, e.g., the forward rules of a RuleBasedTransliterator. An "A-B" transliterator transliterates A to B when operating in the forward direction, and B to A when operating in the reverse direction.
- Field Value:
- 0
public static final int REVERSE
Direction constant indicating the reverse direction in a transliterator, e.g., the reverse rules of a RuleBasedTransliterator. An "A-B" transliterator transliterates A to B when operating in the forward direction, and B to A when operating in the reverse direction.
- Field Value:
- 1
protected Transliterator(String ID, UnicodeFilter filter)
Default constructor.
- Parameters:
ID
- the string identifier for this transliteratorfilter
- the filter. Any character for which filter.contains() returns false will not be altered by this transliterator. If filter is null then no filtering is applied.
protected final String baseToRules(boolean escapeUnprintable)
Returns a rule string for this transliterator. This is a non-overrideable base class implementation that subclasses may call. It simply munges the ID into the correct format, that is, "foo" => "::foo".
- Parameters:
escapeUnprintable
- if true, then unprintable characters will be converted to escape form backslash-'u' or backslash-'U'.
public static final Transliterator createFromRules(String ID, String rules, int dir)
Returns aTransliterator
object constructed from the given rule string. This will be a RuleBasedTransliterator, if the rule string contains only rules, or a CompoundTransliterator, if it contains ID blocks, or a NullTransliterator, if it contains ID blocks which parse as empty for the given direction.
public void filteredTransliterate(Replaceable text, Transliterator.Position index, boolean incremental)
Transliterate a substring of text, as specified by index, taking filters into account. This method is for subclasses that need to delegate to another transliterator, such as CompoundTransliterator.
- Parameters:
text
- the text to be transliteratedindex
- the position indicesincremental
- if TRUE, then assume more characters may be inserted at index.limit, and postpone processing to accomodate future incoming characters
public final void finishTransliteration(Replaceable text, Transliterator.Position index)
Finishes any pending transliterations that were waiting for more characters. Clients should call this method as the last call after a sequence of one or more calls totransliterate()
.
- Parameters:
text
- the buffer holding transliterated and untransliterated text.index
- the array of indices previously passed totransliterate
public static final Enumeration getAvailableIDs()
Returns an enumeration over the programmatic names of registeredTransliterator
objects. This includes both system transliterators and user transliterators registered usingregisterClass()
. The enumerated names may be passed togetInstance()
.
- Returns:
- An
Enumeration
overString
objects
- See Also:
getInstance
,registerClass(String,Class,String)
public static final Enumeration getAvailableSources()
Returns an enumeration over the source names of registered transliterators. Source names may be passed to getAvailableTargets() to obtain available targets for each source.
public static final Enumeration getAvailableTargets(String source)
Returns an enumeration over the target names of registered transliterators having a given source name. Target names may be passed to getAvailableVariants() to obtain available variants for each source and target pair.
public static final Enumeration getAvailableVariants(String source, String target)
Returns an enumeration over the variant names of registered transliterators having a given source name and target name.
public static final String getDisplayName(String ID)
Returns a name for this transliterator that is appropriate for display to the user in the default locale. SeegetDisplayName(String,Locale)
for details.
public static String getDisplayName(String id, Locale inLocale)
Returns a name for this transliterator that is appropriate for display to the user in the given locale. This name is taken from the locale resource data in the standard manner of thejava.text
package. If no localized names exist in the system resource bundles, a name is synthesized using a localizedMessageFormat
pattern from the resource data. The arguments to this pattern are an integer followed by one or two strings. The integer is the number of strings, either 1 or 2. The strings are formed by splitting the ID for this transliterator at the first '-'. If there is no '-', then the entire ID forms the only string.
- Parameters:
inLocale
- the Locale in which the display name should be localized.
- See Also:
MessageFormat
public static String getDisplayName(String id, ULocale inLocale)
Returns a name for this transliterator that is appropriate for display to the user in the given locale. This name is taken from the locale resource data in the standard manner of thejava.text
package. If no localized names exist in the system resource bundles, a name is synthesized using a localizedMessageFormat
pattern from the resource data. The arguments to this pattern are an integer followed by one or two strings. The integer is the number of strings, either 1 or 2. The strings are formed by splitting the ID for this transliterator at the first '-'. If there is no '-', then the entire ID forms the only string.
- Parameters:
inLocale
- the ULocale in which the display name should be localized.
- See Also:
MessageFormat
public Transliterator[] getElements()
Return the elements that make up this transliterator. For example, if the transliterator "NFD;Jamo-Latin;Latin-Greek" were created, the return value of this method would be an array of the three transliterator objects that make up that transliterator: [NFD, Jamo-Latin, Latin-Greek]. If this transliterator is not composed of other transliterators, then this method will return an array of length one containing a reference to this transliterator.
- Returns:
- an array of one or more transliterators that make up this transliterator
public final UnicodeFilter getFilter()
Returns the filter used by this transliterator, or null if this transliterator uses no filter.
public final String getID()
Returns a programmatic identifier for this transliterator. If this identifier is passed togetInstance()
, it will return this object, if it has been registered.
public static final Transliterator getInstance(String ID)
Returns aTransliterator
object given its ID. The ID must be either a system transliterator ID or a ID registered usingregisterClass()
.
- Parameters:
ID
- a valid ID, as enumerated bygetAvailableIDs()
- Returns:
- A
Transliterator
object with the given ID
public static Transliterator getInstance(String ID, int dir)
Returns aTransliterator
object given its ID. The ID must be either a system transliterator ID or a ID registered usingregisterClass()
.
- Parameters:
ID
- a valid ID, as enumerated bygetAvailableIDs()
dir
- either FORWARD or REVERSE. If REVERSE then the inverse of the given ID is instantiated.
- Returns:
- A
Transliterator
object with the given ID
public final Transliterator getInverse()
Returns this transliterator's inverse. See the class documentation for details. This implementation simply inverts the two entities in the ID and attempts to retrieve the resulting transliterator. That is, ifgetID()
returns "A-B", then this method will return the result ofgetInstance("B-A")
, ornull
if that call fails. Subclasses with knowledge of their inverse may wish to override this method.
- Returns:
- a transliterator that is an inverse, not necessarily exact, of this transliterator, or
null
if no such transliterator is registered.
- See Also:
registerClass(String,Class,String)
public final int getMaximumContextLength()
Returns the length of the longest context required by this transliterator. This is preceding context. The default value is zero, but subclasses can change this by callingsetMaximumContextLength()
. For example, if a transliterator translates "ddd" (where d is any digit) to "555" when preceded by "(ddd)", then the preceding context length is 5, the length of "(ddd)".
- Returns:
- The maximum number of preceding context characters this transliterator needs to examine
public final UnicodeSet getSourceSet()
Returns the set of all characters that may be modified in the input text by this Transliterator. This incorporates this object's current filter; if the filter is changed, the return value of this function will change. The default implementation returns an empty set. Some subclasses may overridehandleGetSourceSet()
to return a more precise result. The return result is approximate in any case and is intended for use by tests, tools, or utilities.
- See Also:
getTargetSet()
,handleGetSourceSet()
public UnicodeSet getTargetSet()
Returns the set of all characters that may be generated as replacement text by this transliterator. The default implementation returns the empty set. Some subclasses may override this method to return a more precise result. The return result is approximate in any case and is intended for use by tests, tools, or utilities requiring such meta-information.
- See Also:
getTargetSet()
protected UnicodeSet handleGetSourceSet()
Framework method that returns the set of all characters that may be modified in the input text by this Transliterator, ignoring the effect of this object's filter. The base class implementation returns the empty set. Subclasses that wish to implement this should override this method.
- Returns:
- the set of characters that this transliterator may modify. The set may be modified, so subclasses should return a newly-created object.
- See Also:
getSourceSet()
,getTargetSet()
protected abstract void handleTransliterate(Replaceable text, Transliterator.Position pos, boolean incremental)
Abstract method that concrete subclasses define to implement their transliteration algorithm. This method handles both incremental and non-incremental transliteration. LetoriginalStart
refer to the value ofpos.start
upon entry.Implementations of this method should also obey the following invariants:
- If
incremental
is false, then this method should transliterate all characters betweenpos.start
andpos.limit
. Upon returnpos.start
must ==pos.limit
.- If
incremental
is true, then this method should transliterate all characters betweenpos.start
andpos.limit
that can be unambiguously transliterated, regardless of future insertions of text atpos.limit
. Upon return,pos.start
should be in the range [originalStart
,pos.limit
).pos.start
should be positioned such that characters [originalStart
,pos.start
) will not be changed in the future by this transliterator and characters [pos.start
,pos.limit
) are unchanged.Subclasses may safely assume that all characters in [
pos.limit
andpos.contextLimit
should be updated to reflect changes in length of the text betweenpos.start
andpos.limit
. The differencepos.contextLimit - pos.limit
should not change.pos.contextStart
should not change.- Upon return, neither
pos.start
norpos.limit
should be less thanoriginalStart
.- Text before
originalStart
and text afterpos.limit
should not change.- Text before
pos.contextStart
and text afterpos.contextLimit
should be ignored.pos.start
,pos.limit
) are filtered. In other words, the filter has already been applied by the time this method is called. SeefilteredTransliterate()
. This method is not for public consumption. Calling this method directly will transliterate [pos.start
,pos.limit
) without applying the filter. End user code should calltransliterate()
instead of this method. Subclass code should callfilteredTransliterate()
instead of this method.
- Parameters:
text
- the buffer holding transliterated and untransliterated textpos
- the indices indicating the start, limit, context start, and context limit of the text.incremental
- if true, assume more text may be inserted atpos.limit
and act accordingly. Otherwise, transliterate all text betweenpos.start
andpos.limit
and movepos.start
up topos.limit
.
- See Also:
transliterate
public static void registerAlias(String aliasID, String realID)
Register an ID as an alias of another ID. Instantiating alias ID produces the same result as instantiating the original ID. This is generally used to create short aliases of compound IDs.
- Parameters:
aliasID
- The new ID being registered.realID
- The existing ID that the new ID should be an alias of.
public static void registerClass(String ID, Class transClass, String displayName)
Registers a subclass ofTransliterator
with the system. This subclass must have a public constructor taking no arguments. When that constructor is called, the resulting object must return theID
passed to this method if itsgetID()
method is called.
- Parameters:
ID
- the result ofgetID()
for this transliteratortransClass
- a subclass ofTransliterator
- See Also:
unregister(String)
public static void registerFactory(String ID, Transliterator.Factory factory)
Register a factory object with the given ID. The factory method should return a new instance of the given transliterator.
- Parameters:
ID
- the ID of this transliteratorfactory
- the factory object
public static void registerInstance(Transliterator trans)
Register a Transliterator object with the given ID.
- Parameters:
trans
- the Transliterator object
public void setFilter(UnicodeFilter filter)
Changes the filter used by this transliterator. If the filter is set to null then no filtering will occur. Callers must take care if a transliterator is in use by multiple threads. The filter should not be changed by one thread while another thread may be transliterating.
protected final void setID(String id)
Set the programmatic identifier for this transliterator. Only for use by subclasses.
protected void setMaximumContextLength(int a)
Method for subclasses to use to set the maximum context length.
- See Also:
getMaximumContextLength()
public String toRules(boolean escapeUnprintable)
Returns a rule string for this transliterator.
- Parameters:
escapeUnprintable
- if true, then unprintable characters will be converted to escape form backslash-'u' or backslash-'U'.
public final String transliterate(String text)
Transliterate an entire string and returns the result. Convenience method.
- Parameters:
text
- the string to be transliterated
- Returns:
- The transliterated text
public final void transliterate(Replaceable text)
Transliterates an entire string in place. Convenience method.
- Parameters:
text
- the string to be transliterated
public final void transliterate(Replaceable text, Transliterator.Position index)
Transliterates the portion of the text buffer that can be transliterated unambiguosly. This is a convenience method; seetransliterate(Replaceable,Transliterator.Position,String)
for details.
- Parameters:
text
- the buffer holding transliterated and untransliterated textindex
- the start and limit of the text, the position of the cursor, and the start and limit of transliteration.
public final void transliterate(Replaceable text, Transliterator.Position index, String insertion)
Transliterates the portion of the text buffer that can be transliterated unambiguosly after new text has been inserted, typically as a result of a keyboard event. The new text ininsertion
will be inserted intotext
atindex.contextLimit
, advancingindex.contextLimit
byinsertion.length()
. Then the transliterator will try to transliterate characters oftext
betweenindex.start
andindex.contextLimit
. Characters beforeindex.start
will not be changed. Upon return, values inindex
will be updated.index.contextStart
will be advanced to the first character that future calls to this method will read.index.start
andindex.contextLimit
will be adjusted to delimit the range of text that future calls to this method may change. Typical usage of this method begins with an initial call withindex.contextStart
andindex.contextLimit
set to indicate the portion oftext
to be transliterated, andindex.start == index.contextStart
. Thereafter,index
can be used without modification in future calls, provided that all changes totext
are made via this method. This method assumes that future calls may be made that will insert new text into the buffer. As a result, it only performs unambiguous transliterations. After the last call to this method, there may be untransliterated text that is waiting for more input to resolve an ambiguity. In order to perform these pending transliterations, clients should callfinishTransliteration(Replaceable,Transliterator.Position)
after the last call to this method has been made.
- Parameters:
text
- the buffer holding transliterated and untransliterated textindex
- the start and limit of the text, the position of the cursor, and the start and limit of transliteration.insertion
- text to be inserted and possibly transliterated into the translation buffer atindex.contextLimit
. Ifnull
then no text is inserted.
public final void transliterate(Replaceable text, Transliterator.Position index, int insertion)
Transliterates the portion of the text buffer that can be transliterated unambiguosly after a new character has been inserted, typically as a result of a keyboard event. This is a convenience method; seetransliterate(Replaceable,Transliterator.Position,String)
for details.
- Parameters:
text
- the buffer holding transliterated and untransliterated textindex
- the start and limit of the text, the position of the cursor, and the start and limit of transliteration.insertion
- text to be inserted and possibly transliterated into the translation buffer atindex.contextLimit
.
public final int transliterate(Replaceable text, int start, int limit)
Transliterates a segment of a string, with optional filtering.
- Parameters:
text
- the string to be transliteratedstart
- the beginning index, inclusive;0 <= start <= limit
.limit
- the ending index, exclusive;start <= limit <= text.length()
.
- Returns:
- The new limit index. The text previously occupying
[start, limit)
has been transliterated, possibly to a string of a different length, at[start,
new-limit)
, where new-limit is the return value. If the input offsets are out of bounds, the returned value is -1 and the input string remains unchanged.
public static void unregister(String ID)
Unregisters a transliterator or class. This may be either a system transliterator or a user transliterator or class.
- Parameters:
ID
- the ID of the transliterator or class
- See Also:
registerClass(String,Class,String)