Java Doc for Transliterator.java in  » Internationalization-Localization » icu4j » com » ibm » icu » text » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Although script conversion is its most common use, a transliterator can actually perform a more general class of tasks. In fact, Transliterator defines a very general API which specifies only that a segment of the input text is replaced by new text. The particulars of this conversion are determined entirely by subclasses of Transliterator.

Transliterators are stateless

Transliterator objects are stateless; they retain no information between calls to transliterate(). As a result, threads may share transliterators without synchronizing them. This might seem to limit the complexity of the transliteration operation. In practice, subclasses perform complex transliterations by delaying the replacement of text until it is known that no other replacements are possible. In other words, although the Transliterator objects are stateless, the source text itself embodies all the needed information, and delayed operation allows arbitrary complexity.

Batch transliteration

The simplest way to perform transliteration is all at once, on a string of existing text. This is referred to as batch transliteration. For example, given a string input and a transliterator t, the call

String result = t.transliterate(input);

Keyboard transliteration

Somewhat more involved is keyboard, or incremental transliteration. This is the transliteration of text that is arriving from some source (typically the user's keyboard) one character at a time, or in some other piecemeal fashion.

In keyboard transliteration, a Replaceable buffer stores the text. As text is inserted, as much as possible is transliterated on the fly. This means a GUI that displays the contents of the buffer may show text being modified as each new character arrives.

Consider the simple RuleBasedTransliterator:

th>{theta}
t>{tau}

t>|{tau}
{tau}h>{theta}

Keyboard transliteration methods maintain a set of three indices that are updated with each call to transliterate(), including the cursor, start, and limit. These indices are changed by the method, and they are passed in and out via a Position object. The start index marks the beginning of the substring that the transliterator will look at. It is advanced as text becomes committed (but it is not the committed index; that's the cursor). The cursor index, described above, marks the point at which the transliterator last stopped, either because it reached the end, or because it required more characters to disambiguate between possible inputs. The cursor can also be explicitly set by rules in a RuleBasedTransliterator. Any characters before the cursor index are frozen; future keyboard transliteration calls within this input sequence will not change them. New text is inserted at the limit index, which marks the end of the substring that the transliterator looks at.

Because keyboard transliteration assumes that more characters are to arrive, it is conservative in its operation. It only transliterates when it can do so unambiguously. Otherwise it waits for more characters to arrive. When the client code knows that no more characters are forthcoming, perhaps because the user has performed some input termination operation, then it should call finishTransliteration() to complete any pending transliterations.

Inverses

Pairs of transliterators may be inverses of one another. For example, if transliterator A transliterates characters by incrementing their Unicode value (so "abc" -> "def"), and transliterator B decrements character values, then A is an inverse of B and vice versa. If we compose A with B in a compound transliterator, the result is the indentity transliterator, that is, a transliterator that does not change its input text. The Transliterator method getInverse() returns a transliterator's inverse, if one exists, or null otherwise. However, the result of getInverse() usually will not be a true mathematical inverse. This is because true inverse transliterators are difficult to formulate. For example, consider two transliterators: AB, which transliterates the character 'A' to 'B', and BA, which transliterates 'B' to 'A'. It might seem that these are exact inverses, since

"A" x AB -> "B"
"B" x BA -> "A"

"ABCD" x AB -> "BBCD"
"BBCD" x BA -> "AACD"

IDs and display names

A transliterator is designated by a short identifier string or ID. IDs follow the format source-destination, where source describes the entity being replaced, and destination describes the entity replacing source. The entities may be the names of scripts, particular sequences of characters, or whatever else it is that the transliterator converts to or from. For example, a transliterator from Russian to Latin might be named "Russian-Latin". A transliterator from keyboard escape sequences to Latin-1 characters might be named "KeyboardEscape-Latin1". By convention, system entity names are in English, with the initial letters of words capitalized; user entity names may follow any format so long as they do not contain dashes.

In addition to programmatic IDs, transliterator objects have display names for presentation in user interfaces, returned by Transliterator.getDisplayName .

Factory methods and registration

In general, client code should use the factory method getInstance() to obtain an instance of a transliterator given its ID. Valid IDs may be enumerated using getAvailableIDs(). Since transliterators are stateless, multiple calls to getInstance() with the same ID will return the same object.

In addition to the system transliterators registered at startup, user transliterators may be registered by calling registerInstance() at run time. To register a transliterator subclass without instantiating it (until it is needed), users may call registerClass().

Composed transliterators

In addition to built-in system transliterators like "Latin-Greek", there are also built-in composed transliterators. These are implemented by composing two or more component transliterators. For example, if we have scripts "A", "B", "C", and "D", and we want to transliterate between all pairs of them, then we need to write 12 transliterators: "A-B", "A-C", "A-D", "B-A",..., "D-A", "D-B", "D-C". If it is possible to convert all scripts to an intermediate script "M", then instead of writing 12 rule sets, we only need to write 8: "A~M", "B~M", "C~M", "D~M", "M~A", "M~B", "M~C", "M~D". (This might not seem like a big win, but it's really 2n vs. n² - n, so as n gets larger the gain becomes significant. With 9 scripts, it's 18 vs. 72 rule sets, a big difference.) Note the use of "~" rather than "-" for the script separator here; this indicates that the given transliterator is intended to be composed with others, rather than be used as is.

Composed transliterators can be instantiated as usual. For example, the system transliterator "Devanagari-Gujarati" is a composed transliterator built internally as "Devanagari~InterIndic;InterIndic~Gujarati". When this transliterator is instantiated, it appears externally to be a standard transliterator (e.g., getID() returns "Devanagari-Gujarati").

Subclassing

Subclasses must implement the abstract method handleTransliterate().

Subclasses should override the transliterate() method taking a Replaceable and the transliterate() method taking a String and StringBuffer if the performance of these methods can be improved over the performance obtained by the default implementations in this class.

Copyright © IBM Corporation 1999. All rights reserved.
author:
   Alan Liu

If no localized names exist in the system resource bundles, a name is synthesized using a localized MessageFormat pattern from the resource data. The arguments to this pattern are an integer followed by one or two strings. The integer is the number of strings, either 1 or 2. The strings are formed by splitting the ID for this transliterator at the first '-'. If there is no '-', then the entire ID forms the only string.
Parameters:
  inLocale - the Locale in which the display name should belocalized.
See Also:   java.text.MessageFormat

If no localized names exist in the system resource bundles, a name is synthesized using a localized MessageFormat pattern from the resource data. The arguments to this pattern are an integer followed by one or two strings. The integer is the number of strings, either 1 or 2. The strings are formed by splitting the ID for this transliterator at the first '-'. If there is no '-', then the entire ID forms the only string.
Parameters:
  inLocale - the ULocale in which the display name should belocalized.
See Also:   java.text.MessageFormat

If this transliterator is not composed of other transliterators, then this method will return an array of length one containing a reference to this transliterator. an array of one or more transliterators that make upthis transliterator

Subclasses with knowledge of their inverse may wish to override this method. a transliterator that is an inverse, not necessarilyexact, of this transliterator, or null if no suchtransliterator is registered.
See Also:   Transliterator.registerClass

• If incremental is false, then this method should transliterate all characters between pos.start and pos.limit. Upon return pos.start must == pos.limit.
• If incremental is true, then this method should transliterate all characters between pos.start and pos.limit that can be unambiguously transliterated, regardless of future insertions of text at pos.limit. Upon return, pos.start should be in the range [originalStart, pos.limit). pos.start should be positioned such that characters [originalStart, pos.start) will not be changed in the future by this transliterator and characters [pos.start, pos.limit) are unchanged.

Implementations of this method should also obey the following invariants:

• pos.limit and pos.contextLimit should be updated to reflect changes in length of the text between pos.start and pos.limit. The difference pos.contextLimit - pos.limit should not change.
• pos.contextStart should not change.
• Upon return, neither pos.start nor pos.limit should be less than originalStart.
• Text before originalStart and text after pos.limit should not change.
• Text before pos.contextStart and text after pos.contextLimit should be ignored.

Subclasses may safely assume that all characters in [pos.start, pos.limit) are filtered. In other words, the filter has already been applied by the time this method is called. See filteredTransliterate().

This method is not for public consumption. Calling this method directly will transliterate [pos.start, pos.limit) without applying the filter. End user code should call transliterate() instead of this method. Subclass code should call filteredTransliterate() instead of this method.

Parameters:
  text - the buffer holding transliterated anduntransliterated text
Parameters:
  pos - the indices indicating the start, limit, contextstart, and context limit of the text.
Parameters:
  incremental - if true, assume more text may be inserted atpos.limit and act accordingly. Otherwise,transliterate all text between pos.start andpos.limit and move pos.start up topos.limit.
See Also:   Transliterator.transliterate

NFC => NFD
 Any-NFC => Any-NFD
 NFD => NFC
 Any-NFD => Any-NFC

The relationship is symmetrical; registering (a, b) is equivalent to registering (b, a).

The relevant IDs must still be registered separately as factories or classes.

Only the targets are specified. Special inverses always have the form Any-Target1 <=> Any-Target2. The target should have canonical casing (the casing desired to be produced when an inverse is formed) and should contain no whitespace or other extraneous characters.
Parameters:
  target - the target against which to register the inverse
Parameters:
  inverseTarget - the inverse of target, that isAny-target.getInverse() => Any-inverseTarget
Parameters:
  bidirectional - if true, register the reverse relationas well, that is, Any-inverseTarget.getInverse() => Any-target

Callers must take care if a transliterator is in use by multiple threads. The filter should not be changed by one thread while another thread may be transliterating.

Upon return, values in index will be updated. index.contextStart will be advanced to the first character that future calls to this method will read. index.start and index.contextLimit will be adjusted to delimit the range of text that future calls to this method may change.

Typical usage of this method begins with an initial call with index.contextStart and index.contextLimit set to indicate the portion of text to be transliterated, and index.start == index.contextStart. Thereafter, index can be used without modification in future calls, provided that all changes to text are made via this method.

This method assumes that future calls may be made that will insert new text into the buffer. As a result, it only performs unambiguous transliterations. After the last call to this method, there may be untransliterated text that is waiting for more input to resolve an ambiguity. In order to perform these pending transliterations, clients should call Transliterator.finishTransliteration after the last call to this method has been made.
Parameters:
  text - the buffer holding transliterated and untransliterated text
Parameters:
  index - the start and limit of the text, the positionof the cursor, and the start and limit of transliteration.
Parameters:
  insertion - text to be inserted and possiblytransliterated into the translation buffer atindex.contextLimit. If null then no textis inserted.
See Also:   Transliterator.handleTransliterate
exception:
  IllegalArgumentException - if indexis invalid