| java.lang.Object
 com.ibm.icu.text.Normalizer
| Normalizer | | final public class Normalizer implements Cloneable(Code) | | Unicode Normalization
Unicode normalization API
normalize transforms Unicode text into an equivalent composed or
decomposed form, allowing for easier sorting and searching of text.
normalize supports the standard normalization forms described in
Unicode Standard Annex #15 — Unicode Normalization Forms.
Characters with accents or other adornments can be encoded in
several different ways in Unicode. For example, take the character A-acute.
In Unicode, this can be encoded as a single character (the
"composed" form):
00C1 LATIN CAPITAL LETTER A WITH ACUTE
or as two separate characters (the "decomposed" form):
0041 LATIN CAPITAL LETTER A
0301 COMBINING ACUTE ACCENT
To a user of your program, however, both of these sequences should be
treated as the same "user-level" character "A with acute accent". When you
are searching or comparing text, you must ensure that these two sequences are
treated equivalently. In addition, you must handle characters with more than
one accent. Sometimes the order of a character's combining accents is
significant, while in other cases accent sequences in different orders are
really equivalent.
Similarly, the string "ffi" can be encoded as three separate letters:
0066 LATIN SMALL LETTER F
0066 LATIN SMALL LETTER F
0069 LATIN SMALL LETTER I
<\p>
or as the single character
FB03 LATIN SMALL LIGATURE FFI
<\p>
The ffi ligature is not a distinct semantic character, and strictly speaking
it shouldn't be in Unicode at all, but it was included for compatibility
with existing character sets that already provided it. The Unicode standard
identifies such characters by giving them "compatibility" decompositions
into the corresponding semantic characters. When sorting and searching, you
will often want to use these mappings.
normalize helps solve these problems by transforming text into
the canonical composed and decomposed forms as shown in the first example
above. In addition, you can have it perform compatibility decompositions so
that you can treat compatibility characters the same as their equivalents.
Finally, normalize rearranges accents into the proper canonical
order, so that you do not have to worry about accent rearrangement on your
own.
Form FCD, "Fast C or D", is also designed for collation.
It allows to work on strings that are not necessarily normalized
with an algorithm (like in collation) that works under "canonical closure",
i.e., it treats precomposed characters and their decomposed equivalents the
same.
It is not a normalization form because it does not provide for uniqueness of
representation. Multiple strings may be canonically equivalent (their NFDs
are identical) and may all conform to FCD without being identical themselves.
The form is defined such that the "raw decomposition", the recursive
canonical decomposition of each character, results in a string that is
canonically ordered. This means that precomposed characters are allowed for
as long as their decompositions do not need canonical reordering.
Its advantage for a process like collation is that all NFD and most NFC texts
- and many unnormalized texts - already conform to FCD and do not need to be
normalized (NFD) for such a process. The FCD quick check will return YES for
most strings in practice.
normalize(FCD) may be implemented with NFD.
For more details on FCD see the collation design document:
http://dev.icu-project.org/cgi-bin/viewcvs.cgi/~checkout~/icuhtml/design/collation/ICU_collation_design.htm
ICU collation performs either NFD or FCD normalization automatically if
normalization is turned on for the collator object. Beyond collation and
string search, normalized strings may be useful for string equivalence
comparisons, transliteration/transcription, unique representations, etc.
The W3C generally recommends to exchange texts in NFC.
Note also that most legacy character encodings use only precomposed forms and
often do not encode any combining marks by themselves. For conversion to such
character encodings the Unicode text needs to be normalized to NFC.
For more usage examples, see the Unicode Standard Annex.
|
| Inner Class :public static class Mode | |
| Inner Class :final public static class QuickCheckResult | |
| Field Summary | |
| final public static int | COMPARE_CODE_POINT_ORDER
Option bit for compare:
Compare strings in code point order instead of code unit order. | | final public static int | COMPARE_IGNORE_CASE
Option bit for compare:
Perform case-insensitive comparison. | | final public static int | COMPARE_NORM_OPTIONS_SHIFT
Lowest-order bit number of compare() options bits corresponding to
normalization options bits. | | final public static Mode | COMPOSE
Canonical decomposition followed by canonical composition. | | final public static Mode | COMPOSE_COMPAT
Compatibility decomposition followed by canonical composition. | | final public static Mode | DECOMP
Canonical decomposition. | | final public static Mode | DECOMP_COMPAT
Compatibility decomposition. | | final public static Mode | DEFAULT
Default normalization. | | final public static int | DONE
Constant indicating that the end of the iteration has been reached. | | final public static Mode | FCD
"Fast C or D" form. | | final public static int | FOLD_CASE_DEFAULT
| | final public static int | FOLD_CASE_EXCLUDE_SPECIAL_I
Option value for case folding: exclude the mappings for dotted I
and dotless i marked with 'I' in CaseFolding.txt. | | final public static int | IGNORE_HANGUL
Option to disable Hangul/Jamo composition and decomposition.
This option applies to Korean text,
which can be represented either in the Jamo alphabet or in Hangul
characters, which are really just two or three Jamo combined
into one visual glyph. | | final public static int | INPUT_IS_FCD
Option bit for compare:
Both input strings are assumed to fulfill FCD conditions. | | final public static QuickCheckResult | MAYBE
Indicates it cannot be determined if string is in the normalized
format without further thorough checks. | | final public static Mode | NFC
Canonical decomposition followed by canonical composition. | | final public static Mode | NFD
Canonical decomposition. | | final public static Mode | NFKC
Compatibility decomposition followed by canonical composition. | | final public static Mode | NFKD
Compatibility decomposition. | | final public static QuickCheckResult | NO
| | final public static Mode | NONE
No decomposition/composition. | | final public static Mode | NO_OP
Null operation for use with the
Normalizer.Normalizer constructors and the static
Normalizer.normalize normalize method. | | final public static int | UNICODE_3_2
Options bit set value to select Unicode 3.2 normalization
(except NormalizationCorrections). | | final public static QuickCheckResult | YES
|
| Constructor Summary | |
| public | Normalizer(String str, Mode mode, int opt)
Creates a new Normalizer object for iterating over the
normalized form of a given string.
The options parameter specifies which optional
Normalizer features are to be enabled for this object.
Parameters:
str - The string to be normalized. | | public | Normalizer(CharacterIterator iter, Mode mode, int opt)
Creates a new Normalizer object for iterating over the
normalized form of the given text.
Parameters:
iter - The input text to be normalized. | | public | Normalizer(UCharacterIterator iter, Mode mode, int options)
Creates a new Normalizer object for iterating over the
normalized form of the given text.
Parameters:
iter - The input text to be normalized. |
| Method Summary | |
| public Object | clone()
Clones this Normalizer object. | | public static int | compare(char[] s1, int s1Start, int s1Limit, char[] s2, int s2Start, int s2Limit, int options)
Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Canonical equivalence between two strings is defined as their normalized
forms (NFD or NFC) being identical.
This function compares strings incrementally instead of normalizing
(and optionally case-folding) both strings entirely,
improving performance significantly.
Bulk normalization is only necessary if the strings do not fulfill the
FCD conditions. | | public static int | compare(String s1, String s2, int options)
Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Convenience method.
Parameters:
s1 - First source string.
Parameters:
s2 - Second source string.
Parameters:
options - A bit set of options:- FOLD_CASE_DEFAULT or 0 is used for default options:Case-sensitive comparison in code unit order, and the input stringsare quick-checked for FCD.- INPUT_IS_FCDSet if the caller knows that both s1 and s2 fulfill the FCD conditions. | | public static int | compare(char[] s1, char[] s2, int options)
Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Convenience method.
Parameters:
s1 - First source string.
Parameters:
s2 - Second source string.
Parameters:
options - A bit set of options:- FOLD_CASE_DEFAULT or 0 is used for default options:Case-sensitive comparison in code unit order, and the input stringsare quick-checked for FCD.- INPUT_IS_FCDSet if the caller knows that both s1 and s2 fulfill the FCD conditions. | | public static int | compare(int char32a, int char32b, int options)
Convenience method that can have faster implementation
by not allocating buffers. | | public static int | compare(int char32a, String str2, int options)
Convenience method that can have faster implementation
by not allocating buffers. | | public static String | compose(String str, boolean compat)
Compose a string.
The string will be composed to according the the specified mode.
Parameters:
str - The string to compose.
Parameters:
compat - If true the string will be composed accoding to NFKC rules and if false will be composed according to NFC rules. | | public static String | compose(String str, boolean compat, int options)
Compose a string. | | public static int | compose(char[] source, char[] target, boolean compat, int options)
Compose a string.
The string will be composed to according the the specified mode.
Parameters:
source - The char array to compose.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
compat - If true the char array will be composed accoding to NFKC rules and if false will be composed according to NFC rules.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static int | compose(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, boolean compat, int options)
Compose a string.
The string will be composed to according the the specified mode.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
compat - If true the char array will be composed accoding to NFKC rules and if false will be composed according to NFC rules.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static int | concatenate(char[] left, int leftStart, int leftLimit, char[] right, int rightStart, int rightLimit, char[] dest, int destStart, int destLimit, Normalizer.Mode mode, int options)
Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
With the input strings already being normalized,
this function will use next() and previous()
to find the adjacent end pieces of the input strings.
Only the concatenation of these end pieces will be normalized and
then concatenated with the remaining parts of the input strings.
It is allowed to have dest==left to avoid copying the entire left string.
Parameters:
left - Left source array, may be same as dest.
Parameters:
leftStart - start in the left array.
Parameters:
leftLimit - limit in the left array (==length)
Parameters:
right - Right source array.
Parameters:
rightStart - start in the right array.
Parameters:
rightLimit - limit in the right array (==length)
Parameters:
dest - The output buffer; can be null if destStart==destLimit==0 for pure preflighting.
Parameters:
destStart - start in the destination array
Parameters:
destLimit - limit in the destination array (==length)
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static String | concatenate(char[] left, char[] right, Mode mode, int options)
Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
For details see concatenate
Parameters:
left - Left source string.
Parameters:
right - Right source string.
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static String | concatenate(String left, String right, Mode mode, int options)
Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
For details see concatenate
Parameters:
left - Left source string.
Parameters:
right - Right source string.
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public int | current()
| | public static String | decompose(String str, boolean compat)
Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
str - The string to decompose.
Parameters:
compat - If true the string will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules. | | public static String | decompose(String str, boolean compat, int options)
Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
str - The string to decompose.
Parameters:
compat - If true the string will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static int | decompose(char[] source, char[] target, boolean compat, int options)
Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
source - The char array to decompose.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
compat - If true the char array will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules. | | public static int | decompose(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, boolean compat, int options)
Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
compat - If true the char array will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules.
Parameters:
options - The normalization options, ORed together (0 for no options). | | public int | endIndex()
| | public int | first()
| | public int | getBeginIndex()
Retrieve the index of the start of the input text. | | public int | getEndIndex()
Retrieve the index of the end of the input text. | | public static int | getFC_NFKC_Closure(int c, char[] dest)
| | public static String | getFC_NFKC_Closure(int c)
| | public int | getIndex()
Retrieve the current iteration position in the input text that is
being normalized. | | public int | getLength()
| | public Mode | getMode()
| | public int | getOption(int option)
Determine whether an option is turned on or off. | | public int | getText(char[] fillIn)
| | public String | getText()
| | static VersionInfo | getUnicodeVersion()
| | public static boolean | isNFSkippable(int c, Mode mode)
| | public static boolean | isNormalized(char[] src, int start, int limit, Mode mode, int options)
Test if a string is in a given normalization form. | | public static boolean | isNormalized(String str, Mode mode, int options)
| | public static boolean | isNormalized(int char32, Mode mode, int options)
| | public int | last()
Return the last character in the normalized text-> This resets
the Normalizer's position to be just before the
the input text corresponding to that normalized character. | | public int | next()
Return the next character in the normalized text and advance
the iteration position by one. | | public static String | normalize(String str, Mode mode, int options)
Normalizes a String using the given normalization operation.
The options parameter specifies which optional
Normalizer features are to be enabled for this operation.
Currently the only available option is
Normalizer.UNICODE_3_2 .
If you want the default behavior corresponding to one of the standard
Unicode Normalization Forms, use 0 for this argument.
Parameters:
str - the input string to be normalized.
Parameters:
mode - the normalization mode
Parameters:
options - the optional features to be enabled. | | public static String | normalize(String src, Mode mode)
Normalize a string. | | public static int | normalize(char[] source, char[] target, Mode mode, int options)
Normalize a string.
The string will be normalized according the the specified normalization
mode and options.
Parameters:
source - The char array to normalize.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
mode - The normalization mode; one of Normalizer.NONE, Normalizer.NFD, Normalizer.NFC, Normalizer.NFKC, Normalizer.NFKD, Normalizer.DEFAULT
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static int | normalize(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, Mode mode, int options)
Normalize a string.
The string will be normalized according the the specified normalization
mode and options.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
mode - The normalization mode; one of Normalizer.NONE, Normalizer.NFD, Normalizer.NFC, Normalizer.NFKC, Normalizer.NFKD, Normalizer.DEFAULT
Parameters:
options - The normalization options, ORed together (0 for no options). | | public static String | normalize(int char32, Mode mode, int options)
| | public static String | normalize(int char32, Mode mode)
| | public int | previous()
Return the previous character in the normalized text and decrement
the iteration position by one. | | public static QuickCheckResult | quickCheck(String source, Mode mode)
Convenience method. | | public static QuickCheckResult | quickCheck(String source, Mode mode, int options)
Convenience method. | | public static QuickCheckResult | quickCheck(char[] source, Mode mode, int options)
Convenience method. | | public static QuickCheckResult | quickCheck(char[] source, int start, int limit, Mode mode, int options)
Performing quick check on a string, to quickly determine if the string is
in a particular normalization format.
Three types of result can be returned Normalizer.YES, Normalizer.NO or
Normalizer.MAYBE. | | public void | reset()
Reset the index to the beginning of the text. | | public int | setIndex(int index)
Set the iteration position in the input text that is being normalized
and return the first normalized character at that position.
Note: This method sets the position in the input text,
while
Normalizer.next and
Normalizer.previous iterate through characters
in the normalized output. | | public void | setIndexOnly(int index)
Set the iteration position in the input text that is being normalized,
without any immediate normalization. | | public void | setMode(Mode newMode)
Set the normalization mode for this object.
Note:If the normalization mode is changed while iterating
over a string, calls to
Normalizer.next and
Normalizer.previous may
return previously buffers characters in the old normalization mode
until the iteration is able to re-sync at the next base character.
It is safest to call
Normalizer.setText setText() ,
Normalizer.first ,
Normalizer.last , etc. | | public void | setOption(int option, boolean value)
Set options that affect this Normalizer's operation.
Options do not change the basic composition or decomposition operation
that is being performed , but they control whether
certain optional portions of the operation are done.
Currently the only available option is:
Parameters:
option - the option whose value is to be set.
Parameters:
value - the new setting for the option. | | public void | setText(StringBuffer newText)
Set the input text over which this Normalizer will iterate. | | public void | setText(char[] newText)
Set the input text over which this Normalizer will iterate. | | public void | setText(String newText)
Set the input text over which this Normalizer will iterate. | | public void | setText(CharacterIterator newText)
Set the input text over which this Normalizer will iterate. | | public void | setText(UCharacterIterator newText)
Set the input text over which this Normalizer will iterate. | | public int | startIndex()
Retrieve the index of the start of the input text. |
| COMPARE_CODE_POINT_ORDER | | final public static int COMPARE_CODE_POINT_ORDER(Code) | | Option bit for compare:
Compare strings in code point order instead of code unit order.
|
| COMPARE_IGNORE_CASE | | final public static int COMPARE_IGNORE_CASE(Code) | | Option bit for compare:
Perform case-insensitive comparison.
|
| COMPARE_NORM_OPTIONS_SHIFT | | final public static int COMPARE_NORM_OPTIONS_SHIFT(Code) | | Lowest-order bit number of compare() options bits corresponding to
normalization options bits.
The options parameter for compare() uses most bits for
itself and for various comparison and folding flags.
The most significant bits, however, are shifted down and passed on
to the normalization implementation.
(That is, from compare(..., options, ...),
options>>COMPARE_NORM_OPTIONS_SHIFT will be passed on to the
internal normalization functions.)
See Also: Normalizer.compare |
| DEFAULT | | final public static Mode DEFAULT(Code) | | Default normalization.
|
| DONE | | final public static int DONE(Code) | | Constant indicating that the end of the iteration has been reached.
This is guaranteed to have the same value as
UCharacterIterator.DONE .
|
| FCD | | final public static Mode FCD(Code) | | "Fast C or D" form.
|
| FOLD_CASE_DEFAULT | | final public static int FOLD_CASE_DEFAULT(Code) | | Option bit for compare:
Case sensitively compare the strings
|
| FOLD_CASE_EXCLUDE_SPECIAL_I | | final public static int FOLD_CASE_EXCLUDE_SPECIAL_I(Code) | | Option value for case folding: exclude the mappings for dotted I
and dotless i marked with 'I' in CaseFolding.txt.
|
| IGNORE_HANGUL | | final public static int IGNORE_HANGUL(Code) | | Option to disable Hangul/Jamo composition and decomposition.
This option applies to Korean text,
which can be represented either in the Jamo alphabet or in Hangul
characters, which are really just two or three Jamo combined
into one visual glyph. Since Jamo takes up more storage space than
Hangul, applications that process only Hangul text may wish to turn
this option on when decomposing text.
The Unicode standard treates Hangul to Jamo conversion as a
canonical decomposition, so this option must be turned off if you
wish to transform strings into one of the standard
Unicode Normalization Forms.
See Also: Normalizer.setOption |
| INPUT_IS_FCD | | final public static int INPUT_IS_FCD(Code) | | Option bit for compare:
Both input strings are assumed to fulfill FCD conditions.
|
| MAYBE | | final public static QuickCheckResult MAYBE(Code) | | Indicates it cannot be determined if string is in the normalized
format without further thorough checks.
|
| NFC | | final public static Mode NFC(Code) | | Canonical decomposition followed by canonical composition.
|
| NFD | | final public static Mode NFD(Code) | | Canonical decomposition.
|
| NFKC | | final public static Mode NFKC(Code) | | Compatibility decomposition followed by canonical composition.
|
| NFKD | | final public static Mode NFKD(Code) | | Compatibility decomposition.
|
| NO | | final public static QuickCheckResult NO(Code) | | Indicates that string is not in the normalized format
|
| NONE | | final public static Mode NONE(Code) | | No decomposition/composition.
|
| NO_OP | | final public static Mode NO_OP(Code) | | Null operation for use with the
Normalizer.Normalizer constructors and the static
Normalizer.normalize normalize method. This value tells
the Normalizer to do nothing but return unprocessed characters
from the underlying String or CharacterIterator. If you have code which
requires raw text at some times and normalized text at others, you can
use NO_OP for the cases where you want raw text, rather
than having a separate code path that bypasses Normalizer
altogether.
See Also: Normalizer.setMode
See Also: Normalizer.NONE |
| UNICODE_3_2 | | final public static int UNICODE_3_2(Code) | | Options bit set value to select Unicode 3.2 normalization
(except NormalizationCorrections).
At most one Unicode version can be selected at a time.
|
| YES | | final public static QuickCheckResult YES(Code) | | Indicates that string is in the normalized format
|
| Normalizer | | public Normalizer(String str, Mode mode, int opt)(Code) | | Creates a new Normalizer object for iterating over the
normalized form of a given string.
The options parameter specifies which optional
Normalizer features are to be enabled for this object.
Parameters:
str - The string to be normalized. The normalizationwill start at the beginning of the string.
Parameters:
mode - The normalization mode.
Parameters:
opt - Any optional features to be enabled.Currently the only available option is Normalizer.UNICODE_3_2.If you want the default behavior corresponding to one of thestandard Unicode Normalization Forms, use 0 for this argument. |
| Normalizer | | public Normalizer(CharacterIterator iter, Mode mode, int opt)(Code) | | Creates a new Normalizer object for iterating over the
normalized form of the given text.
Parameters:
iter - The input text to be normalized. The normalizationwill start at the beginning of the string.
Parameters:
mode - The normalization mode.
Parameters:
opt - Any optional features to be enabled.Currently the only available option is Normalizer.UNICODE_3_2.If you want the default behavior corresponding to one of thestandard Unicode Normalization Forms, use 0 for this argument. |
| Normalizer | | public Normalizer(UCharacterIterator iter, Mode mode, int options)(Code) | | Creates a new Normalizer object for iterating over the
normalized form of the given text.
Parameters:
iter - The input text to be normalized. The normalizationwill start at the beginning of the string.
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). |
| clone | | public Object clone()(Code) | | Clones this Normalizer object. All properties of this
object are duplicated in the new object, including the cloning of any
CharacterIterator that was passed in to the constructor
or to
Normalizer.setText(CharacterIterator) setText .
However, the text storage underlying
the CharacterIterator is not duplicated unless the
iterator's clone method does so.
|
| compare | | public static int compare(char[] s1, int s1Start, int s1Limit, char[] s2, int s2Start, int s2Limit, int options)(Code) | | Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Canonical equivalence between two strings is defined as their normalized
forms (NFD or NFC) being identical.
This function compares strings incrementally instead of normalizing
(and optionally case-folding) both strings entirely,
improving performance significantly.
Bulk normalization is only necessary if the strings do not fulfill the
FCD conditions. Only in this case, and only if the strings are relatively
long, is memory allocated temporarily.
For FCD strings and short non-FCD strings there is no memory allocation.
Semantically, this is equivalent to
strcmp[CodePointOrder](foldCase(NFD(s1)), foldCase(NFD(s2)))
where code point order and foldCase are all optional.
Parameters:
s1 - First source character array.
Parameters:
s1Start - start index of source
Parameters:
s1Limit - limit of the source
Parameters:
s2 - Second source character array.
Parameters:
s2Start - start index of the source
Parameters:
s2Limit - limit of the source
Parameters:
options - A bit set of options:- FOLD_CASE_DEFAULT or 0 is used for default options:Case-sensitive comparison in code unit order, and the input stringsare quick-checked for FCD.- INPUT_IS_FCDSet if the caller knows that both s1 and s2 fulfill the FCD conditions.If not set, the function will quickCheck for FCDand normalize if necessary.- COMPARE_CODE_POINT_ORDERSet to choose code point order instead of code unit order- COMPARE_IGNORE_CASESet to compare strings case-insensitively using case folding,instead of case-sensitively.If set, then the following case folding options are used. <0 or 0 or >0 as usual for string comparisons
See Also: Normalizer.normalize
See Also: Normalizer.FCD |
| compare | | public static int compare(String s1, String s2, int options)(Code) | | Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Convenience method.
Parameters:
s1 - First source string.
Parameters:
s2 - Second source string.
Parameters:
options - A bit set of options:- FOLD_CASE_DEFAULT or 0 is used for default options:Case-sensitive comparison in code unit order, and the input stringsare quick-checked for FCD.- INPUT_IS_FCDSet if the caller knows that both s1 and s2 fulfill the FCD conditions. If not set, the function will quickCheck for FCDand normalize if necessary.- COMPARE_CODE_POINT_ORDERSet to choose code point order instead of code unit order- COMPARE_IGNORE_CASESet to compare strings case-insensitively using case folding,instead of case-sensitively.If set, then the following case folding options are used. <0 or 0 or >0 as usual for string comparisons
See Also: Normalizer.normalize
See Also: Normalizer.FCD |
| compare | | public static int compare(char[] s1, char[] s2, int options)(Code) | | Compare two strings for canonical equivalence.
Further options include case-insensitive comparison and
code point order (as opposed to code unit order).
Convenience method.
Parameters:
s1 - First source string.
Parameters:
s2 - Second source string.
Parameters:
options - A bit set of options:- FOLD_CASE_DEFAULT or 0 is used for default options:Case-sensitive comparison in code unit order, and the input stringsare quick-checked for FCD.- INPUT_IS_FCDSet if the caller knows that both s1 and s2 fulfill the FCD conditions. If not set, the function will quickCheck for FCDand normalize if necessary.- COMPARE_CODE_POINT_ORDERSet to choose code point order instead of code unit order- COMPARE_IGNORE_CASESet to compare strings case-insensitively using case folding,instead of case-sensitively.If set, then the following case folding options are used. <0 or 0 or >0 as usual for string comparisons
See Also: Normalizer.normalize
See Also: Normalizer.FCD |
| compare | | public static int compare(int char32a, int char32b, int options)(Code) | | Convenience method that can have faster implementation
by not allocating buffers.
Parameters:
char32a - the first code point to be checked against the
Parameters:
char32b - the second code point
Parameters:
options - A bit set of options |
| compare | | public static int compare(int char32a, String str2, int options)(Code) | | Convenience method that can have faster implementation
by not allocating buffers.
Parameters:
char32a - the first code point to be checked against
Parameters:
str2 - the second string
Parameters:
options - A bit set of options |
| compose | | public static String compose(String str, boolean compat)(Code) | | Compose a string.
The string will be composed to according the the specified mode.
Parameters:
str - The string to compose.
Parameters:
compat - If true the string will be composed accoding to NFKC rules and if false will be composed according to NFC rules. String The composed string |
| compose | | public static String compose(String str, boolean compat, int options)(Code) | | Compose a string.
The string will be composed to according the the specified mode.
Parameters:
str - The string to compose.
Parameters:
compat - If true the string will be composed accoding to NFKC rules and if false will be composed according to NFC rules.
Parameters:
options - The only recognized option is UNICODE_3_2 String The composed string |
| compose | | public static int compose(char[] source, char[] target, boolean compat, int options)(Code) | | Compose a string.
The string will be composed to according the the specified mode.
Parameters:
source - The char array to compose.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
compat - If true the char array will be composed accoding to NFKC rules and if false will be composed according to NFC rules.
Parameters:
options - The normalization options, ORed together (0 for no options). int The total buffer size needed;if greater than length of result, the output was truncated.
exception:
IndexOutOfBoundsException - if target.length is less than the required length |
| compose | | public static int compose(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, boolean compat, int options)(Code) | | Compose a string.
The string will be composed to according the the specified mode.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
compat - If true the char array will be composed accoding to NFKC rules and if false will be composed according to NFC rules.
Parameters:
options - The normalization options, ORed together (0 for no options). int The total buffer size needed;if greater than length of result, the output was truncated.
exception:
IndexOutOfBoundsException - if target.length is less than the required length |
| concatenate | | public static int concatenate(char[] left, int leftStart, int leftLimit, char[] right, int rightStart, int rightLimit, char[] dest, int destStart, int destLimit, Normalizer.Mode mode, int options)(Code) | | Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
With the input strings already being normalized,
this function will use next() and previous()
to find the adjacent end pieces of the input strings.
Only the concatenation of these end pieces will be normalized and
then concatenated with the remaining parts of the input strings.
It is allowed to have dest==left to avoid copying the entire left string.
Parameters:
left - Left source array, may be same as dest.
Parameters:
leftStart - start in the left array.
Parameters:
leftLimit - limit in the left array (==length)
Parameters:
right - Right source array.
Parameters:
rightStart - start in the right array.
Parameters:
rightLimit - limit in the right array (==length)
Parameters:
dest - The output buffer; can be null if destStart==destLimit==0 for pure preflighting.
Parameters:
destStart - start in the destination array
Parameters:
destLimit - limit in the destination array (==length)
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). Length of output (number of chars) when successful or IndexOutOfBoundsException
exception:
IndexOutOfBoundsException - whose message has the string representation of destination capacity required.
See Also: Normalizer.normalize
See Also: Normalizer.next
See Also: Normalizer.previous
exception:
IndexOutOfBoundsException - if target capacity is less than therequired length |
| concatenate | | public static String concatenate(char[] left, char[] right, Mode mode, int options)(Code) | | Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
For details see concatenate
Parameters:
left - Left source string.
Parameters:
right - Right source string.
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). result
See Also: Normalizer.concatenate
See Also: Normalizer.normalize
See Also: Normalizer.next
See Also: Normalizer.previous
See Also: Normalizer.concatenate |
| concatenate | | public static String concatenate(String left, String right, Mode mode, int options)(Code) | | Concatenate normalized strings, making sure that the result is normalized
as well.
If both the left and the right strings are in
the normalization form according to "mode",
then the result will be
dest=normalize(left+right, mode)
For details see concatenate
Parameters:
left - Left source string.
Parameters:
right - Right source string.
Parameters:
mode - The normalization mode.
Parameters:
options - The normalization options, ORed together (0 for no options). result
See Also: Normalizer.concatenate
See Also: Normalizer.normalize
See Also: Normalizer.next
See Also: Normalizer.previous
See Also: Normalizer.concatenate |
| current | | public int current()(Code) | | Return the current character in the normalized text->
The codepoint as an int |
| decompose | | public static String decompose(String str, boolean compat)(Code) | | Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
str - The string to decompose.
Parameters:
compat - If true the string will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules. String The decomposed string |
| decompose | | public static String decompose(String str, boolean compat, int options)(Code) | | Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
str - The string to decompose.
Parameters:
compat - If true the string will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules.
Parameters:
options - The normalization options, ORed together (0 for no options). String The decomposed string |
| decompose | | public static int decompose(char[] source, char[] target, boolean compat, int options)(Code) | | Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
source - The char array to decompose.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
compat - If true the char array will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules. int The total buffer size needed;if greater than length of result,the output was truncated.
Parameters:
options - The normalization options, ORed together (0 for no options).
exception:
IndexOutOfBoundsException - if the target capacity is less thanthe required length |
| decompose | | public static int decompose(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, boolean compat, int options)(Code) | | Decompose a string.
The string will be decomposed to according the the specified mode.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
compat - If true the char array will be decomposed accoding to NFKD rules and if false will be decomposed according to NFD rules.
Parameters:
options - The normalization options, ORed together (0 for no options). int The total buffer size needed;if greater than length of result,the output was truncated.
exception:
IndexOutOfBoundsException - if the target capacity is less thanthe required length |
| endIndex | | public int endIndex()(Code) | | Retrieve the index of the end of the input text-> This is the end index
of the CharacterIterator or the length of the String
over which this Normalizer is iterating
The current iteration position |
| first | | public int first()(Code) | | Return the first character in the normalized text-> This resets
the Normalizer's position to the beginning of the text->
The codepoint as an int |
| getBeginIndex | | public int getBeginIndex()(Code) | | Retrieve the index of the start of the input text. This is the begin
index of the CharacterIterator or the start (i.e. 0) of the
String over which this Normalizer is iterating
The codepoint as an int
See Also: Normalizer.startIndex |
| getEndIndex | | public int getEndIndex()(Code) | | Retrieve the index of the end of the input text. This is the end index
of the CharacterIterator or the length of the String
over which this Normalizer is iterating
The codepoint as an int
See Also: Normalizer.endIndex |
| getFC_NFKC_Closure | | public static int getFC_NFKC_Closure(int c, char[] dest)(Code) | | Gets the FC_NFKC closure set from the normalization data
Parameters:
c - The code point whose closure set is to be retrieved
Parameters:
dest - The char array to recive the closure set |
| getFC_NFKC_Closure | | public static String getFC_NFKC_Closure(int c)(Code) | | Gets the FC_NFKC closure set from the normalization data
Parameters:
c - The the code point whose closure set is to be retrieved String representation of the closure set |
| getIndex | | public int getIndex()(Code) | | Retrieve the current iteration position in the input text that is
being normalized. This method is useful in applications such as
searching, where you need to be able to determine the position in
the input text that corresponds to a given normalized output character.
Note: This method sets the position in the input, while
Normalizer.next and
Normalizer.previous iterate through characters in the
output. This means that there is not necessarily a one-to-one
correspondence between characters returned by next and
previous and the indices passed to and returned from
setIndex and
Normalizer.getIndex .
The current iteration position |
| getLength | | public int getLength()(Code) | | Gets the length of underlying text storage
the length |
| getText | | public int getText(char[] fillIn)(Code) | | Gets the underlying text storage
Parameters:
fillIn - the char buffer to fill the UTF-16 units.The length of the buffer should be equal to the length of theunderlying text storage
throws:
IndexOutOfBoundsException -
See Also: Normalizer.getLength |
| getText | | public String getText()(Code) | | Returns the text under iteration as a string
a copy of the text under iteration. |
| getUnicodeVersion | | static VersionInfo getUnicodeVersion()(Code) | | Fetches the Unicode version burned into the Normalization data file
VersionInfo version information of the normalizer |
| isNFSkippable | | public static boolean isNFSkippable(int c, Mode mode)(Code) | | Internal API
|
| isNormalized | | public static boolean isNormalized(char[] src, int start, int limit, Mode mode, int options)(Code) | | Test if a string is in a given normalization form.
This is semantically equivalent to source.equals(normalize(source, mode)).
Unlike quickCheck(), this function returns a definitive result,
never a "maybe".
For NFD, NFKD, and FCD, both functions work exactly the same.
For NFC and NFKC where quickCheck may return "maybe", this function will
perform further tests to arrive at a true/false result.
Parameters:
src - The input array of characters to be checked to see if it is normalized
Parameters:
start - The strart index in the source
Parameters:
limit - The limit index in the source
Parameters:
mode - the normalization mode
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2 Boolean value indicating whether the source string is in the"mode" normalization form |
| isNormalized | | public static boolean isNormalized(String str, Mode mode, int options)(Code) | | Convenience Method
Parameters:
str - the input string to be checked to see if it is normalized
Parameters:
mode - the normalization mode
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2
See Also: Normalizer.isNormalized |
| isNormalized | | public static boolean isNormalized(int char32, Mode mode, int options)(Code) | | Convenience Method
Parameters:
char32 - the input code point to be checked to see if it is normalized
Parameters:
mode - the normalization mode
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2
See Also: Normalizer.isNormalized |
| last | | public int last()(Code) | | Return the last character in the normalized text-> This resets
the Normalizer's position to be just before the
the input text corresponding to that normalized character.
The codepoint as an int |
| next | | public int next()(Code) | | Return the next character in the normalized text and advance
the iteration position by one. If the end
of the text has already been reached,
Normalizer.DONE is returned.
The codepoint as an int |
| normalize | | public static String normalize(String str, Mode mode, int options)(Code) | | Normalizes a String using the given normalization operation.
The options parameter specifies which optional
Normalizer features are to be enabled for this operation.
Currently the only available option is
Normalizer.UNICODE_3_2 .
If you want the default behavior corresponding to one of the standard
Unicode Normalization Forms, use 0 for this argument.
Parameters:
str - the input string to be normalized.
Parameters:
mode - the normalization mode
Parameters:
options - the optional features to be enabled. String the normalized string |
| normalize | | public static String normalize(String src, Mode mode)(Code) | | Normalize a string.
The string will be normalized according the the specified normalization
mode and options.
Parameters:
src - The string to normalize.
Parameters:
mode - The normalization mode; one of Normalizer.NONE, Normalizer.NFD, Normalizer.NFC, Normalizer.NFKC, Normalizer.NFKD, Normalizer.DEFAULT the normalized string |
| normalize | | public static int normalize(char[] source, char[] target, Mode mode, int options)(Code) | | Normalize a string.
The string will be normalized according the the specified normalization
mode and options.
Parameters:
source - The char array to normalize.
Parameters:
target - A char buffer to receive the normalized text.
Parameters:
mode - The normalization mode; one of Normalizer.NONE, Normalizer.NFD, Normalizer.NFC, Normalizer.NFKC, Normalizer.NFKD, Normalizer.DEFAULT
Parameters:
options - The normalization options, ORed together (0 for no options). int The total buffer size needed;if greater than length of result, the output was truncated.
exception:
IndexOutOfBoundsException - if the target capacity is less than the required length |
| normalize | | public static int normalize(char[] src, int srcStart, int srcLimit, char[] dest, int destStart, int destLimit, Mode mode, int options)(Code) | | Normalize a string.
The string will be normalized according the the specified normalization
mode and options.
Parameters:
src - The char array to compose.
Parameters:
srcStart - Start index of the source
Parameters:
srcLimit - Limit index of the source
Parameters:
dest - The char buffer to fill in
Parameters:
destStart - Start index of the destination buffer
Parameters:
destLimit - End index of the destination buffer
Parameters:
mode - The normalization mode; one of Normalizer.NONE, Normalizer.NFD, Normalizer.NFC, Normalizer.NFKC, Normalizer.NFKD, Normalizer.DEFAULT
Parameters:
options - The normalization options, ORed together (0 for no options). int The total buffer size needed;if greater than length of result, the output was truncated.
exception:
IndexOutOfBoundsException - if the target capacity is less than the required length |
| normalize | | public static String normalize(int char32, Mode mode, int options)(Code) | | Normalize a codepoint accoding to the given mode
Parameters:
char32 - The input string to be normalized.
Parameters:
mode - The normalization mode
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2 String The normalized string
See Also: Normalizer.UNICODE_3_2 |
| normalize | | public static String normalize(int char32, Mode mode)(Code) | | Conveinience method to normalize a codepoint accoding to the given mode
Parameters:
char32 - The input string to be normalized.
Parameters:
mode - The normalization mode String The normalized string
See Also: Normalizer.UNICODE_3_2
See Also: |
| previous | | public int previous()(Code) | | Return the previous character in the normalized text and decrement
the iteration position by one. If the beginning
of the text has already been reached,
Normalizer.DONE is returned.
The codepoint as an int |
| quickCheck | | public static QuickCheckResult quickCheck(String source, Mode mode)(Code) | | Convenience method.
Parameters:
source - string for determining if it is in a normalized format
Parameters:
mode - normalization format (Normalizer.NFC,Normalizer.NFD, Normalizer.NFKC,Normalizer.NFKD) Return code to specify if the text is normalized or not (Normalizer.YES, Normalizer.NO or Normalizer.MAYBE) |
| quickCheck | | public static QuickCheckResult quickCheck(String source, Mode mode, int options)(Code) | | Convenience method.
Parameters:
source - string for determining if it is in a normalized format
Parameters:
mode - normalization format (Normalizer.NFC,Normalizer.NFD, Normalizer.NFKC,Normalizer.NFKD)
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2 Return code to specify if the text is normalized or not (Normalizer.YES, Normalizer.NO or Normalizer.MAYBE) |
| quickCheck | | public static QuickCheckResult quickCheck(char[] source, Mode mode, int options)(Code) | | Convenience method.
Parameters:
source - Array of characters for determining if it is in a normalized format
Parameters:
mode - normalization format (Normalizer.NFC,Normalizer.NFD, Normalizer.NFKC,Normalizer.NFKD)
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2 Return code to specify if the text is normalized or not (Normalizer.YES, Normalizer.NO or Normalizer.MAYBE) |
| quickCheck | | public static QuickCheckResult quickCheck(char[] source, int start, int limit, Mode mode, int options)(Code) | | Performing quick check on a string, to quickly determine if the string is
in a particular normalization format.
Three types of result can be returned Normalizer.YES, Normalizer.NO or
Normalizer.MAYBE. Result Normalizer.YES indicates that the argument
string is in the desired normalized format, Normalizer.NO determines that
argument string is not in the desired normalized format. A
Normalizer.MAYBE result indicates that a more thorough check is required,
the user may have to put the string in its normalized form and compare
the results.
Parameters:
source - string for determining if it is in a normalized format
Parameters:
start - the start index of the source
Parameters:
limit - the limit index of the source it is equal to the length
Parameters:
mode - normalization format (Normalizer.NFC,Normalizer.NFD, Normalizer.NFKC,Normalizer.NFKD)
Parameters:
options - Options for use with exclusion set an tailored NormalizationThe only option that is currently recognized is UNICODE_3_2 Return code to specify if the text is normalized or not (Normalizer.YES, Normalizer.NO orNormalizer.MAYBE) |
| reset | | public void reset()(Code) | | Reset the index to the beginning of the text.
This is equivalent to setIndexOnly(startIndex)).
|
| setIndex | | public int setIndex(int index)(Code) | | Set the iteration position in the input text that is being normalized
and return the first normalized character at that position.
Note: This method sets the position in the input text,
while
Normalizer.next and
Normalizer.previous iterate through characters
in the normalized output. This means that there is not
necessarily a one-to-one correspondence between characters returned
by next and previous and the indices passed to and
returned from setIndex and
Normalizer.getIndex .
Parameters:
index - the desired index in the input text-> the first normalized character that is the result of iteratingforward starting at the given index.
throws:
IllegalArgumentException - if the given index is less thanNormalizer.getBeginIndex or greater than Normalizer.getEndIndex. The codepoint as an int |
| setIndexOnly | | public void setIndexOnly(int index)(Code) | | Set the iteration position in the input text that is being normalized,
without any immediate normalization.
After setIndexOnly(), getIndex() will return the same index that is
specified here.
Parameters:
index - the desired index in the input text. |
| setMode | | public void setMode(Mode newMode)(Code) | | Set the normalization mode for this object.
Note:If the normalization mode is changed while iterating
over a string, calls to
Normalizer.next and
Normalizer.previous may
return previously buffers characters in the old normalization mode
until the iteration is able to re-sync at the next base character.
It is safest to call
Normalizer.setText setText() ,
Normalizer.first ,
Normalizer.last , etc. after calling setMode.
Parameters:
newMode - the new mode for this Normalizer.The supported modes are:
See Also: Normalizer.getMode |
| setOption | | public void setOption(int option, boolean value)(Code) | | Set options that affect this Normalizer's operation.
Options do not change the basic composition or decomposition operation
that is being performed , but they control whether
certain optional portions of the operation are done.
Currently the only available option is:
Parameters:
option - the option whose value is to be set.
Parameters:
value - the new setting for the option. Use true toturn the option on and false to turn it off.
See Also: Normalizer.getOption |
| setText | | public void setText(StringBuffer newText)(Code) | | Set the input text over which this Normalizer will iterate.
The iteration position is set to the beginning of the input text->
Parameters:
newText - The new string to be normalized. |
| setText | | public void setText(char[] newText)(Code) | | Set the input text over which this Normalizer will iterate.
The iteration position is set to the beginning of the input text->
Parameters:
newText - The new string to be normalized. |
| setText | | public void setText(String newText)(Code) | | Set the input text over which this Normalizer will iterate.
The iteration position is set to the beginning of the input text->
Parameters:
newText - The new string to be normalized. |
| setText | | public void setText(CharacterIterator newText)(Code) | | Set the input text over which this Normalizer will iterate.
The iteration position is set to the beginning of the input text->
Parameters:
newText - The new string to be normalized. |
| setText | | public void setText(UCharacterIterator newText)(Code) | | Set the input text over which this Normalizer will iterate.
The iteration position is set to the beginning of the string.
Parameters:
newText - The new string to be normalized. |
| startIndex | | public int startIndex()(Code) | | Retrieve the index of the start of the input text. This is the begin
index of the CharacterIterator or the start (i.e. 0) of the
String over which this Normalizer is iterating
The current iteration position |
|
|