|
This interface declares the methods a
de.susebox.jtopas.Tokenizer needs
to deal with separators.
Separators are single characters that are either not very important in the
tokenizing process or are sufficiently characterized by their token image.
Examples are opening and closing parentheses, commas, semicolons etc. Their
main feature is that they separate other tokens in the same way as whiterspaces
and comments.
See Also: de.susebox.jtopas.Tokenizer
See Also: de.susebox.jtopas.TokenizerProperties
See Also: de.susebox.jtopas.spi.DataMapper
author:
Heiko Blau |