org.netbeans.api.lexer
The entrance point into Lexer API is
{@link org.netbeans.api.lexer.TokenHierarchy} class with
its static methods that provide its instance for the given input source.
{@link org.netbeans.api.lexer.TokenHierarchy#tokenSequence()}
allows to walk through the tokens that were created for the given input.
{@link org.netbeans.api.lexer.Token} models a single token recognized in the input.
It carries a token identification {@link org.netbeans.api.lexer.TokenId} returned by
{@link org.netbeans.api.lexer.Token#id()}
and token text represented as {@link java.lang.CharSequence}
returned by {@link org.netbeans.api.lexer.Token#text()}.
{@link org.netbeans.api.lexer.Language} represents
all the token ids for a single language.
Each token id may belong to one or more token categories that allow to better operate
tokens of the same type (e.g. keywords or operators).
org.netbeans.spi.lexer
The main abstract class in the Lexer SPI that must be implemented
is {@link org.netbeans.spi.lexer.LanguageHierarchy} that mainly defines
set of token ids and token categories for the new language
and its {@link org.netbeans.spi.lexer.Lexer}.
The language hierarchy may also define embedded languages.