| org.netbeans.spi.lexer |
org.netbeans.spi.lexer
The main abstract class in the Lexer SPI that must be implemented
is {@link org.netbeans.spi.lexer.LanguageHierarchy} that mainly defines
set of token ids and token categories for the new language
and its {@link org.netbeans.spi.lexer.Lexer}.
The language hierarchy may also define embedded languages.
|
| Java Source File Name | Type | Comment |
| EmbeddingPresence.java | enum | Defines whether a default embedding can be present for the given token id or not.
It allows to speed up TokenSequence.embedded() calls considerably in most cases.
This only affects the default embedding creation. |
| LanguageEmbedding.java | Class | Description of a particular language embedding including
starting and ending skipped regions of a token containing this embedding
and a definition of an embedded language hierarchy. |
| LanguageHierarchy.java | Class | Definition of a language, its lexer and its embedded languages. |
| LanguageProvider.java | Class | The Language provider. |
| Lexer.java | Interface | Lexer reads input characters from
LexerInput and groups
them into tokens.
The lexer delegates token creation
to
TokenFactory.createToken(TokenId) .
Token factory instance should be given to the lexer in its constructor.
The lexer must be able to express its internal lexing
state at token boundaries and it must be able
to restart lexing from such state.
It is expected that if the input characters following the restart point
would not change then the lexer will return the same tokens
regardless whether it was restarted at the restart point
or run from the input begining as a batch lexer.
Testing of the lexers:
Testing of newly written lexers can be performed in several ways.
The most simple way is to test batch lexing first
(see e.g.
org.netbeans.lib.lexer.test.simple.SimpleLexerBatchTest in lexer module tests).
Then an "incremental" behavior of the new lexer can be tested
(see e.g. |
| LexerInput.java | Class | Provides characters to feed the
Lexer . |
| LexerRestartInfo.java | Class | Lexer restart info contains all the necessary information for restarting
of a lexer mainly the lexer input, state and token factory. |
| MutableTextInput.java | Class | Mutable attributed character sequence allowing to listen for changes in its text. |
| TokenFactory.java | Class | Lexer should delegate all the token instances creation to this class. |
| TokenHierarchyControl.java | Class | Control class for managing token hierarchy of a mutable text input. |
| TokenPropertyProvider.java | Interface | Provides extra properties of a token. |
| TokenValidator.java | Interface | Token validator checks whether an existing token was affected
by just performed input source modification so that it needs to be relexed. |