| Method Summary |
|
| char | childTokenCharAt(int rawOffset, int index)
Get character of a token from the character sequence represented
by this support.
Parameters:
rawOffset - raw offset of the child token.The given offset value may need to be preprocessed before using (it dependson a nature of the token list).
Parameters:
index - index inside the token's text that should be returned.This value cannot be simply added to the previous parameterfor mutable token lists as the value could errorneously pointinto a middle of the offset gap then. |
| int | childTokenOffset(int rawOffset)
Get absolute offset of the child token with the given raw offset
in the underlying input.
Parameters:
rawOffset - raw offset of the child token. |
| int | endOffset()
Get offset where the last token of this token list should end. |
| InputAttributes | inputAttributes()
Extra attributes related to the input being lexed. |
| boolean | isContinuous()
Returns true if the underlying token list does not contain offset ranges
that would not be covered by tokens. |
| boolean | isRemoved()
Check if this token list is removed from token hierarchy. |
| LanguagePath | languagePath()
Language path of this token list. |
| int | lookahead(int index)
Get lookahead information for the token at the existing token index.
Lookahead is number of characters that the lexer has read
past the end of the given token in order to recognize it in the text.
This information allows the lexer to know whether modifications
past the end of the token can affect its validity.
In general only mutable token lists benefit from this information
but non-mutable token lists may store the information as well for testing
purposes.
Parameters:
index - index of the existing token. |
| int | modCount()
Get number of modifications which mutated this token list.
Token sequence remembers this number when it gets constructed
and checks this number when it moves between tokens
and if there is an extra modification performed it throws
IllegalStateException.
This is also used to check whether this token list corresponds to mutable input
or not because unmodifiable lists return -1 from this method.
For branch token lists the
TokenList.updateStartOffsetShift() ensures
that the value returned by this method is most up-to-date
(equals to the root list's one).
number of modifications performed to the list.
Returns -1 if this list is constructed for immutable input and cannot be mutated. |
| AbstractToken<T> | replaceFlyToken(int index, AbstractToken<T> flyToken, int offset)
Replace flyweight token at the given index with its non-flyweight copy.
This may be requested by TokenSequence.offsetToken().
Parameters:
index - >=0 index of the flyweight token in this list.
Parameters:
flyToken - non-null flyweight token. |
| TokenList | root()
Get the root token list of the token list hierarchy. |
| Set<T> | skipTokenIds()
Get set of token ids to be skipped during token creation. |
| int | startOffset()
Get offset where a first token of this token list should start. |
| Object | state(int index)
Get state information for the token at the existing token index.
It is an object defining lexer's state after recognition
of the given token.
This information allows to restart the lexer at the end of the given token.
In general only mutable token lists benefit from this information
but non-mutable token lists may store the information as well for testing
purposes.
Parameters:
index - index of the existing token. |
| int | tokenCount()
Get total count of tokens in the list. |
| int | tokenCountCurrent()
Return present number of tokens in the token list but do not create
any new tokens (because of possible lazy token creation).
This is necessary e.g. |
| TokenHierarchyOperation | tokenHierarchyOperation()
Get token hierarchy operation for this token list or null
if this token list does not have any token hierarchy. |
| int | tokenOffset(int index)
Get absolute offset of the token at the given index in the token list. |
| Object | tokenOrEmbeddingContainer(int index)
Get token or
EmbeddingContainer at given index in this list.
The method's implementation may need to be synchronized as multiple
threads can access it at the same time.
The requested index value may be arbitrarily high
(e.g. |
| void | wrapToken(int index, EmbeddingContainer<T> embeddingContainer)
Wrap the token by a branch token list due to language embedding
that exists for the token. |