| java.lang.Object
 org.syntax.jedit.tokenmarker.TokenMarker
All known Subclasses: org.syntax.jedit.tokenmarker.BatchFileTokenMarker, org.syntax.jedit.tokenmarker.HTMLTokenMarker, org.syntax.jedit.tokenmarker.EiffelTokenMarker, org.syntax.jedit.tokenmarker.ShellScriptTokenMarker, org.syntax.jedit.tokenmarker.CTokenMarker, org.syntax.jedit.tokenmarker.PropsTokenMarker, org.syntax.jedit.tokenmarker.MakefileTokenMarker, org.syntax.jedit.tokenmarker.PatchTokenMarker, org.syntax.jedit.tokenmarker.PerlTokenMarker, org.syntax.jedit.tokenmarker.PHPTokenMarker, org.syntax.jedit.tokenmarker.PythonTokenMarker, org.syntax.jedit.tokenmarker.SQLTokenMarker, org.syntax.jedit.tokenmarker.TeXTokenMarker,
| TokenMarker | | abstract public class TokenMarker (Code) | | A token marker that splits lines of text into tokens. Each token carries
a length field and an identification tag that can be mapped to a color
for painting that token.
For performance reasons, the linked list of tokens is reused after each
line is tokenized. Therefore, the return value of markTokens
should only be used for immediate painting. Notably, it cannot be
cached.
author:
Slava Pestov
version:
$Id: TokenMarker.java 5365 2007-11-01 05:24:32Z davmac $
See Also: org.syntax.jedit.Token |
| Inner Class :public class LineInfo | |
| Field Summary | |
| protected Token | firstToken
The first token in the list. | | protected int | lastLine
The last tokenized line. | | protected Token | lastToken
The last token in the list. | | protected int | length
The number of lines in the model being tokenized. | | protected LineInfo[] | lineInfo
An array for storing information about lines. | | protected boolean | nextLineRequested
True if the next line should be painted. |
| Constructor Summary | |
| protected | TokenMarker()
Creates a new TokenMarker. |
| Method Summary | |
| protected void | addToken(int length, byte id)
Adds a token to the token list. | | public void | deleteLines(int index, int lines)
Informs the token marker that line have been deleted from
the document. | | protected void | ensureCapacity(int index)
Ensures that the lineInfo array can contain the
specified index. | | public int | getLineCount()
Returns the number of lines in this token marker. | | public void | insertLines(int index, int lines)
Informs the token marker that lines have been inserted into
the document. | | public boolean | isNextLineRequested()
Returns true if the next line should be repainted. | | public Token | markTokens(Segment line, int lineIndex)
A wrapper for the lower-level markTokensImpl method
that is called to split a line up into tokens. | | abstract protected byte | markTokensImpl(byte token, Segment line, int lineIndex)
An abstract method that splits a line up into tokens. | | public boolean | supportsMultilineTokens()
Returns if the token marker supports tokens that span multiple
lines. |
| firstToken | | protected Token firstToken(Code) | | The first token in the list. This should be used as the return
value from markTokens().
|
| lastLine | | protected int lastLine(Code) | | The last tokenized line.
|
| lastToken | | protected Token lastToken(Code) | | The last token in the list. New tokens are added here.
This should be set to null before a new line is to be tokenized.
|
| length | | protected int length(Code) | | The number of lines in the model being tokenized. This can be
less than the length of the lineInfo array.
|
| lineInfo | | protected LineInfo[] lineInfo(Code) | | An array for storing information about lines. It is enlarged and
shrunk automatically by the insertLines() and
deleteLines() methods.
|
| nextLineRequested | | protected boolean nextLineRequested(Code) | | True if the next line should be painted.
|
| TokenMarker | | protected TokenMarker()(Code) | | Creates a new TokenMarker. This DOES NOT create
a lineInfo array; an initial call to insertLines()
does that.
|
| addToken | | protected void addToken(int length, byte id)(Code) | | Adds a token to the token list.
Parameters:
length - The length of the token
Parameters:
id - The id of the token |
| deleteLines | | public void deleteLines(int index, int lines)(Code) | | Informs the token marker that line have been deleted from
the document. This removes the lines in question from the
lineInfo array.
Parameters:
index - The first line number
Parameters:
lines - The number of lines |
| ensureCapacity | | protected void ensureCapacity(int index)(Code) | | Ensures that the lineInfo array can contain the
specified index. This enlarges it if necessary. No action is
taken if the array is large enough already.
It should be unnecessary to call this under normal
circumstances; insertLine() should take care of
enlarging the line info array automatically.
Parameters:
index - The array index |
| getLineCount | | public int getLineCount()(Code) | | Returns the number of lines in this token marker.
|
| insertLines | | public void insertLines(int index, int lines)(Code) | | Informs the token marker that lines have been inserted into
the document. This inserts a gap in the lineInfo
array.
Parameters:
index - The first line number
Parameters:
lines - The number of lines |
| isNextLineRequested | | public boolean isNextLineRequested()(Code) | | Returns true if the next line should be repainted. This
will return true after a line has been tokenized that starts
a multiline token that continues onto the next line.
|
| markTokens | | public Token markTokens(Segment line, int lineIndex)(Code) | | A wrapper for the lower-level markTokensImpl method
that is called to split a line up into tokens.
Parameters:
line - The line
Parameters:
lineIndex - The line number |
| markTokensImpl | | abstract protected byte markTokensImpl(byte token, Segment line, int lineIndex)(Code) | | An abstract method that splits a line up into tokens. It
should parse the line, and call addToken() to
add syntax tokens to the token list. Then, it should return
the initial token type for the next line.
For example if the current line contains the start of a
multiline comment that doesn't end on that line, this method
should return the comment token type so that it continues on
the next line.
Parameters:
token - The initial token type for this line
Parameters:
line - The line to be tokenized
Parameters:
lineIndex - The index of the line in the document,starting at 0 The initial token type for the next line |
| supportsMultilineTokens | | public boolean supportsMultilineTokens()(Code) | | Returns if the token marker supports tokens that span multiple
lines. If this is true, the object using this token marker is
required to pass all lines in the document to the
markTokens() method (in turn).
The default implementation returns true; it should be overridden
to return false on simpler token markers for increased speed.
|
|
|