| java.lang.Object
 org.netbeans.lib.lexer.test.LexerTestUtilities
| LexerTestUtilities | | final public class LexerTestUtilities (Code) | | Various utilities related to lexer's and token testing.
author:
mmetelka |
| Method Summary | |
| public static void | assertCollectionsEqual(Collection expected, Collection actual)
| | public static void | assertCollectionsEqual(String message, Collection expected, Collection actual)
| | public static void | assertConsistency(TokenHierarchy hi)
| | public static void | assertNextTokenEquals(TokenSequence ts, TokenId id, String text)
| | public static void | assertNextTokenEquals(String message, TokenSequence ts, TokenId id, String text)
| | public static void | assertTextEquals(CharSequence expected, CharSequence actual)
| | public static void | assertTextEquals(String message, CharSequence expected, CharSequence actual)
| | public static void | assertTokenEquals(TokenSequence ts, TokenId id, String text, int offset)
| | public static void | assertTokenEquals(String message, TokenSequence ts, TokenId id, String text, int offset)
Compare TokenSequence.token() to the given
token id, text and offset.
Parameters:
offset - expected offset. | | public static void | assertTokenEquals(TokenSequence ts, TokenId id, String text, int offset, int lookahead, Object state)
| | public static void | assertTokenEquals(String message, TokenSequence ts, TokenId id, String text, int offset, int lookahead, Object state)
| | public static void | assertTokenFlyweight(Token token)
| | public static void | assertTokenNotFlyweight(Token token)
| | public static void | assertTokenOffsetsEqual(String message, int offset1, int offset2)
| | public static void | assertTokenSequencesEqual(TokenSequence expected, TokenHierarchy expectedHi, TokenSequence actual, TokenHierarchy actualHi, boolean testLookaheadAndState)
| | public static void | assertTokenSequencesEqual(String message, TokenSequence expected, TokenHierarchy expectedHi, TokenSequence actual, TokenHierarchy actualHi, boolean testLookaheadAndState)
Compare contents of the given token sequences by moving through all their
tokens.
Token hierarchies are given to check implementations
of the Token.offset(TokenHierarchy) - useful for checking of token snapshots.
Parameters:
message - message to display (may be null).
Parameters:
expected - non-null token sequence to be compared to the other token sequence.
Parameters:
expectedHi - token hierarchy to which expected relates.
Parameters:
actual - non-null token sequence to be compared to the other token sequence.
Parameters:
actualHi - token hierarchy to which actual relates.
Parameters:
testLookaheadAndState - whether lookahead and states should be checkedor not. | | public static void | checkTokenDump(NbTestCase test, String relFilePath, Language language)
Check whether token descriptions dump file (a file with added suffix ".tokens.txt")
for the given input file exists and whether it has the same content
like the one obtained by lexing the input file.
It allows to test whether the tested lexer still produces the same tokens.
The method will only pass successfully if both the input file and token descriptions
files exist and the token descriptions file contains the same information
as the generated files.
If the token descriptions file does not exist the method will create it.
As the lexer's behavior at the EOF is important and should be well tested
there is a support for splitting input file virtually into multiple inputs
by virtual EOF - see TokenDumpTokenId for details.
Also there is possibility to specify special chars
- see TokenDumpTokenId for details.
Parameters:
test - non-null test (used for calling test.getDataDir()).
Parameters:
relFilePath - non-null file path relative to datadir of the test.
For example if "testfiles/testinput.mylang.txt" gets passed the test method willsearch for new File(test.getDataDir() + "testfiles/testinput.mylang.txt"),read its content, lex it and create token descriptions. | | public static boolean | collectionsEqual(Collection c1, Collection c2)
| | public static String | debugText(CharSequence text)
Return the given text as String
translating the special characters (and '\') into escape sequences.
Parameters:
text - non-null text to be debugged. | | public static String | debugTextOrNull(CharSequence text)
Return the given text as String
translating the special characters (and '\') into escape sequences.
Parameters:
text - non-null text to be debugged. | | public static List<Integer> | flyweightDistribution(TokenSequence ts)
Compute distribution of flyweight token lengths accross the given token sequence.
Parameters:
ts - non-null token sequence. | | public static int | flyweightTextLength(TokenSequence ts)
Compute total number of characters represented by flyweight tokens
in the given token sequence.
Parameters:
ts - non-null token sequence. | | public static int | flyweightTokenCount(TokenSequence ts)
Compute number of flyweight tokens in the given token sequence.
Parameters:
ts - non-null token sequence. | | public static DocumentEvent | getLastDocumentEvent(Document doc)
| | public static TokenHierarchyEvent | getLastTokenHierarchyEvent(Document doc)
| | public static void | incCheck(Document doc, boolean nested)
| | public static void | initLastDocumentEventListening(Document doc)
| | public static void | initLastTokenHierarchyEventListening(Document doc)
| | public static int | lookahead(TokenSequence ts)
Get lookahead for the token to which the token sequence is positioned. | | public static void | setTesting(boolean testing)
Set whether the lexer should run in testing mode where there are some
additional correctness checks performed. | | public static Object | state(TokenSequence ts)
Get state for the token to which the token sequence is positioned. | | public static boolean | textEquals(CharSequence text1, CharSequence text2)
Compare whether the two character sequences represent the same text. | | public static TokenList<T> | tokenList(TokenSequence<T> ts)
Get token list from the given token sequence for testing purposes. |
| assertTokenEquals | | public static void assertTokenEquals(String message, TokenSequence ts, TokenId id, String text, int offset)(Code) | | Compare TokenSequence.token() to the given
token id, text and offset.
Parameters:
offset - expected offset. It may be -1 to prevent offset testing. |
| assertTokenFlyweight | | public static void assertTokenFlyweight(Token token)(Code) | | |
| assertTokenNotFlyweight | | public static void assertTokenNotFlyweight(Token token)(Code) | | |
| assertTokenOffsetsEqual | | public static void assertTokenOffsetsEqual(String message, int offset1, int offset2)(Code) | | |
| assertTokenSequencesEqual | | public static void assertTokenSequencesEqual(String message, TokenSequence expected, TokenHierarchy expectedHi, TokenSequence actual, TokenHierarchy actualHi, boolean testLookaheadAndState)(Code) | | Compare contents of the given token sequences by moving through all their
tokens.
Token hierarchies are given to check implementations
of the Token.offset(TokenHierarchy) - useful for checking of token snapshots.
Parameters:
message - message to display (may be null).
Parameters:
expected - non-null token sequence to be compared to the other token sequence.
Parameters:
expectedHi - token hierarchy to which expected relates.
Parameters:
actual - non-null token sequence to be compared to the other token sequence.
Parameters:
actualHi - token hierarchy to which actual relates.
Parameters:
testLookaheadAndState - whether lookahead and states should be checkedor not. Generally it should be true but for snapshots checking it mustbe false because snapshots do not hold lookaheads and states. |
| checkTokenDump | | public static void checkTokenDump(NbTestCase test, String relFilePath, Language language) throws Exception(Code) | | Check whether token descriptions dump file (a file with added suffix ".tokens.txt")
for the given input file exists and whether it has the same content
like the one obtained by lexing the input file.
It allows to test whether the tested lexer still produces the same tokens.
The method will only pass successfully if both the input file and token descriptions
files exist and the token descriptions file contains the same information
as the generated files.
If the token descriptions file does not exist the method will create it.
As the lexer's behavior at the EOF is important and should be well tested
there is a support for splitting input file virtually into multiple inputs
by virtual EOF - see TokenDumpTokenId for details.
Also there is possibility to specify special chars
- see TokenDumpTokenId for details.
Parameters:
test - non-null test (used for calling test.getDataDir()).
Parameters:
relFilePath - non-null file path relative to datadir of the test.
For example if "testfiles/testinput.mylang.txt" gets passed the test method willsearch for new File(test.getDataDir() + "testfiles/testinput.mylang.txt"),read its content, lex it and create token descriptions. Then it will search for new File(test.getDataDir() + "testfiles/testinput.mylang.txt.tokens.txt")and it will compare the file content with the generated descriptions. |
| debugText | | public static String debugText(CharSequence text)(Code) | | Return the given text as String
translating the special characters (and '\') into escape sequences.
Parameters:
text - non-null text to be debugged. non-null string containing the debug text. |
| debugTextOrNull | | public static String debugTextOrNull(CharSequence text)(Code) | | Return the given text as String
translating the special characters (and '\') into escape sequences.
Parameters:
text - non-null text to be debugged. non-null string containing the debug text or "". |
| flyweightDistribution | | public static List<Integer> flyweightDistribution(TokenSequence ts)(Code) | | Compute distribution of flyweight token lengths accross the given token sequence.
Parameters:
ts - non-null token sequence. non-null list containing number of the flyweight tokens that have the lengthequal to the index in the list. |
| flyweightTextLength | | public static int flyweightTextLength(TokenSequence ts)(Code) | | Compute total number of characters represented by flyweight tokens
in the given token sequence.
Parameters:
ts - non-null token sequence. number of characters contained in the flyweight tokensin the token sequence. |
| flyweightTokenCount | | public static int flyweightTokenCount(TokenSequence ts)(Code) | | Compute number of flyweight tokens in the given token sequence.
Parameters:
ts - non-null token sequence. number of flyweight tokens in the token sequence. |
| incCheck | | public static void incCheck(Document doc, boolean nested)(Code) | | |
| initLastDocumentEventListening | | public static void initLastDocumentEventListening(Document doc)(Code) | | |
| initLastTokenHierarchyEventListening | | public static void initLastTokenHierarchyEventListening(Document doc)(Code) | | |
| lookahead | | public static int lookahead(TokenSequence ts)(Code) | | Get lookahead for the token to which the token sequence is positioned.
The method uses reflection to get reference to tokenList field in token sequence.
|
| setTesting | | public static void setTesting(boolean testing)(Code) | | Set whether the lexer should run in testing mode where there are some
additional correctness checks performed.
|
| state | | public static Object state(TokenSequence ts)(Code) | | Get state for the token to which the token sequence is positioned.
The method uses reflection to get reference to tokenList field in token sequence.
|
| textEquals | | public static boolean textEquals(CharSequence text1, CharSequence text2)(Code) | | Compare whether the two character sequences represent the same text.
|
|
|