| |
|
| org.antlr.tool.AssignTokenTypesWalker
AssignTokenTypesWalker | public class AssignTokenTypesWalker extends antlr.TreeParser implements AssignTokenTypesWalkerTokenTypes(Code) | | [Warning: TJP says that this is probably out of date as of 11/19/2005,
but since it's probably still useful, I'll leave in. Don't have energy
to update at the moment.]
Compute the token types for all literals and rules etc.. There are
a few different cases to consider for grammar types and a few situations
within.
CASE 1 : pure parser grammar
a) Any reference to a token gets a token type.
b) The tokens section may alias a token name to a string or char
CASE 2 : pure lexer grammar
a) Import token vocabulary if available. Set token types for any new tokens
to values above last imported token type
b) token rule definitions get token types if not already defined
c) literals do NOT get token types
CASE 3 : merged parser / lexer grammar
a) Any char or string literal gets a token type in a parser rule
b) Any reference to a token gets a token type if not referencing
a fragment lexer rule
c) The tokens section may alias a token name to a string or char
which must add a rule to the lexer
d) token rule definitions get token types if not already defined
e) token rule definitions may also alias a token name to a literal.
E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that
references to either in the parser grammar will yield the token type
What this pass does:
0. Collects basic info about the grammar like grammar name and type;
Oh, I have go get the options in case they affect the token types.
E.g., tokenVocab option.
Imports any token vocab name/type pairs into a local hashtable.
1. Finds a list of all literals and token names.
2. Finds a list of all token name rule definitions;
no token rules implies pure parser.
3. Finds a list of all simple token rule defs of form " : ;"
and aliases them.
4. Walks token names table and assign types to any unassigned
5. Walks aliases and assign types to referenced literals
6. Walks literals, assigning types if untyped
4. Informs the Grammar object of the type definitions such as:
g.defineToken(, ttype);
g.defineToken(, ttype);
g.defineToken(, ttype);
where some of the ttype values will be the same for aliases tokens.
|
UNASSIGNED_IN_PARSER_RULE | final protected static Integer UNASSIGNED_IN_PARSER_RULE(Code) | | |
stringLiterals | protected Map stringLiterals(Code) | | |
tokenRuleDefs | protected Set tokenRuleDefs(Code) | | Track actual lexer rule defs so we don't get repeated token defs in
generated lexer.
|
AssignTokenTypesWalker | public AssignTokenTypesWalker()(Code) | | |
aliasTokenIDsAndLiterals | protected void aliasTokenIDsAndLiterals()(Code) | | |
alternative | final public void alternative(AST _t) throws RecognitionException(Code) | | |
assignStringTypes | protected void assignStringTypes()(Code) | | |
assignTokenIDTypes | protected void assignTokenIDTypes()(Code) | | |
assignTypes | protected void assignTypes()(Code) | | |
ast_suffix | final public void ast_suffix(AST _t) throws RecognitionException(Code) | | |
atom | final public void atom(AST _t) throws RecognitionException(Code) | | |
attrScope | final public void attrScope(AST _t) throws RecognitionException(Code) | | |
block | final public void block(AST _t) throws RecognitionException(Code) | | |
charSet | final public void charSet(AST _t) throws RecognitionException(Code) | | |
charSetElement | final public void charSetElement(AST _t) throws RecognitionException(Code) | | |
ebnf | final public void ebnf(AST _t) throws RecognitionException(Code) | | |
element | final public void element(AST _t) throws RecognitionException(Code) | | |
exceptionGroup | final public void exceptionGroup(AST _t) throws RecognitionException(Code) | | |
exceptionHandler | final public void exceptionHandler(AST _t) throws RecognitionException(Code) | | |
finallyClause | final public void finallyClause(AST _t) throws RecognitionException(Code) | | |
grammar | final public void grammar(AST _t, Grammar g) throws RecognitionException(Code) | | |
grammarSpec | final public void grammarSpec(AST _t) throws RecognitionException(Code) | | |
modifier | final public void modifier(AST _t) throws RecognitionException(Code) | | |
notifyGrammarObject | protected void notifyGrammarObject()(Code) | | |
option | final public void option(AST _t, Map opts) throws RecognitionException(Code) | | |
optionValue | final public Object optionValue(AST _t) throws RecognitionException(Code) | | |
optionsSpec | final public Map optionsSpec(AST _t) throws RecognitionException(Code) | | |
reportError | public void reportError(RecognitionException ex)(Code) | | |
rewrite | final public void rewrite(AST _t) throws RecognitionException(Code) | | |
rule | final public void rule(AST _t) throws RecognitionException(Code) | | |
ruleScopeSpec | final public void ruleScopeSpec(AST _t) throws RecognitionException(Code) | | |
rules | final public void rules(AST _t) throws RecognitionException(Code) | | |
tokenSpec | final public void tokenSpec(AST _t) throws RecognitionException(Code) | | |
tokensSpec | final public void tokensSpec(AST _t) throws RecognitionException(Code) | | |
trackString | protected void trackString(GrammarAST t)(Code) | | Track string literals in any non-lexer rule (could be in tokens{} section)
|
tree | final public void tree(AST _t) throws RecognitionException(Code) | | |
|
|
|