| java.lang.Object
 org.antlr.codegen.CodeGenerator
| CodeGenerator | | public class CodeGenerator (Code) | | ANTLR's code generator.
Generate recognizers derived from grammars. Language independence
achieved through the use of StringTemplateGroup objects. All output
strings are completely encapsulated in the group files such as Java.stg.
Some computations are done that are unused by a particular language.
This generator just computes and sets the values into the templates;
the templates are free to use or not use the information.
To make a new code generation target, define X.stg for language X
by copying from existing Y.stg most closely releated to your language;
e.g., to do CSharp.stg copy Java.stg. The template group file has a
bunch of templates that are needed by the code generator. You can add
a new target w/o even recompiling ANTLR itself. The language=X option
in a grammar file dictates which templates get loaded/used.
Some language like C need both parser files and header files. Java needs
to have a separate file for the cyclic DFA as ANTLR generates bytecodes
directly (which cannot be in the generated parser Java file). To facilitate
this,
cyclic can be in same file, but header, output must be searpate. recognizer
is in outptufile.
|
| Method Summary | |
| protected boolean | canGenerateSwitch(DFAState s)
You can generate a switch rather than if-then-else for a DFA state
if there are no semantic predicates and the number of edge label
values is small enough; e.g., don't generate a switch for a state
containing an edge label such as 20..52330 (the resulting byte codes
would overflow the method 65k limit probably). | | public String | createUniqueLabel(String name)
Create a label to track a token / rule reference's result. | | protected StringTemplate | genLabelExpr(StringTemplateGroup templates, Transition edge, int k)
Generate an expression for traversing an edge. | | public StringTemplate | genLookaheadDecision(StringTemplate recognizerST, DFA dfa)
Generate code that computes the predicted alt given a DFA. | | public StringTemplate | genRecognizer()
Given the grammar to which we are attached, walk the AST associated
with that grammar to create NFAs. | | protected StringTemplate | genSemanticPredicateExpr(StringTemplateGroup templates, Transition edge)
| | public StringTemplate | genSetExpr(StringTemplateGroup templates, IntSet set, int k, boolean partOfDFA)
| | protected void | genTokenTypeConstants(StringTemplate code)
Set attributes tokens and literals attributes in the incoming
code template. | | protected void | genTokenTypeNames(StringTemplate code)
Generate a token names table that maps token type to a printable
name: either the label like INT or the literal like "begin". | | protected StringTemplate | genTokenVocabOutput()
Generate a token vocab file with all the token names/types. | | public void | generateLocalFOLLOW(GrammarAST referencedElementNode, String referencedElementName, String enclosingRuleName, int elementIndex)
Error recovery in ANTLR recognizers. | | public StringTemplate | generateSpecialState(DFAState s)
A special state is huge (too big for state tables) or has a predicated
edge. | | public StringTemplateGroup | getBaseTemplates()
| | public String | getRecognizerFileName(String name, int type)
| | public StringTemplate | getRecognizerST()
| | public StringTemplateGroup | getTemplates()
| | public String | getTokenTypeAsTargetLabel(int ttype)
Get a meaningful name for a token type useful during code generation.
Literals without associated names are converted to the string equivalent
of their integer values. | | public String | getVocabFileName()
What is the name of the vocab file generated for this grammar?
Returns null if no .tokens file should be generated. | | public void | issueInvalidAttributeError(String x, String y, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)
| | public void | issueInvalidAttributeError(String x, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)
| | public void | issueInvalidScopeError(String x, String y, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)
| | protected void | loadLanguageTarget(String language)
| | public void | loadTemplates(String language)
| | public void | setDebug(boolean debug)
| | public void | setProfile(boolean profile)
| | public void | setTrace(boolean trace)
| | public List | translateAction(String ruleName, GrammarAST actionTree)
| | protected void | translateActionAttributeReferences(Map actions)
Actions may reference $x::y attributes, call translateAction on
each action and replace that action in the Map. | | protected void | translateActionAttributeReferencesForSingleScope(Rule r, Map scopeActions)
| | public List | translateArgAction(String ruleName, GrammarAST actionTree)
Translate an action like [3,"foo",a[3]] and return a List of the
translated actions. | | public StringTemplate | translateTemplateConstructor(String ruleName, int outerAltNum, antlr.Token actionToken, String templateActionText)
Given a template constructor action like %foo(a={...}) in
an action, translate it to the appropriate template constructor
from the templateLib. | | protected void | verifyActionScopesOkForTarget(Map actions)
Some targets will have some extra scopes like C++ may have
'@headerfile:name {action}' or something. | | public void | write(StringTemplate code, String fileName)
|
| EMIT_TEMPLATE_DELIMITERS | | public static boolean EMIT_TEMPLATE_DELIMITERS(Code) | | |
| GENERATE_SWITCHES_WHEN_POSSIBLE | | public boolean GENERATE_SWITCHES_WHEN_POSSIBLE(Code) | | |
| GEN_ACYCLIC_DFA_INLINE | | public static boolean GEN_ACYCLIC_DFA_INLINE(Code) | | |
| MAX_SWITCH_CASE_LABELS | | public int MAX_SWITCH_CASE_LABELS(Code) | | When generating SWITCH statements, some targets might need to limit
the size (based upon the number of case labels). Generally, this
limit will be hit only for lexers where wildcard in a UNICODE
vocabulary environment would generate a SWITCH with 65000 labels.
|
| MIN_SWITCH_ALTS | | public int MIN_SWITCH_ALTS(Code) | | |
| VOCAB_FILE_EXTENSION | | final public static String VOCAB_FILE_EXTENSION(Code) | | I have factored out the generation of cyclic DFAs to separate class
|
| baseTemplates | | protected StringTemplateGroup baseTemplates(Code) | | The basic output templates without AST or templates stuff; this will be
the templates loaded for the language such as Java.stg *and* the Dbg
stuff if turned on. This is used for generating syntactic predicates.
|
| classpathTemplateRootDirectoryName | | public String classpathTemplateRootDirectoryName(Code) | | |
| debug | | protected boolean debug(Code) | | Generate debugging event method calls
|
| grammar | | public Grammar grammar(Code) | | Which grammar are we generating code for? Each generator
is attached to a specific grammar.
|
| headerFileST | | protected StringTemplate headerFileST(Code) | | |
| language | | protected String language(Code) | | What language are we generating?
|
| lineWidth | | protected int lineWidth(Code) | | |
| outputFileST | | protected StringTemplate outputFileST(Code) | | |
| profile | | protected boolean profile(Code) | | Track runtime parsing information about decisions etc...
This requires the debugging event mechanism to work.
|
| recognizerST | | protected StringTemplate recognizerST(Code) | | |
| target | | public Target target(Code) | | The target specifies how to write out files and do other language
specific actions.
|
| templates | | protected StringTemplateGroup templates(Code) | | Where are the templates this generator should use to generate code?
|
| tool | | protected Tool tool(Code) | | A reference to the ANTLR tool so we can learn about output directories
and such.
|
| trace | | protected boolean trace(Code) | | Create a Tracer object and make the recognizer invoke this.
|
| uniqueLabelNumber | | protected int uniqueLabelNumber(Code) | | Used to create unique labels
|
| vocabFilePattern | | final protected static String vocabFilePattern(Code) | | |
| canGenerateSwitch | | protected boolean canGenerateSwitch(DFAState s)(Code) | | You can generate a switch rather than if-then-else for a DFA state
if there are no semantic predicates and the number of edge label
values is small enough; e.g., don't generate a switch for a state
containing an edge label such as 20..52330 (the resulting byte codes
would overflow the method 65k limit probably).
|
| createUniqueLabel | | public String createUniqueLabel(String name)(Code) | | Create a label to track a token / rule reference's result.
Technically, this is a place where I break model-view separation
as I am creating a variable name that could be invalid in a
target language, however, label ::= is probably ok in
all languages we care about.
|
| genLabelExpr | | protected StringTemplate genLabelExpr(StringTemplateGroup templates, Transition edge, int k)(Code) | | Generate an expression for traversing an edge.
|
| genLookaheadDecision | | public StringTemplate genLookaheadDecision(StringTemplate recognizerST, DFA dfa)(Code) | | Generate code that computes the predicted alt given a DFA. The
recognizerST can be either the main generated recognizerTemplate
for storage in the main parser file or a separate file. It's up to
the code that ultimately invokes the codegen.g grammar rule.
Regardless, the output file and header file get a copy of the DFAs.
|
| genRecognizer | | public StringTemplate genRecognizer()(Code) | | Given the grammar to which we are attached, walk the AST associated
with that grammar to create NFAs. Then create the DFAs for all
decision points in the grammar by converting the NFAs to DFAs.
Finally, walk the AST again to generate code.
Either 1 or 2 files are written:
recognizer: the main parser/lexer/treewalker item
header file: language like C/C++ need extern definitions
The target, such as JavaTarget, dictates which files get written.
|
| genSemanticPredicateExpr | | protected StringTemplate genSemanticPredicateExpr(StringTemplateGroup templates, Transition edge)(Code) | | |
| genSetExpr | | public StringTemplate genSetExpr(StringTemplateGroup templates, IntSet set, int k, boolean partOfDFA)(Code) | | For intervals such as [3..3, 30..35], generate an expression that
tests the lookahead similar to LA(1)==3 || (LA(1)>=30&&LA(1)<=35)
|
| genTokenTypeConstants | | protected void genTokenTypeConstants(StringTemplate code)(Code) | | Set attributes tokens and literals attributes in the incoming
code template. This is not the token vocab interchange file, but
rather a list of token type ID needed by the recognizer.
|
| genTokenTypeNames | | protected void genTokenTypeNames(StringTemplate code)(Code) | | Generate a token names table that maps token type to a printable
name: either the label like INT or the literal like "begin".
|
| genTokenVocabOutput | | protected StringTemplate genTokenVocabOutput()(Code) | | Generate a token vocab file with all the token names/types. For example:
ID=7
FOR=8
'for'=8
This is independent of the target language; used by antlr internally
|
| generateLocalFOLLOW | | public void generateLocalFOLLOW(GrammarAST referencedElementNode, String referencedElementName, String enclosingRuleName, int elementIndex)(Code) | | Error recovery in ANTLR recognizers.
Based upon original ideas:
Algorithms + Data Structures = Programs by Niklaus Wirth
and
A note on error recovery in recursive descent parsers:
http://portal.acm.org/citation.cfm?id=947902.947905
Later, Josef Grosch had some good ideas:
Efficient and Comfortable Error Recovery in Recursive Descent Parsers:
ftp://www.cocolab.com/products/cocktail/doca4.ps/ell.ps.zip
Like Grosch I implemented local FOLLOW sets that are combined at run-time
upon error to avoid parsing overhead.
|
| generateSpecialState | | public StringTemplate generateSpecialState(DFAState s)(Code) | | A special state is huge (too big for state tables) or has a predicated
edge. Generate a simple if-then-else. Cannot be an accept state as
they have no emanating edges. Don't worry about switch vs if-then-else
because if you get here, the state is super complicated and needs an
if-then-else. This is used by the new DFA scheme created June 2006.
|
| getBaseTemplates | | public StringTemplateGroup getBaseTemplates()(Code) | | |
| getRecognizerST | | public StringTemplate getRecognizerST()(Code) | | |
| getTemplates | | public StringTemplateGroup getTemplates()(Code) | | |
| getTokenTypeAsTargetLabel | | public String getTokenTypeAsTargetLabel(int ttype)(Code) | | Get a meaningful name for a token type useful during code generation.
Literals without associated names are converted to the string equivalent
of their integer values. Used to generate x==ID and x==34 type comparisons
etc... Essentially we are looking for the most obvious way to refer
to a token type in the generated code. If in the lexer, return the
char literal translated to the target language. For example, ttype=10
will yield '\n' from the getTokenDisplayName method. That must
be converted to the target languages literals. For most C-derived
languages no translation is needed.
|
| getVocabFileName | | public String getVocabFileName()(Code) | | What is the name of the vocab file generated for this grammar?
Returns null if no .tokens file should be generated.
|
| issueInvalidAttributeError | | public void issueInvalidAttributeError(String x, String y, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)(Code) | | |
| issueInvalidAttributeError | | public void issueInvalidAttributeError(String x, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)(Code) | | |
| issueInvalidScopeError | | public void issueInvalidScopeError(String x, String y, Rule enclosingRule, antlr.Token actionToken, int outerAltNum)(Code) | | |
| loadLanguageTarget | | protected void loadLanguageTarget(String language)(Code) | | |
| loadTemplates | | public void loadTemplates(String language)(Code) | | load the main language.stg template group file
|
| setDebug | | public void setDebug(boolean debug)(Code) | | |
| setProfile | | public void setProfile(boolean profile)(Code) | | |
| setTrace | | public void setTrace(boolean trace)(Code) | | |
| translateActionAttributeReferences | | protected void translateActionAttributeReferences(Map actions)(Code) | | Actions may reference $x::y attributes, call translateAction on
each action and replace that action in the Map.
|
| translateActionAttributeReferencesForSingleScope | | protected void translateActionAttributeReferencesForSingleScope(Rule r, Map scopeActions)(Code) | | Use for translating rule @init{...} actions that have no scope
|
| translateArgAction | | public List translateArgAction(String ruleName, GrammarAST actionTree)(Code) | | Translate an action like [3,"foo",a[3]] and return a List of the
translated actions. Because actions are translated to a list of
chunks, this returns List>.
Simple ',' separator is assumed.
|
| translateTemplateConstructor | | public StringTemplate translateTemplateConstructor(String ruleName, int outerAltNum, antlr.Token actionToken, String templateActionText)(Code) | | Given a template constructor action like %foo(a={...}) in
an action, translate it to the appropriate template constructor
from the templateLib. This translates a *piece* of the action.
|
| verifyActionScopesOkForTarget | | protected void verifyActionScopesOkForTarget(Map actions)(Code) | | Some targets will have some extra scopes like C++ may have
'@headerfile:name {action}' or something. Make sure the
target likes the scopes in action table.
|
|
|