| java.lang.Object
 fri.patterns.interpreter.parsergenerator.Token
| Token | | public class Token (Code) | | Lexer-Parser communication struct. Utility methods defining EPSILON and terminals.
Definition of all global constants for parsergenerator packages.
This class defines special token identifiers: "token" (Token.TOKEN) and
"ignored" (Token.IGNORED), which are needed when specifiying a syntax with
mixed parser and lexer rules.
This class defines special symbols needed to define character sets with Token.UPTO
(e.g. A..Z) and intersections with Token.BUTNOT (e.g. `char` - `newline`).
author:
(c) 2000, Fritz Ritzberger |
| Inner Class :public static class Address implements Comparable | |
| Inner Class :public static class Range implements Comparable | |
| Field Summary | |
| final public static String | ARTIFICIAL_NONTERMINAL_START_CHARACTER
The character used to mark artificial nonterminal (e.g. | | final public static String | BUTNOT
Symbol used to define lexer character set intersections, e.g. | | final public static char | CHAR_QUOTE
Delimiter for literal terminal. | | final public static char | COMMAND_QUOTE
Delimiter for lexer rules (terminal). | | final public static String | EPSILON
This special token symbol means "end of input" to the Parser, which stops calling getNextToken() then. | | final public static String | IGNORED
Reserved identifier that marks tokens not to deliver to parser, e.g. | | final public static char | STRING_QUOTE
Delimiter for literal terminals. | | final public static String | TOKEN
Reserved identifier that marks nonterminals the lexer should deliver to parser. | | final public static String | UPTO
Symbol used to define lexer character sets, e.g. | | final public Range | range
The start and end Address of this Token. | | final public String | symbol
The syntax symbol this Token represents. | | final public Object | text
The text that was scanned for this Token. |
| Method Summary | |
| public static boolean | isEpsilon(Token token)
Epsilon means end of input, EOF, no more bytes available. | | public static boolean | isEpsilon(String symbol)
Epsilon means end of input, no more bytes available. | | public static boolean | isTerminal(String symbol)
Distinction of 'terminals' and nonterminals:
terminals are either starting with digit or are enclosed in quotes '"` or equal to EPSILON. |
| ARTIFICIAL_NONTERMINAL_START_CHARACTER | | final public static String ARTIFICIAL_NONTERMINAL_START_CHARACTER(Code) | | The character used to mark artificial nonterminal (e.g. made from "prolog?"). Every artificial rule starts with it.
|
| BUTNOT | | final public static String BUTNOT(Code) | | Symbol used to define lexer character set intersections, e.g. char - "/*" - "//".
|
| CHAR_QUOTE | | final public static char CHAR_QUOTE(Code) | | Delimiter for literal terminal.
|
| COMMAND_QUOTE | | final public static char COMMAND_QUOTE(Code) | | Delimiter for lexer rules (terminal).
|
| EPSILON | | final public static String EPSILON(Code) | | This special token symbol means "end of input" to the Parser, which stops calling getNextToken() then.
|
| IGNORED | | final public static String IGNORED(Code) | | Reserved identifier that marks tokens not to deliver to parser, e.g. ignored ::= spaces ;.
|
| STRING_QUOTE | | final public static char STRING_QUOTE(Code) | | Delimiter for literal terminals.
|
| TOKEN | | final public static String TOKEN(Code) | | Reserved identifier that marks nonterminals the lexer should deliver to parser.
|
| UPTO | | final public static String UPTO(Code) | | Symbol used to define lexer character sets, e.g. a..z.
|
| range | | final public Range range(Code) | | The start and end Address of this Token.
|
| symbol | | final public String symbol(Code) | | The syntax symbol this Token represents.
|
| text | | final public Object text(Code) | | The text that was scanned for this Token.
|
| isEpsilon | | public static boolean isEpsilon(Token token)(Code) | | Epsilon means end of input, EOF, no more bytes available.
true if passed token is the EPSILON-symbol. |
| isEpsilon | | public static boolean isEpsilon(String symbol)(Code) | | Epsilon means end of input, no more bytes available.
true if Token symbol not null and is the EPSILON-symbol. |
| isTerminal | | public static boolean isTerminal(String symbol)(Code) | | Distinction of 'terminals' and nonterminals:
terminals are either starting with digit or are enclosed in quotes '"` or equal to EPSILON.
|
|
|