| fri.patterns.interpreter.parsergenerator.Lexer
All known Subclasses: fri.patterns.interpreter.parsergenerator.lexer.LexerImpl,
| Lexer | | public interface Lexer (Code) | | A Lexer reads input bytes (InputStream) or characters (Reader), until
one of its terminals is fulfilled. This happens when the Parser calls
getNextToken(). The terminals will be set by the Parser on init.
Usage:
SyntaxSeparation separation = new SyntaxSeparation(new Syntax(myRules));
LexerBuilder builder = new LexerBuilder(separation.getLexerSyntax(), separation.getIgnoredSymbols());
Lexer lexer = builder.getLexer(inputStream);
lexer.setTerminals(separation.getTokenSymbols());
Token token;
do {
token = lexer.getNextToken(null);
System.err.println("token.symbol="+token.symbol+", text >"+token.text+"<");
}
while (token.symbol != null && Token.isEpsilon(token) == false);
boolean error = token.symbol == null;
See Also: fri.patterns.interpreter.parsergenerator.lexer.LexerImpl
author:
(c) 2000, Fritz Ritzberger |
| Inner Class :public interface TokenListener | |
| addTokenListener | | public void addTokenListener(Lexer.TokenListener listener)(Code) | | Installs a TokenListener that wants to know about every read Token, even it is ignored.
Parameters:
listener - the Lexer.Listener implementation to install. |
| clear | | public void clear()(Code) | | Reset the Lexer for another pass.
|
| dump | | public void dump(PrintStream out)(Code) | | Dump the current text and the scan position.
|
| getNextToken | | public Token getNextToken(Map tokenSymbols) throws IOException(Code) | | Returns the next token from input. This is done trying to satisy the parser hints, or
by using contained character consumers, which are obtained by the lexer strategy.
Parameters:
tokenSymbols - a Map that contains token symbols (in "key" field) expected by the Parser, can be null (slower). a Token with a terminal symbol and its instance text, or a Token with null symbol for error. |
| removeTokenListener | | public void removeTokenListener(Lexer.TokenListener listener)(Code) | | Removes a TokenListener from this Lexer.
Parameters:
listener - the Lexer.Listener implementation to remove. |
| setDebug | | public void setDebug(boolean debug)(Code) | | Turn on and off debug mode.
|
| setInput | | public void setInput(Object text) throws IOException(Code) | | Set the input to be processed by the Lexer.
Parameters:
text - can be String, StringBuffer, File, InputStream, Reader. |
| setTerminals | | public void setTerminals(List terminals)(Code) | | Tells the Lexer the terminals (tokens) to scan, called on init. Every terminal is a String
that satisfies the facts defined in Token.isTerminal() (EPSILON or delimited by quotes).
Parameters:
terminals - List of String containing all terminals of the parser syntax. |
|
|