getTokens() Get the tokens in the internal List.
WARNING: Adding tokens to this list requires the
SinkTokenizer.reset() method to be called in order for them
to be made available.
Override this method to cache only certain tokens, or new tokens based
on the old tokens.
Parameters: t - The org.apache.lucene.analysis.Token to add to the sink
Reset the internal data structures to the start at the front of the list of tokens. Should be called
if tokens were added to the list after an invocation of
SinkTokenizer.next(Token) throws: IOException -
Fields inherited from org.apache.lucene.analysis.Tokenizer