Java Doc for WikipediaTokenizerImpl.java in  » Net » lucene-connector » org » apache » lucene » wikipedia » analysis » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Java Source Code / Java Documentation 1.  6.0 JDK Core 2.  6.0 JDK Modules 3.  6.0 JDK Modules com.sun 4.  6.0 JDK Modules com.sun.java 5.  6.0 JDK Modules sun 6.  6.0 JDK Platform 7.  Ajax 8.  Apache Harmony Java SE 9.  Aspect oriented 10.  Authentication Authorization 11.  Blogger System 12.  Build 13.  Byte Code 14.  Cache 15.  Chart 16.  Chat 17.  Code Analyzer 18.  Collaboration 19.  Content Management System 20.  Database Client 21.  Database DBMS 22.  Database JDBC Connection Pool 23.  Database ORM 24.  Development 25.  EJB Server geronimo 26.  EJB Server GlassFish 27.  EJB Server JBoss 4.2.1 28.  EJB Server resin 3.1.5 29.  ERP CRM Financial 30.  ESB 31.  Forum 32.  GIS 33.  Graphic Library 34.  Groupware 35.  HTML Parser 36.  IDE 37.  IDE Eclipse 38.  IDE Netbeans 39.  Installer 40.  Internationalization Localization 41.  Inversion of Control 42.  Issue Tracking 43.  J2EE 44.  JBoss 45.  JMS 46.  JMX 47.  Library 48.  Mail Clients 49.  Net 50.  Parser 51.  PDF 52.  Portal 53.  Profiler 54.  Project Management 55.  Report 56.  RSS RDF 57.  Rule Engine 58.  Science 59.  Scripting 60.  Search Engine 61.  Security 62.  Sevlet Container 63.  Source Control 64.  Swing Library 65.  Template Engine 66.  Test Coverage 67.  Testing 68.  UML 69.  Web Crawler 70.  Web Framework 71.  Web Mail 72.  Web Server 73.  Web Services 74.  Web Services apache cxf 2.0.1 75.  Web Services AXIS2 76.  Wiki Engine 77.  Workflow Engines 78.  XML 79.  XML UI Java Java Tutorial Java Open Source Jar File Download Java Articles Java Products Java by API Photoshop Tutorials Maya Tutorials Flash Tutorials 3ds-Max Tutorials Illustrator Tutorials GIMP Tutorials C# / C Sharp C# / CSharp Tutorial C# / CSharp Open Source ASP.Net ASP.NET Tutorial JavaScript DHTML JavaScript Tutorial JavaScript Reference HTML / CSS HTML CSS Reference C / ANSI-C C Tutorial C++ C++ Tutorial Ruby PHP Python Python Tutorial Python Open Source SQL Server / T-SQL SQL Server / T-SQL Tutorial Oracle PL / SQL Oracle PL/SQL Tutorial PostgreSQL SQL / MySQL MySQL Tutorial VB.Net VB.Net Tutorial Flash / Flex / ActionScript VBA / Excel / Access / Word XML XML Tutorial Microsoft Office PowerPoint 2007 Tutorial Microsoft Office Excel 2007 Tutorial Microsoft Office Word 2007 Tutorial

Java Source Code / Java Documentation » Net » lucene connector » org.apache.lucene.wikipedia.analysis   Source Cross Reference   Class Diagram   Java Document (Java Doc)   java.lang.Object    org.apache.lucene.wikipedia.analysis.WikipediaTokenizerImpl WikipediaTokenizerImpl class WikipediaTokenizerImpl (Code) This class is a scanner generated by JFlex 1.4.1 on 1/4/08 3:30 PM from the specification file /Volumes/User/grantingersoll/projects/lucene/Lucene-Trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizerImpl.jflex Field Summary final public static  int ACRONYM      final public static  int ALPHANUM      final public static  int APOSTROPHE      final public static  int BOLD      final public static  int BOLD_ITALICS      final public static  int CATEGORY      final public static  int CATEGORY_STATE      final public static  int CITATION      final public static  int CJ      final public static  int COMPANY      final public static  int DOUBLE_BRACE_STATE      final public static  int DOUBLE_EQUALS_STATE      final public static  int EMAIL      final public static  int EXTERNAL_LINK      final public static  int EXTERNAL_LINK_STATE      final public static  int EXTERNAL_LINK_URL      final public static  int FIVE_SINGLE_QUOTES_STATE      final public static  int HEADING      final public static  int HOST      final public static  int INTERNAL_LINK      final public static  int INTERNAL_LINK_STATE      final public static  int ITALICS      final public static  int NUM      final public static  int STRING      final public static  int SUB_HEADING      final public static  int THREE_SINGLE_QUOTES_STATE      final public static  String[] TOKEN_TYPES      final public static  int TWO_SINGLE_QUOTES_STATE      final public static  int YYEOF      final public static  int YYINITIAL      Constructor Summary   WikipediaTokenizerImpl(java.io.Reader in)      Creates a new scanner There is also a java.io.InputStream version of this constructor.   WikipediaTokenizerImpl(java.io.InputStream in)      Creates a new scanner. Method Summary public  int getNextToken()      Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs. final public  int getPositionIncrement()      final  void getText(Token t, int tokType)      Fills Lucene token with the current token text. final public  void yybegin(int newState)      final public  int yychar()      final public  char yycharat(int pos)      Returns the character at position pos from the matched text. final public  void yyclose()      Closes the input stream. final public  int yylength()      Returns the length of the matched text region. public  void yypushback(int number)      Pushes the specified amount of characters back into the input stream. final public  void yyreset(java.io.Reader reader)      Resets the scanner to read from a new input stream. final public  int yystate()      Returns the current lexical state. final public  String yytext()      Returns the text matched by the current regular expression. Field Detail ACRONYM final public static int ACRONYM(Code) ALPHANUM final public static int ALPHANUM(Code) APOSTROPHE final public static int APOSTROPHE(Code) BOLD final public static int BOLD(Code) BOLD_ITALICS final public static int BOLD_ITALICS(Code) CATEGORY final public static int CATEGORY(Code) CATEGORY_STATE final public static int CATEGORY_STATE(Code) CITATION final public static int CITATION(Code) CJ final public static int CJ(Code) COMPANY final public static int COMPANY(Code) DOUBLE_BRACE_STATE final public static int DOUBLE_BRACE_STATE(Code) lexical states DOUBLE_EQUALS_STATE final public static int DOUBLE_EQUALS_STATE(Code) EMAIL final public static int EMAIL(Code) EXTERNAL_LINK final public static int EXTERNAL_LINK(Code) EXTERNAL_LINK_STATE final public static int EXTERNAL_LINK_STATE(Code) EXTERNAL_LINK_URL final public static int EXTERNAL_LINK_URL(Code) FIVE_SINGLE_QUOTES_STATE final public static int FIVE_SINGLE_QUOTES_STATE(Code) HEADING final public static int HEADING(Code) HOST final public static int HOST(Code) INTERNAL_LINK final public static int INTERNAL_LINK(Code) INTERNAL_LINK_STATE final public static int INTERNAL_LINK_STATE(Code) ITALICS final public static int ITALICS(Code) NUM final public static int NUM(Code) STRING final public static int STRING(Code) SUB_HEADING final public static int SUB_HEADING(Code) THREE_SINGLE_QUOTES_STATE final public static int THREE_SINGLE_QUOTES_STATE(Code) TOKEN_TYPES final public static String[] TOKEN_TYPES(Code) TWO_SINGLE_QUOTES_STATE final public static int TWO_SINGLE_QUOTES_STATE(Code) YYEOF final public static int YYEOF(Code) This character denotes the end of file YYINITIAL final public static int YYINITIAL(Code) Constructor Detail WikipediaTokenizerImpl WikipediaTokenizerImpl(java.io.Reader in)(Code) Creates a new scanner There is also a java.io.InputStream version of this constructor. Parameters:   in - the java.io.Reader to read input from. WikipediaTokenizerImpl WikipediaTokenizerImpl(java.io.InputStream in)(Code) Creates a new scanner. There is also java.io.Reader version of this constructor. Parameters:   in - the java.io.Inputstream to read input from. Method Detail getNextToken public int getNextToken() throws java.io.IOException(Code) Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs. the next token exception:   java.io.IOException - if any I/O-Error occurs getPositionIncrement final public int getPositionIncrement()(Code) getText final void getText(Token t, int tokType)(Code) Fills Lucene token with the current token text. yybegin final public void yybegin(int newState)(Code) Enters a new lexical state Parameters:   newState - the new lexical state yychar final public int yychar()(Code) yycharat final public char yycharat(int pos)(Code) Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster Parameters:   pos - the position of the character to fetch. A value from 0 to yylength()-1. the character at position pos yyclose final public void yyclose() throws java.io.IOException(Code) Closes the input stream. yylength final public int yylength()(Code) Returns the length of the matched text region. yypushback public void yypushback(int number)(Code) Pushes the specified amount of characters back into the input stream. They will be read again by then next call of the scanning method Parameters:   number - the number of characters to be read again.This number must not be greater than yylength()! yyreset final public void yyreset(java.io.Reader reader)(Code) Resets the scanner to read from a new input stream. Does not close the old reader. All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL. Parameters:   reader - the new input stream yystate final public int yystate()(Code) Returns the current lexical state. yytext final public String yytext()(Code) Returns the text matched by the current regular expression. Methods inherited from java.lang.Object native protected Object clone() throws CloneNotSupportedException(Code)(Java Doc) public boolean equals(Object obj)(Code)(Java Doc) protected void finalize() throws Throwable(Code)(Java Doc) final native public Class getClass()(Code)(Java Doc) native public int hashCode()(Code)(Java Doc) final native public void notify()(Code)(Java Doc) final native public void notifyAll()(Code)(Java Doc) public String toString()(Code)(Java Doc) final native public void wait(long timeout) throws InterruptedException(Code)(Java Doc) final public void wait(long timeout, int nanos) throws InterruptedException(Code)(Java Doc) final public void wait() throws InterruptedException(Code)(Java Doc)