Java Doc for WikipediaTokenizerImpl.java in  » Net » lucene-connector » org » apache » lucene » wikipedia » analysis » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Java Source Code / Java Documentation
1. 6.0 JDK Core
2. 6.0 JDK Modules
3. 6.0 JDK Modules com.sun
4. 6.0 JDK Modules com.sun.java
5. 6.0 JDK Modules sun
6. 6.0 JDK Platform
7. Ajax
8. Apache Harmony Java SE
9. Aspect oriented
10. Authentication Authorization
11. Blogger System
12. Build
13. Byte Code
14. Cache
15. Chart
16. Chat
17. Code Analyzer
18. Collaboration
19. Content Management System
20. Database Client
21. Database DBMS
22. Database JDBC Connection Pool
23. Database ORM
24. Development
25. EJB Server geronimo
26. EJB Server GlassFish
27. EJB Server JBoss 4.2.1
28. EJB Server resin 3.1.5
29. ERP CRM Financial
30. ESB
31. Forum
32. GIS
33. Graphic Library
34. Groupware
35. HTML Parser
36. IDE
37. IDE Eclipse
38. IDE Netbeans
39. Installer
40. Internationalization Localization
41. Inversion of Control
42. Issue Tracking
43. J2EE
44. JBoss
45. JMS
46. JMX
47. Library
48. Mail Clients
49. Net
50. Parser
51. PDF
52. Portal
53. Profiler
54. Project Management
55. Report
56. RSS RDF
57. Rule Engine
58. Science
59. Scripting
60. Search Engine
61. Security
62. Sevlet Container
63. Source Control
64. Swing Library
65. Template Engine
66. Test Coverage
67. Testing
68. UML
69. Web Crawler
70. Web Framework
71. Web Mail
72. Web Server
73. Web Services
74. Web Services apache cxf 2.0.1
75. Web Services AXIS2
76. Wiki Engine
77. Workflow Engines
78. XML
79. XML UI
Java
Java Tutorial
Java Open Source
Jar File Download
Java Articles
Java Products
Java by API
Photoshop Tutorials
Maya Tutorials
Flash Tutorials
3ds-Max Tutorials
Illustrator Tutorials
GIMP Tutorials
C# / C Sharp
C# / CSharp Tutorial
C# / CSharp Open Source
ASP.Net
ASP.NET Tutorial
JavaScript DHTML
JavaScript Tutorial
JavaScript Reference
HTML / CSS
HTML CSS Reference
C / ANSI-C
C Tutorial
C++
C++ Tutorial
Ruby
PHP
Python
Python Tutorial
Python Open Source
SQL Server / T-SQL
SQL Server / T-SQL Tutorial
Oracle PL / SQL
Oracle PL/SQL Tutorial
PostgreSQL
SQL / MySQL
MySQL Tutorial
VB.Net
VB.Net Tutorial
Flash / Flex / ActionScript
VBA / Excel / Access / Word
XML
XML Tutorial
Microsoft Office PowerPoint 2007 Tutorial
Microsoft Office Excel 2007 Tutorial
Microsoft Office Word 2007 Tutorial
Java Source Code / Java Documentation » Net » lucene connector » org.apache.lucene.wikipedia.analysis 
Source Cross Reference  Class Diagram Java Document (Java Doc) 


java.lang.Object
   org.apache.lucene.wikipedia.analysis.WikipediaTokenizerImpl

WikipediaTokenizerImpl
class WikipediaTokenizerImpl (Code)
This class is a scanner generated by JFlex 1.4.1 on 1/4/08 3:30 PM from the specification file /Volumes/User/grantingersoll/projects/lucene/Lucene-Trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizerImpl.jflex


Field Summary
final public static  intACRONYM
    
final public static  intALPHANUM
    
final public static  intAPOSTROPHE
    
final public static  intBOLD
    
final public static  intBOLD_ITALICS
    
final public static  intCATEGORY
    
final public static  intCATEGORY_STATE
    
final public static  intCITATION
    
final public static  intCJ
    
final public static  intCOMPANY
    
final public static  intDOUBLE_BRACE_STATE
    
final public static  intDOUBLE_EQUALS_STATE
    
final public static  intEMAIL
    
final public static  intEXTERNAL_LINK
    
final public static  intEXTERNAL_LINK_STATE
    
final public static  intEXTERNAL_LINK_URL
    
final public static  intFIVE_SINGLE_QUOTES_STATE
    
final public static  intHEADING
    
final public static  intHOST
    
final public static  intINTERNAL_LINK
    
final public static  intINTERNAL_LINK_STATE
    
final public static  intITALICS
    
final public static  intNUM
    
final public static  intSTRING
    
final public static  intSUB_HEADING
    
final public static  intTHREE_SINGLE_QUOTES_STATE
    
final public static  String[]TOKEN_TYPES
    
final public static  intTWO_SINGLE_QUOTES_STATE
    
final public static  intYYEOF
    
final public static  intYYINITIAL
    

Constructor Summary
 WikipediaTokenizerImpl(java.io.Reader in)
     Creates a new scanner There is also a java.io.InputStream version of this constructor.
 WikipediaTokenizerImpl(java.io.InputStream in)
     Creates a new scanner.

Method Summary
public  intgetNextToken()
     Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
final public  intgetPositionIncrement()
    
final  voidgetText(Token t, int tokType)
     Fills Lucene token with the current token text.
final public  voidyybegin(int newState)
    
final public  intyychar()
    
final public  charyycharat(int pos)
     Returns the character at position pos from the matched text.
final public  voidyyclose()
     Closes the input stream.
final public  intyylength()
     Returns the length of the matched text region.
public  voidyypushback(int number)
     Pushes the specified amount of characters back into the input stream.
final public  voidyyreset(java.io.Reader reader)
     Resets the scanner to read from a new input stream.
final public  intyystate()
     Returns the current lexical state.
final public  Stringyytext()
     Returns the text matched by the current regular expression.

Field Detail
ACRONYM
final public static int ACRONYM(Code)



ALPHANUM
final public static int ALPHANUM(Code)



APOSTROPHE
final public static int APOSTROPHE(Code)



BOLD
final public static int BOLD(Code)



BOLD_ITALICS
final public static int BOLD_ITALICS(Code)



CATEGORY
final public static int CATEGORY(Code)



CATEGORY_STATE
final public static int CATEGORY_STATE(Code)



CITATION
final public static int CITATION(Code)



CJ
final public static int CJ(Code)



COMPANY
final public static int COMPANY(Code)



DOUBLE_BRACE_STATE
final public static int DOUBLE_BRACE_STATE(Code)
lexical states



DOUBLE_EQUALS_STATE
final public static int DOUBLE_EQUALS_STATE(Code)



EMAIL
final public static int EMAIL(Code)



EXTERNAL_LINK
final public static int EXTERNAL_LINK(Code)



EXTERNAL_LINK_STATE
final public static int EXTERNAL_LINK_STATE(Code)



EXTERNAL_LINK_URL
final public static int EXTERNAL_LINK_URL(Code)



FIVE_SINGLE_QUOTES_STATE
final public static int FIVE_SINGLE_QUOTES_STATE(Code)



HEADING
final public static int HEADING(Code)



HOST
final public static int HOST(Code)



INTERNAL_LINK
final public static int INTERNAL_LINK(Code)



INTERNAL_LINK_STATE
final public static int INTERNAL_LINK_STATE(Code)



ITALICS
final public static int ITALICS(Code)



NUM
final public static int NUM(Code)



STRING
final public static int STRING(Code)



SUB_HEADING
final public static int SUB_HEADING(Code)



THREE_SINGLE_QUOTES_STATE
final public static int THREE_SINGLE_QUOTES_STATE(Code)



TOKEN_TYPES
final public static String[] TOKEN_TYPES(Code)



TWO_SINGLE_QUOTES_STATE
final public static int TWO_SINGLE_QUOTES_STATE(Code)



YYEOF
final public static int YYEOF(Code)
This character denotes the end of file



YYINITIAL
final public static int YYINITIAL(Code)




Constructor Detail
WikipediaTokenizerImpl
WikipediaTokenizerImpl(java.io.Reader in)(Code)
Creates a new scanner There is also a java.io.InputStream version of this constructor.
Parameters:
  in - the java.io.Reader to read input from.



WikipediaTokenizerImpl
WikipediaTokenizerImpl(java.io.InputStream in)(Code)
Creates a new scanner. There is also java.io.Reader version of this constructor.
Parameters:
  in - the java.io.Inputstream to read input from.




Method Detail
getNextToken
public int getNextToken() throws java.io.IOException(Code)
Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs. the next token
exception:
  java.io.IOException - if any I/O-Error occurs



getPositionIncrement
final public int getPositionIncrement()(Code)



getText
final void getText(Token t, int tokType)(Code)
Fills Lucene token with the current token text.



yybegin
final public void yybegin(int newState)(Code)
Enters a new lexical state
Parameters:
  newState - the new lexical state



yychar
final public int yychar()(Code)



yycharat
final public char yycharat(int pos)(Code)
Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster
Parameters:
  pos - the position of the character to fetch. A value from 0 to yylength()-1. the character at position pos



yyclose
final public void yyclose() throws java.io.IOException(Code)
Closes the input stream.



yylength
final public int yylength()(Code)
Returns the length of the matched text region.



yypushback
public void yypushback(int number)(Code)
Pushes the specified amount of characters back into the input stream. They will be read again by then next call of the scanning method
Parameters:
  number - the number of characters to be read again.This number must not be greater than yylength()!



yyreset
final public void yyreset(java.io.Reader reader)(Code)
Resets the scanner to read from a new input stream. Does not close the old reader. All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.
Parameters:
  reader - the new input stream



yystate
final public int yystate()(Code)
Returns the current lexical state.



yytext
final public String yytext()(Code)
Returns the text matched by the current regular expression.



Methods inherited from java.lang.Object
native protected Object clone() throws CloneNotSupportedException(Code)(Java Doc)
public boolean equals(Object obj)(Code)(Java Doc)
protected void finalize() throws Throwable(Code)(Java Doc)
final native public Class getClass()(Code)(Java Doc)
native public int hashCode()(Code)(Java Doc)
final native public void notify()(Code)(Java Doc)
final native public void notifyAll()(Code)(Java Doc)
public String toString()(Code)(Java Doc)
final native public void wait(long timeout) throws InterruptedException(Code)(Java Doc)
final public void wait(long timeout, int nanos) throws InterruptedException(Code)(Java Doc)
final public void wait() throws InterruptedException(Code)(Java Doc)

www.java2java.com | Contact Us
Copyright 2009 - 12 Demo Source and Support. All rights reserved.
All other trademarks are property of their respective owners.