Java Doc for CrawlServer.java in  » Web-Crawler » heritrix » org » archive » crawler » datamodel » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Java Source Code / Java Documentation 1.  6.0 JDK Core 2.  6.0 JDK Modules 3.  6.0 JDK Modules com.sun 4.  6.0 JDK Modules com.sun.java 5.  6.0 JDK Modules sun 6.  6.0 JDK Platform 7.  Ajax 8.  Apache Harmony Java SE 9.  Aspect oriented 10.  Authentication Authorization 11.  Blogger System 12.  Build 13.  Byte Code 14.  Cache 15.  Chart 16.  Chat 17.  Code Analyzer 18.  Collaboration 19.  Content Management System 20.  Database Client 21.  Database DBMS 22.  Database JDBC Connection Pool 23.  Database ORM 24.  Development 25.  EJB Server geronimo 26.  EJB Server GlassFish 27.  EJB Server JBoss 4.2.1 28.  EJB Server resin 3.1.5 29.  ERP CRM Financial 30.  ESB 31.  Forum 32.  GIS 33.  Graphic Library 34.  Groupware 35.  HTML Parser 36.  IDE 37.  IDE Eclipse 38.  IDE Netbeans 39.  Installer 40.  Internationalization Localization 41.  Inversion of Control 42.  Issue Tracking 43.  J2EE 44.  JBoss 45.  JMS 46.  JMX 47.  Library 48.  Mail Clients 49.  Net 50.  Parser 51.  PDF 52.  Portal 53.  Profiler 54.  Project Management 55.  Report 56.  RSS RDF 57.  Rule Engine 58.  Science 59.  Scripting 60.  Search Engine 61.  Security 62.  Sevlet Container 63.  Source Control 64.  Swing Library 65.  Template Engine 66.  Test Coverage 67.  Testing 68.  UML 69.  Web Crawler 70.  Web Framework 71.  Web Mail 72.  Web Server 73.  Web Services 74.  Web Services apache cxf 2.0.1 75.  Web Services AXIS2 76.  Wiki Engine 77.  Workflow Engines 78.  XML 79.  XML UI Java Java Tutorial Java Open Source Jar File Download Java Articles Java Products Java by API Photoshop Tutorials Maya Tutorials Flash Tutorials 3ds-Max Tutorials Illustrator Tutorials GIMP Tutorials C# / C Sharp C# / CSharp Tutorial C# / CSharp Open Source ASP.Net ASP.NET Tutorial JavaScript DHTML JavaScript Tutorial JavaScript Reference HTML / CSS HTML CSS Reference C / ANSI-C C Tutorial C++ C++ Tutorial Ruby PHP Python Python Tutorial Python Open Source SQL Server / T-SQL SQL Server / T-SQL Tutorial Oracle PL / SQL Oracle PL/SQL Tutorial PostgreSQL SQL / MySQL MySQL Tutorial VB.Net VB.Net Tutorial Flash / Flex / ActionScript VBA / Excel / Access / Word XML XML Tutorial Microsoft Office PowerPoint 2007 Tutorial Microsoft Office Excel 2007 Tutorial Microsoft Office Word 2007 Tutorial

Java Source Code / Java Documentation » Web Crawler » heritrix » org.archive.crawler.datamodel   Source Cross Reference   Class Diagram   Java Document (Java Doc)   java.lang.Object    org.archive.crawler.datamodel.CrawlServer CrawlServer public class CrawlServer implements Serializable,CrawlSubstats.HasCrawlSubstats(Code) Represents a single remote "server". A server is a service on a host. There might be more than one service on a host differentiated by a port number. author:    gojomo Field Summary final public static  long MIN_ROBOTS_RETRIES      final public static  long ROBOTS_NOT_FETCHED      protected  int consecutiveConnectionErrors       long robotsFetched       Checksum robotstxtChecksum       CrawlSubstats substats       boolean validRobots      Constructor Summary public   CrawlServer(String h)      Creates a new CrawlServer object. Method Summary public  void addCredentialAvatar(CredentialAvatar ca)      Add an avatar. public  Set getCredentialAvatars()      Credential avatars for this server. public  String getName()      public  int getPort()      Get the port number for this server. public  RobotsExclusionPolicy getRobots()      Get the robots exclusion policy for this server. public  long getRobotsFetchedTime()      public static  String getServerKey(CandidateURI cauri)      Get key to use doing lookup on server instances. Parameters:   cauri - CandidateURI we're to get server key for. public  SettingsHandler getSettingsHandler()      Get the settings handler. public  CrawlSubstats getSubstats()      public  boolean hasCredentialAvatars()      public  void incrementConsecutiveConnectionErrors()      public  boolean isValidRobots()      If true then valid robots.txt information has been retrieved. public  void resetConsecutiveConnectionErrors()      public  void setRobots(RobotsExclusionPolicy policy)      Set the robots exclusion policy for this server. public  void setSettingsHandler(SettingsHandler settingsHandler)      Set the settings handler to be used by this server. public  String toString()      public  void updateRobots(CrawlURI curi)      Update the robots exclusion policy. Field Detail MIN_ROBOTS_RETRIES final public static long MIN_ROBOTS_RETRIES(Code) only check if robots-fetch is perhaps superfluous after this many tries ROBOTS_NOT_FETCHED final public static long ROBOTS_NOT_FETCHED(Code) consecutiveConnectionErrors protected int consecutiveConnectionErrors(Code) robotsFetched long robotsFetched(Code) robotstxtChecksum Checksum robotstxtChecksum(Code) substats CrawlSubstats substats(Code) validRobots boolean validRobots(Code) Constructor Detail CrawlServer public CrawlServer(String h)(Code) Creates a new CrawlServer object. Parameters:   h - the host string for the server. Method Detail addCredentialAvatar public void addCredentialAvatar(CredentialAvatar ca)(Code) Add an avatar. Parameters:   ca - Credential avatar to add to set of avatars. getCredentialAvatars public Set getCredentialAvatars()(Code) Credential avatars for this server. Returns null if none. getName public String getName()(Code) The server string which might include a port number. getPort public int getPort()(Code) Get the port number for this server. the port number or -1 if not known (uses default for protocol) getRobots public RobotsExclusionPolicy getRobots()(Code) Get the robots exclusion policy for this server. the robots exclusion policy for this server. getRobotsFetchedTime public long getRobotsFetchedTime()(Code) Returns the time when robots.txt was fetched. getServerKey public static String getServerKey(CandidateURI cauri) throws URIException(Code) Get key to use doing lookup on server instances. Parameters:   cauri - CandidateURI we're to get server key for. String to use as server key. throws:   URIException - getSettingsHandler public SettingsHandler getSettingsHandler()(Code) Get the settings handler. the settings handler. getSubstats public CrawlSubstats getSubstats()(Code) hasCredentialAvatars public boolean hasCredentialAvatars()(Code) True if there are avatars attached to this instance. incrementConsecutiveConnectionErrors public void incrementConsecutiveConnectionErrors()(Code) isValidRobots public boolean isValidRobots()(Code) If true then valid robots.txt information has been retrieved. If false either no attempt has been made to fetch robots.txt or the attempt failed. Returns the validRobots. resetConsecutiveConnectionErrors public void resetConsecutiveConnectionErrors()(Code) setRobots public void setRobots(RobotsExclusionPolicy policy)(Code) Set the robots exclusion policy for this server. Parameters:   policy - the policy to set. setSettingsHandler public void setSettingsHandler(SettingsHandler settingsHandler)(Code) Set the settings handler to be used by this server. Parameters:   settingsHandler - the settings handler to be used by this server. toString public String toString()(Code) updateRobots public void updateRobots(CrawlURI curi)(Code) Update the robots exclusion policy. Parameters:   curi - the crawl URI containing the fetched robots.txt throws:   IOException - Methods inherited from java.lang.Object native protected Object clone() throws CloneNotSupportedException(Code)(Java Doc) public boolean equals(Object obj)(Code)(Java Doc) protected void finalize() throws Throwable(Code)(Java Doc) final native public Class getClass()(Code)(Java Doc) native public int hashCode()(Code)(Java Doc) final native public void notify()(Code)(Java Doc) final native public void notifyAll()(Code)(Java Doc) public String toString()(Code)(Java Doc) final native public void wait(long timeout) throws InterruptedException(Code)(Java Doc) final public void wait(long timeout, int nanos) throws InterruptedException(Code)(Java Doc) final public void wait() throws InterruptedException(Code)(Java Doc)