Java Doc for QuotaEnforcer.java in  » Web-Crawler » heritrix » org » archive » crawler » prefetch » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Java Source Code / Java Documentation 1.  6.0 JDK Core 2.  6.0 JDK Modules 3.  6.0 JDK Modules com.sun 4.  6.0 JDK Modules com.sun.java 5.  6.0 JDK Modules sun 6.  6.0 JDK Platform 7.  Ajax 8.  Apache Harmony Java SE 9.  Aspect oriented 10.  Authentication Authorization 11.  Blogger System 12.  Build 13.  Byte Code 14.  Cache 15.  Chart 16.  Chat 17.  Code Analyzer 18.  Collaboration 19.  Content Management System 20.  Database Client 21.  Database DBMS 22.  Database JDBC Connection Pool 23.  Database ORM 24.  Development 25.  EJB Server geronimo 26.  EJB Server GlassFish 27.  EJB Server JBoss 4.2.1 28.  EJB Server resin 3.1.5 29.  ERP CRM Financial 30.  ESB 31.  Forum 32.  GIS 33.  Graphic Library 34.  Groupware 35.  HTML Parser 36.  IDE 37.  IDE Eclipse 38.  IDE Netbeans 39.  Installer 40.  Internationalization Localization 41.  Inversion of Control 42.  Issue Tracking 43.  J2EE 44.  JBoss 45.  JMS 46.  JMX 47.  Library 48.  Mail Clients 49.  Net 50.  Parser 51.  PDF 52.  Portal 53.  Profiler 54.  Project Management 55.  Report 56.  RSS RDF 57.  Rule Engine 58.  Science 59.  Scripting 60.  Search Engine 61.  Security 62.  Sevlet Container 63.  Source Control 64.  Swing Library 65.  Template Engine 66.  Test Coverage 67.  Testing 68.  UML 69.  Web Crawler 70.  Web Framework 71.  Web Mail 72.  Web Server 73.  Web Services 74.  Web Services apache cxf 2.0.1 75.  Web Services AXIS2 76.  Wiki Engine 77.  Workflow Engines 78.  XML 79.  XML UI Java Java Tutorial Java Open Source Jar File Download Java Articles Java Products Java by API Photoshop Tutorials Maya Tutorials Flash Tutorials 3ds-Max Tutorials Illustrator Tutorials GIMP Tutorials C# / C Sharp C# / CSharp Tutorial C# / CSharp Open Source ASP.Net ASP.NET Tutorial JavaScript DHTML JavaScript Tutorial JavaScript Reference HTML / CSS HTML CSS Reference C / ANSI-C C Tutorial C++ C++ Tutorial Ruby PHP Python Python Tutorial Python Open Source SQL Server / T-SQL SQL Server / T-SQL Tutorial Oracle PL / SQL Oracle PL/SQL Tutorial PostgreSQL SQL / MySQL MySQL Tutorial VB.Net VB.Net Tutorial Flash / Flex / ActionScript VBA / Excel / Access / Word XML XML Tutorial Microsoft Office PowerPoint 2007 Tutorial Microsoft Office Excel 2007 Tutorial Microsoft Office Word 2007 Tutorial

Java Source Code / Java Documentation » Web Crawler » heritrix » org.archive.crawler.prefetch   Source Cross Reference   Class Diagram   Java Document (Java Doc)   org.archive.crawler.framework.Processor    org.archive.crawler.prefetch.QuotaEnforcer QuotaEnforcer public class QuotaEnforcer extends Processor implements FetchStatusCodes(Code) A simple quota enforcer. If the host, server, or frontier group associated with the current CrawlURI is already over its quotas, blocks the current URI's processing with S_BLOCKED_BY_QUOTA. author:    gojomo version:    $Date: 2007-04-06 00:40:50 +0000 (Fri, 06 Apr 2007) $, $Revision: 5040 $ Field Summary final protected static  String ATTR_FORCE_RETIRE      final protected static  String ATTR_GROUP_MAX_ALL_KB      final protected static  String ATTR_GROUP_MAX_FETCH_RESPONSES      final protected static  String ATTR_GROUP_MAX_FETCH_SUCCESSES      final protected static  String ATTR_GROUP_MAX_SUCCESS_KB      final protected static  String ATTR_HOST_MAX_ALL_KB      final protected static  String ATTR_HOST_MAX_FETCH_RESPONSES      final protected static  String ATTR_HOST_MAX_FETCH_SUCCESSES      final protected static  String ATTR_HOST_MAX_SUCCESS_KB      final protected static  String ATTR_SERVER_MAX_ALL_KB      final protected static  String ATTR_SERVER_MAX_FETCH_RESPONSES      final protected static  String ATTR_SERVER_MAX_FETCH_SUCCESSES      final protected static  String ATTR_SERVER_MAX_SUCCESS_KB      final protected static  Boolean DEFAULT_FORCE_RETIRE      final protected static  Long DEFAULT_GROUP_MAX_ALL_KB      final protected static  Long DEFAULT_GROUP_MAX_FETCH_RESPONSES      final protected static  Long DEFAULT_GROUP_MAX_FETCH_SUCCESSES      final protected static  Long DEFAULT_GROUP_MAX_SUCCESS_KB      final protected static  Long DEFAULT_HOST_MAX_ALL_KB      final protected static  Long DEFAULT_HOST_MAX_FETCH_RESPONSES      final protected static  Long DEFAULT_HOST_MAX_FETCH_SUCCESSES      final protected static  Long DEFAULT_HOST_MAX_SUCCESS_KB      final protected static  Long DEFAULT_SERVER_MAX_ALL_KB      final protected static  Long DEFAULT_SERVER_MAX_FETCH_RESPONSES      final protected static  Long DEFAULT_SERVER_MAX_FETCH_SUCCESSES      final protected static  Long DEFAULT_SERVER_MAX_SUCCESS_KB      final protected static  int GROUP      final protected static  int HOST      final protected static  int NAME      final protected static  int RESPONSES      final protected static  int RESPONSE_KB      final protected static  int SERVER      final protected static  int SUCCESSES      final protected static  int SUCCESS_KB      final protected static  String[][] keys      Constructor Summary public   QuotaEnforcer(String name)      Constructor. Method Summary protected  boolean applyQuota(CrawlURI curi, String quotaKey, long actual)      Apply the quota specified by the given key against the actual value provided. protected  boolean checkQuotas(CrawlURI curi, CrawlSubstats.HasCrawlSubstats hasStats, int CAT)      Check all quotas for the given substats and category (server, host, or group). protected  void innerProcess(CrawlURI curi)      Field Detail ATTR_FORCE_RETIRE final protected static String ATTR_FORCE_RETIRE(Code) whether to force-retire when over-quote detected ATTR_GROUP_MAX_ALL_KB final protected static String ATTR_GROUP_MAX_ALL_KB(Code) group max all fetch bytes (including error responses) ATTR_GROUP_MAX_FETCH_RESPONSES final protected static String ATTR_GROUP_MAX_FETCH_RESPONSES(Code) group max fetch responses (including error codes) ATTR_GROUP_MAX_FETCH_SUCCESSES final protected static String ATTR_GROUP_MAX_FETCH_SUCCESSES(Code) group max successful fetches ATTR_GROUP_MAX_SUCCESS_KB final protected static String ATTR_GROUP_MAX_SUCCESS_KB(Code) group max successful fetch bytes ATTR_HOST_MAX_ALL_KB final protected static String ATTR_HOST_MAX_ALL_KB(Code) host max all fetch bytes (including error responses) ATTR_HOST_MAX_FETCH_RESPONSES final protected static String ATTR_HOST_MAX_FETCH_RESPONSES(Code) host max fetch responses (including error codes) ATTR_HOST_MAX_FETCH_SUCCESSES final protected static String ATTR_HOST_MAX_FETCH_SUCCESSES(Code) host max successful fetches ATTR_HOST_MAX_SUCCESS_KB final protected static String ATTR_HOST_MAX_SUCCESS_KB(Code) host max successful fetch bytes ATTR_SERVER_MAX_ALL_KB final protected static String ATTR_SERVER_MAX_ALL_KB(Code) server max all fetch bytes (including error responses) ATTR_SERVER_MAX_FETCH_RESPONSES final protected static String ATTR_SERVER_MAX_FETCH_RESPONSES(Code) server max fetch responses (including error codes) ATTR_SERVER_MAX_FETCH_SUCCESSES final protected static String ATTR_SERVER_MAX_FETCH_SUCCESSES(Code) server max successful fetches ATTR_SERVER_MAX_SUCCESS_KB final protected static String ATTR_SERVER_MAX_SUCCESS_KB(Code) server max successful fetch bytes DEFAULT_FORCE_RETIRE final protected static Boolean DEFAULT_FORCE_RETIRE(Code) DEFAULT_GROUP_MAX_ALL_KB final protected static Long DEFAULT_GROUP_MAX_ALL_KB(Code) DEFAULT_GROUP_MAX_FETCH_RESPONSES final protected static Long DEFAULT_GROUP_MAX_FETCH_RESPONSES(Code) DEFAULT_GROUP_MAX_FETCH_SUCCESSES final protected static Long DEFAULT_GROUP_MAX_FETCH_SUCCESSES(Code) DEFAULT_GROUP_MAX_SUCCESS_KB final protected static Long DEFAULT_GROUP_MAX_SUCCESS_KB(Code) DEFAULT_HOST_MAX_ALL_KB final protected static Long DEFAULT_HOST_MAX_ALL_KB(Code) DEFAULT_HOST_MAX_FETCH_RESPONSES final protected static Long DEFAULT_HOST_MAX_FETCH_RESPONSES(Code) DEFAULT_HOST_MAX_FETCH_SUCCESSES final protected static Long DEFAULT_HOST_MAX_FETCH_SUCCESSES(Code) DEFAULT_HOST_MAX_SUCCESS_KB final protected static Long DEFAULT_HOST_MAX_SUCCESS_KB(Code) DEFAULT_SERVER_MAX_ALL_KB final protected static Long DEFAULT_SERVER_MAX_ALL_KB(Code) DEFAULT_SERVER_MAX_FETCH_RESPONSES final protected static Long DEFAULT_SERVER_MAX_FETCH_RESPONSES(Code) DEFAULT_SERVER_MAX_FETCH_SUCCESSES final protected static Long DEFAULT_SERVER_MAX_FETCH_SUCCESSES(Code) DEFAULT_SERVER_MAX_SUCCESS_KB final protected static Long DEFAULT_SERVER_MAX_SUCCESS_KB(Code) GROUP final protected static int GROUP(Code) HOST final protected static int HOST(Code) NAME final protected static int NAME(Code) RESPONSES final protected static int RESPONSES(Code) RESPONSE_KB final protected static int RESPONSE_KB(Code) SERVER final protected static int SERVER(Code) SUCCESSES final protected static int SUCCESSES(Code) SUCCESS_KB final protected static int SUCCESS_KB(Code) keys final protected static String[][] keys(Code) Constructor Detail QuotaEnforcer public QuotaEnforcer(String name)(Code) Constructor. Parameters:   name - Name of this processor. Method Detail applyQuota protected boolean applyQuota(CrawlURI curi, String quotaKey, long actual)(Code) Apply the quota specified by the given key against the actual value provided. If the quota and actual values rule out processing the given CrawlURI, mark up the CrawlURI appropriately. Parameters:   curi - CrawlURI whose processing is subject to a potential quotalimitation Parameters:   quotaKey - settings key to get applicable quota Parameters:   actual - current value to compare to quota true is CrawlURI is blocked by a quota, false otherwise checkQuotas protected boolean checkQuotas(CrawlURI curi, CrawlSubstats.HasCrawlSubstats hasStats, int CAT)(Code) Check all quotas for the given substats and category (server, host, or group). Parameters:   curi - CrawlURI to mark up with results Parameters:   hasStats - holds CrawlSubstats with actual values to test Parameters:   CAT - category index (SERVER, HOST, GROUP) to quota settings keys true if quota precludes fetching of CrawlURI innerProcess protected void innerProcess(CrawlURI curi)(Code) Fields inherited from org.archive.crawler.framework.Processor final public static String ATTR_DECIDE_RULES(Code)(Java Doc) final public static String ATTR_ENABLED(Code)(Java Doc) protected String attrDecideRules(Code)(Java Doc) Methods inherited from org.archive.crawler.framework.Processor protected void checkForInterrupt() throws InterruptedException(Code)(Java Doc) protected void finalTasks()(Code)(Java Doc) public CrawlController getController()(Code)(Java Doc) protected DecideRule getDecideRule(Object o)(Code)(Java Doc) public Processor getDefaultNextProcessor(CrawlURI curi)(Code)(Java Doc) protected void initialTasks()(Code)(Java Doc) protected void innerProcess(CrawlURI curi) throws InterruptedException(Code)(Java Doc) protected void innerRejectProcess(CrawlURI curi) throws InterruptedException(Code)(Java Doc) protected boolean isContentToProcess(CrawlURI curi)(Code)(Java Doc) protected boolean isExpectedMimeType(String contentType, String expectedPrefix)(Code)(Java Doc) protected boolean isHttpTransactionContentToProcess(CrawlURI curi)(Code)(Java Doc) public void kickUpdate()(Code)(Java Doc) final public void process(CrawlURI curi) throws InterruptedException(Code)(Java Doc) public String report()(Code)(Java Doc) protected boolean rulesAccept(Object o)(Code)(Java Doc) protected boolean rulesAccept(DecideRule rule, Object o)(Code)(Java Doc) public void setDefaultNextProcessor(Processor nextProcessor)(Code)(Java Doc) public Processor spawn(int serialNum)(Code)(Java Doc)