| A simple crawl splitter/mapper, dividing up CandidateURIs/CrawlURIs
between crawlers by diverting some range of URIs to local log files
(which can then be imported to other crawlers).
May operate on a CrawlURI (typically early in the processing chain) or
its CandidateURI outlinks (late in the processing chain, after
LinksScoper), or both (if inserted and configured in both places).
Applies a map() method, supplied by a concrete subclass, to
classKeys to map URIs to crawlers by name.
One crawler name is distinguished as the 'local name'; URIs mapped to
this name are not diverted, but continue to be processed normally.
If using the JMX importUris operation importing URLs dropped by
a
CrawlMapper instance, use recoveryLog style.
author:
gojomo
version:
$Date: 2006-09-26 23:47:15 +0000 (Tue, 26 Sep 2006) $, $Revision: 4671 $ |
