Java Doc for plasmaSwitchboard.java in  » Search-Engine » yacy » de » anomic » plasma » Java Source Code / Java DocumentationJava Source Code and Java Documentation

public static final String BLACKLIST_CLASS = "Blacklist.class"

Name of the setting which Blacklist backend shall be used.

public static final String CRAWLSTACK = "82_crawlstack"

Name of the crawl stacker thread, performing several checks on new URLs to crawl, i.e.

public static final String DBFILE_USER = "DATA/SETTINGS/user.db"

Path to the user-DB, beginning from the YaCy-installation's top-folder.

public static final String HTCACHE_PATH = "proxyCache"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all downloaded webpages and their respective ressources and HTTP-headers are stored.

public static final String HTDOCS_PATH = "htDocsPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all user-ressources (i.e.

public static final String HTROOT_PATH = "htRootPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all original servlets, their stylesheets, scripts, etc.

public static final String INDEX_DIST_ALLOW_WHILE_CRAWLING = "allowDistributeIndexWhileCrawling"

Name of the setting whether Index Distribution shall be allowed while crawling is in progress, i.e.

public static final String LISTS_PATH = "listsPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all user-lists like blacklists, etc.

public static final String PROXY_CACHE_LAYOUT = "proxyCacheLayout"

Name of the setting which file-/folder-layout the proxy cache shall use.

public static final String PROXY_CACHE_LAYOUT_HASH = "hash"

Setting the file-/folder-structure for plasmaSwitchboard.PROXY_CACHE_LAYOUT .

public static final String PROXY_CACHE_LAYOUT_TREE = "tree"

Setting the file-/folder-structure for plasmaSwitchboard.PROXY_CACHE_LAYOUT .

public static final String STORAGE_PEER_HASH = "storagePeerHash"

Name of the setting holding the Peer-Hash where indexes shall be transferred after indexing a webpage.

public static final String WORDCACHE_MAX_COUNT = "wordCacheMaxCount"

Name of the setting how many words the word-cache (or DHT-Out cache) shall contain maximal.

public static final String WORK_PATH = "wordPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all DBs containing "work" of the user are saved.

public static final String BLACKLIST_CLASS = "Blacklist.class"

Name of the setting which Blacklist backend shall be used. Due to different requirements of users, the plasmaURLPattern -interface has been created to support blacklist engines different from YaCy's default

Attention is required when the backend is changed, because different engines may have different syntaxes

public static final String BLACKLIST_CLASS_DEFAULT = "de.anomic.plasma.urlPattern.defaultURLPattern"

Package and name of YaCy's defaultURLPattern default blacklist implementation

public static final String CLEANUP = "90_cleanup"

The cleanup thread which is responsible for pendant cleanup-jobs, news/ranking distribution, etc.

public static final String CRAWLER_THREADS_ACTIVE_MAX = "crawler.MaxActiveThreads"

Name of the setting how many active crawler-threads may maximal be running on the same time

public static final String CRAWLJOB_LOCAL_CRAWL = "50_localcrawl"

Name of the local crawler thread, popping one entry off the Local Crawl Queue, and passing it to the proxy cache enqueue thread to download and further process it

public static final String CRAWLJOB_REMOTE_CRAWL_LOADER = "60_remotecrawlloader"

Name of the remote crawl list loading thread

public static final String CRAWLJOB_REMOTE_TRIGGERED_CRAWL = "62_remotetriggeredcrawl"

Name of the remote triggered crawl thread, responsible for processing a remote crawl received from another peer

public static final String CRAWLSTACK = "82_crawlstack"

Name of the crawl stacker thread, performing several checks on new URLs to crawl, i.e. double-check

public static final String DBFILE_CRAWL_PROFILES = "crawlProfiles0.db"

Name of the file containing the database holding all recent crawl profiles

public static final String DBFILE_BLOG = "blog.db"

Name of the file containing the database holding all blog-entries available on this peer

public static final String DBFILE_BLOGCOMMENTS = "blogComment.db"

Name of the file containing the database holding all blogComment-entries available on this peer

public static final String DBFILE_BOOKMARKS = "bookmarks.db"

Name of the file containing the database holding all bookmarks available on this peer

public static final String DBFILE_BOOKMARKS_DATES = "bookmarkDates.db"

Name of the file containing the database holding all date->bookmark relations

public static final String DBFILE_BOOKMARKS_TAGS = "bookmarkTags.db"

Name of the file containing the database holding all tag->bookmark relations

public static final String DBFILE_CRAWL_ROBOTS = "crawlRobotsTxt.db"

Name of the file containing the database holding all robots.txt-entries of the lately crawled domains

public static final String DBFILE_MESSAGE = "message.db"

Name of the file containing the database holding the user's peer-messages

public static final String DBFILE_OWN_SEED = "mySeed.txt"

Name of the file containing the database holding this peer's seed

public static final String DBFILE_USER = "DATA/SETTINGS/user.db"

Path to the user-DB, beginning from the YaCy-installation's top-folder. It holds all rights the created users have as well as all other needed data about them

public static final String DBFILE_WIKI = "wiki.db"

Name of the file containing the database holding the whole wiki of this peer

public static final String DBFILE_WIKI_BKP = "wiki-bkp.db"

Name of the file containing the database holding all versions but the latest of the wiki-pages of this peer

public static final String DBPATH = "dbPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all databases containing queues are stored

public static final String HTCACHE_PATH = "proxyCache"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all downloaded webpages and their respective ressources and HTTP-headers are stored. It is the location containing the proxy-cache

public static final String HTDOCS_PATH = "htDocsPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all user-ressources (i.e. for the fileshare or the contents displayed on www.peername.yacy) lie. The translated templates of the webinterface will also be put in here

public static final String HTROOT_PATH = "htRootPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all original servlets, their stylesheets, scripts, etc. lie. It is also home of the XML-interface to YaCy

public static final String INDEXER = "80_indexing"

Name of the indexer thread, performing the actual indexing of a website

public static final String INDEX_DIST = "20_dhtdistribution"

Name of the DHT distribution thread, which selects index chunks and transfers them to other peers according to the global DHT rules

public static final String INDEX_DIST_ALLOW = "allowDistributeIndex"

Name of the setting whether Index Distribution shall be allowed (and the DHT-thread therefore started) or not

public static final String INDEX_DIST_ALLOW_WHILE_CRAWLING = "allowDistributeIndexWhileCrawling"

Name of the setting whether Index Distribution shall be allowed while crawling is in progress, i.e. the Local Crawler Queue is filled.

This setting only has effect if plasmaSwitchboard.INDEX_DIST_ALLOW is enabled

public static final String INDEX_DIST_CHUNK_SIZE_MAX = "indexDistribution.maxChunkSize"

Name of the setting specifying how many words the hugest chunk may contain

public static final String INDEX_DIST_CHUNK_SIZE_MIN = "indexDistribution.minChunkSize"

Name of the setting specifying how many words the smallest chunk may contain

public static final String INDEX_DIST_CHUNK_SIZE_START = "indexDistribution.startChunkSize"

Name of the setting specifying how many words the very first chunk will contain when the DHT-thread starts

public static final String INDEX_DIST_DHT_RECEIPT_LIMIT = "indexDistribution.dhtReceiptLimit"

Name of the setting how many words the DHT-In cache may contain maximal before new DHT receipts will be rejected

public static final String INDEX_DIST_GZIP_BODY = "indexDistribution.gzipBody"

Name of the setting whether DHT chunks shall be transferred gzip-encodedly

public static final String INDEX_DIST_TIMEOUT = "indexDistribution.timeout"

Name of the setting how long the timeout for an Index Distribution shall be in milliseconds

public static final String INDEX_PATH = "indexPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where the whole database of known RWIs and URLs as well as dumps of the DHT-In and DHT-Out caches are stored

public static final String LISTS_PATH = "listsPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all user-lists like blacklists, etc. are stored

public static final String PEER_PING = "30_peerping"

Name of the Peer Ping thread which publishes the own peer and retrieves information about other peers connected to the YaCy-network

public static final String PROXY_CACHE_ENQUEUE = "70_cachemanager"

Name of the proxy cache enqueue thread which fetches a given website and saves the site itself as well as it's HTTP-headers in the HTCACHE

public static final String PROXY_CACHE_LAYOUT = "proxyCacheLayout"

Name of the setting which file-/folder-layout the proxy cache shall use. Possible values are plasmaSwitchboard.PROXY_CACHE_LAYOUT_TREE and plasmaSwitchboard.PROXY_CACHE_LAYOUT_HASH

public static final String PROXY_CACHE_LAYOUT_HASH = "hash"

Setting the file-/folder-structure for plasmaSwitchboard.PROXY_CACHE_LAYOUT . Websites are stored using the MD5-sum of their respective URLs. This method prevents collisions on some websites caused by using the plasmaSwitchboard.PROXY_CACHE_LAYOUT_TREE layout.

Similarly to plasmaSwitchboard.PROXY_CACHE_LAYOUT_TREE , the top-folders name is given by the protocol used to fetch the site, followed by either www or – if the hostname does not start with "www" – other. Afterwards the next folder has the rest of the hostname as name, followed by a folder hash which contains a folder consisting of the first two letters of the hash. Another folder named after the 3rd and 4th letters of the hash follows which finally contains the file named after the full 18-characters long hash. Below is an example how files are stored:

 /html/
 /html/www/
 /html/www/example.com/
 /html/www/example.com/hash/
 /html/www/example.com/hash/0d/
 /html/www/example.com/hash/0d/f8/
 /html/www/example.com/hash/0d/f8/0df83a8444f48317d8

public static final String PROXY_CACHE_LAYOUT_TREE = "tree"

Setting the file-/folder-structure for plasmaSwitchboard.PROXY_CACHE_LAYOUT . Websites are stored in a folder-layout according to the layout, the URL purported. The first folder is either http or https depending on the protocol used to fetch the website, descending follows the hostname and the sub-folders on the website up to the actual file itself.

When using tree, be aware that the possibility of inconsistencies between folders and files with the same name may occur which prevent proper storage of the fetched site. Below is an example how files are stored:

 /html/
 /html/www.example.com/
 /html/www.example.com/index/
 /html/www.example.com/index/en/
 /html/www.example.com/index/en/index.html

public static final String PROXY_ONLINE_CAUTION_DELAY = "onlineCautionDelay"

Name of the setting how long indexing should pause after the last time the proxy was used in milliseconds

public static final String PROXY_PREFETCH_DEPTH = "proxyPrefetchDepth"

Name of the setting how deep URLs fetched by proxy usage shall be followed

public static final String RAM_CACHE_BLOG = "ramCacheBlog"

Name of the setting how much memory in bytes should be assigned to the Blog DB for caching purposes

public static final String RAM_CACHE_EURL = "ramCacheEURL"

Name of the setting how much memory in bytes should be assigned to the Erroneous URLs DB for caching purposes

public static final String RAM_CACHE_HTTP = "ramCacheHTTP"

Name of the setting how much memory in bytes should be assigned to the HTTP Headers DB for caching purposes

public static final String RAM_CACHE_LURL = "ramCacheLURL"

Name of the setting how much memory in bytes should be assigned to the Loaded URLs DB for caching purposes

public static final String RAM_CACHE_MESSAGE = "ramCacheMessage"

Name of the setting how much memory in bytes should be assigned to the Message DB for caching purposes

public static final String RAM_CACHE_NURL = "ramCacheNURL"

Name of the setting how much memory in bytes should be assigned to the Noticed URLs DB for caching purposes

public static final String RAM_CACHE_PRE_NURL = "ramCachePreNURL"

Name of the setting how much memory in bytes should be assigned to the Pre-Noticed URLs DB for caching purposes

public static final String RAM_CACHE_PROFILES = "ramCacheProfiles"

Name of the setting how much memory in bytes should be assigned to the Crawl Profiles DB for caching purposes

public static final String RAM_CACHE_ROBOTS = "ramCacheRobots"

Name of the setting how much memory in bytes should be assigned to the robots.txts DB for caching purposes

public static final String RAM_CACHE_RWI = "ramCacheRWI"

Name of the setting how much memory in bytes should be assigned to the RWIs DB for caching purposes

public static final String RAM_CACHE_WIKI = "ramCacheWiki"

Name of the setting how much memory in bytes should be assigned to the Wiki DB for caching purposes

public static final String RANKING_PATH = "rankingPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all ranking files are stored, self-generated as well as received ranking files

public static final String SEED_UPLOAD = "40_peerseedcycle"

Name of the seed upload thread, providing the so-called seed-lists needed during bootstrapping

public static final String STORAGE_PEER_HASH = "storagePeerHash"

Name of the setting holding the Peer-Hash where indexes shall be transferred after indexing a webpage. If this setting is empty, the Storage Peer function is disabled

public static final String WORDCACHE_MAX_COUNT = "wordCacheMaxCount"

Name of the setting how many words the word-cache (or DHT-Out cache) shall contain maximal. Indexing pages if the cache has reached this limit will slow down the indexing process by flushing some of it's entries

public static final String WORK_PATH = "wordPath"

Name of the setting specifying the folder beginning from the YaCy-installation's top-folder, where all DBs containing "work" of the user are saved. Such include bookmarks, messages, wiki, blog