| it.unimi.dsi.mg4j.index.cluster |
|
MG4J: Managing Gigabytes for Java
Index partitioning and clustering.
This package contains the classes that provide the infrastructure for index partitioning
and clustering. The tools the actually perform partitioning can be found in {@link it.unimi.dsi.mg4j.tool}.
An index cluster is a set of local indices that are viewed as a single index. In
a {@linkplain it.unimi.dsi.mg4j.index.cluster.LexicalCluster lexical cluster} each local index has a disjoint set of terms, but the document pointers contained
in each local index refer to the same documents. In a {@linkplain it.unimi.dsi.mg4j.index.cluster.DocumentalCluster documental cluster}
each index contains postings referring to a disjoint subset of a collection.
Clustering indices requires mapping term number and document pointers back and forth between the global
index and local indices. This mapping is provided by {@linkplain it.unimi.dsi.mg4j.index.cluster.DocumentalClusteringStrategy documental clustering strategies}
and {@linkplain it.unimi.dsi.mg4j.index.cluster.LexicalClusteringStrategy lexical clustering strategies}.
Clusters are often generated by partitioning an index (albeit, for instance,
{@link it.unimi.dsi.mg4j.tool.Scan} produces a cluster as output of the indexing process). In this case, a
{@linkplain it.unimi.dsi.mg4j.index.cluster.DocumentalPartitioningStrategy documental partitioning strategy}
or a {@linkplain it.unimi.dsi.mg4j.index.cluster.LexicalPartitioningStrategy lexical partitioning strategy}
explain how to divide and remap term numbers and document pointers. Of course, the clustering and partitioning
strategy must be suitably matched.
|
| Java Source File Name | Type | Comment |
| AbstractIndexClusterIndexReader.java | Class | An abstract implementation of an
IndexReader for an
IndexCluster . |
| ChainedLexicalClusteringStrategy.java | Class | A lexical clustering strategy that uses a chain of responsability to choose the local index:
out of a given list are inquired
until one contains the given term.
If the index cluster has Bloom filters, they will be used to reduce useless accesses to
term maps.
The intended usage of this class is memory/disk lexical partitioning. |
| ClusteringStrategy.java | Interface | A common ancestor interface for all clustering strategies.
Clustering strategies are dual to
.
After partitioning an index, you can see the set of its parts as local indices
of a cluster. |
| ContiguousDocumentalStrategy.java | Class | A documental partitioning and clustering strategy that partitions an index into contiguous segments.
To partition an index in contiguous parts, you must provide an array
of cutpoints, which define each part. |
| ContiguousLexicalStrategy.java | Class | A lexical strategy that partitions terms into contiguous segments.
To partition terms in contiguous segments, you must provide an array
of cutpoints, which define each segment of terms. |
| DocumentalCluster.java | Class | A abstract class representing a cluster of local indices containing separate
set of documents from the same collection.
This class stores the strategy and possibly the
associated to this documental cluster. |
| DocumentalClusterIndexReader.java | Class | An index reader for a
it.unimi.dsi.mg4j.index.cluster.DocumentalCluster . |
| DocumentalClusteringStrategy.java | Interface | A way to associate (quite bidirectionally) local and global document pointers. |
| DocumentalConcatenatedCluster.java | Class | A
it.unimi.dsi.mg4j.index.cluster.DocumentalCluster that concatenates the
postings of its local indices. |
| DocumentalConcatenatedClusterDocumentIterator.java | Class | A document iterator concatenating iterators from local indices. |
| DocumentalConcatenatedClusterIndexIterator.java | Class | An index iterator concatenating iterators from local indices. |
| DocumentalMergedCluster.java | Class | A
it.unimi.dsi.mg4j.index.cluster.DocumentalCluster that merges the
postings of its local indices. |
| DocumentalMergedClusterDocumentIterator.java | Class | A document iterator merging iterators from local indices. |
| DocumentalMergedClusterIndexIterator.java | Class | An index iterator merging iterators from local indices. |
| DocumentalPartitioningStrategy.java | Interface | A way to associate a document with a local index out of a given set and a local document number in the local index. |
| DocumentalStrategies.java | Class | Static utility methods for documental strategies. |
| IdentityDocumentalStrategy.java | Class | A documental strategy that maps identically local to global pointers and viceversa.
When clustering a posteriory a set of previously built renumbered indices
using a
it.unimi.dsi.mg4j.index.cluster.DocumentalMergedCluster ,
local document pointers are actually global document pointers. |
| IndexCluster.java | Class | An abstract index cluster. |
| LexicalCluster.java | Class | A cluster exhibiting local indices referring to the same collection, but
containing different set of terms, as a single index. |
| LexicalClusterIndexReader.java | Class | An index reader for a
. |
| LexicalClusteringStrategy.java | Interface | A way to associate a term with a local index out of a given set. |
| LexicalPartitioningStrategy.java | Interface | A way to associate a term number with a local index out of a given set and a local term number in the local index. |
| LexicalStrategies.java | Class | Static utility methods for lexical strategies. |
| PartitioningStrategy.java | Interface | A common ancestor interface for all partitioning strategies.
When partitioning an index, there are a few pieces of data that
must be provided independenly of the particular strategy chosen. |