| java.lang.Object
 it.unimi.dsi.mg4j.index.Index
it.unimi.dsi.mg4j.index.BitStreamIndex
All known Subclasses: it.unimi.dsi.mg4j.index.MemoryMappedIndex, it.unimi.dsi.mg4j.index.remote.RemoteBitStreamIndex, it.unimi.dsi.mg4j.index.FileIndex, it.unimi.dsi.mg4j.index.InMemoryIndex, it.unimi.dsi.mg4j.index.BitStreamHPIndex,
| Inner Class :public static enum PropertyKeys | |
| Constructor Summary | |
| public | BitStreamIndex(int numberOfDocuments, int numberOfTerms, long numberOfPostings, long numberOfOccurrences, int maxCount, Payload payload, Coding frequencyCoding, Coding pointerCoding, Coding countCoding, Coding positionCoding, int quantum, int height, int bufferSize, TermProcessor termProcessor, String field, Properties properties, StringMap<? extends CharSequence> termMap, PrefixMap<? extends CharSequence> prefixMap, IntList sizes, LongList offsets)
|
| Method Summary | |
| public IndexIterator | documents(CharSequence prefix, int limit)
Returns a
MultiTermIndexIterator over all terms starting with the given prefix,
provided their number does not exceed the given limit and that this index has a
BitStreamIndex.prefixMap . | | protected static String | featureName(Coding coding)
| | public static int | gaussianGolombModulus(long quantumSigma, int shift)
Computes the Gaussian Golomb modulus for a given standard deviation
and shift using fixed-point arithmetic.
The Golomb modulus for (positive and negative)
integers normally distributed with standard deviation σ can be computed using
the formula ⌈ 2 sqrt( 2 / π ) ln(2) σ ⌉.
The resulting Golomb modulus is near to optimal for coding such
integers after they have been passed through
Fast.int2nat(int) . | | protected Constructor<? extends IndexReader> | getConstructor()
| | abstract public InputBitStream | getInputBitStream(int bufferSize)
Returns an input bit stream over the index.
Parameters:
bufferSize - a suggested buffer size. | | abstract public InputStream | getInputStream()
Returns an input stream over the index. | | public IndexReader | getReader(int bufferSize)
| | public static int | golombModulus(int p, int q)
Computes the Golomb modulus for a given fraction using
fixed-point arithmetic and a precomputed table for
small values. | | public static long | quantumSigma(int frequency, int numberOfDocuments, int quantum)
Computes the standard deviation associated to a given quantum and document frequency.
Parameters:
frequency - the document frequency.
Parameters:
numberOfDocuments - the overall number of documents.
Parameters:
quantum - the quantum. |
| DEFAULT_BUFFER_SIZE | | final public static int DEFAULT_BUFFER_SIZE(Code) | | The default buffer size.
|
| DEFAULT_HEIGHT | | final public static int DEFAULT_HEIGHT(Code) | | The default height (fairly low, due to memory consumption).
|
| DEFAULT_QUANTUM | | final public static int DEFAULT_QUANTUM(Code) | | The default quantum.
|
| FIXED_POINT_BITS | | final public static int FIXED_POINT_BITS(Code) | | Fixed number of fractional binary digits used in fixed-point computation of Golomb moduli.
|
| bufferSize | | final public int bufferSize(Code) | | The size of the buffer used to read the bit stream.
|
| height | | final public int height(Code) | | The parameter h (the maximum height of a skip tower), or -1 if this index has no skips.
|
| offsets | | final public LongList offsets(Code) | | The offset of each term, if offsets were loaded or specified at creation time, or null.
|
| prefixMap | | final public PrefixMap<? extends CharSequence> prefixMap(Code) | | The prefix map for this index, or null if the prefix map was not loaded.
|
| quantum | | final public int quantum(Code) | | The quantum, or -1 if this index has no skips.
|
| readerConstructor | | final public Constructor<? extends IndexReader> readerConstructor(Code) | | The constructor that will be used to create new index readers.
|
| termMap | | final public StringMap<? extends CharSequence> termMap(Code) | | The term map for this index, or null if the term map was not loaded.
|
| BitStreamIndex | | public BitStreamIndex(int numberOfDocuments, int numberOfTerms, long numberOfPostings, long numberOfOccurrences, int maxCount, Payload payload, Coding frequencyCoding, Coding pointerCoding, Coding countCoding, Coding positionCoding, int quantum, int height, int bufferSize, TermProcessor termProcessor, String field, Properties properties, StringMap<? extends CharSequence> termMap, PrefixMap<? extends CharSequence> prefixMap, IntList sizes, LongList offsets)(Code) | | |
| featureName | | protected static String featureName(Coding coding)(Code) | | |
| gaussianGolombModulus | | public static int gaussianGolombModulus(long quantumSigma, int shift)(Code) | | Computes the Gaussian Golomb modulus for a given standard deviation
and shift using fixed-point arithmetic.
The Golomb modulus for (positive and negative)
integers normally distributed with standard deviation σ can be computed using
the formula ⌈ 2 sqrt( 2 / π ) ln(2) σ ⌉.
The resulting Golomb modulus is near to optimal for coding such
integers after they have been passed through
Fast.int2nat(int) . Note,
however, that Golomb coding is not optimal for a normal distribution.
This function is used to compute the correct Golomb modulus for skip towers.
Parameters:
quantumSigma - the standard deviation of a quantum as returned by BitStreamIndex.quantumSigma(int,int,int).
Parameters:
shift - a shift parameter. the Golomb modulus for the standard deviation obtained multiplying quantumSigma bythe square root of 2shift-1. |
| getInputBitStream | | abstract public InputBitStream getInputBitStream(int bufferSize) throws IOException(Code) | | Returns an input bit stream over the index.
Parameters:
bufferSize - a suggested buffer size. an input bit stream over the index. |
| getInputStream | | abstract public InputStream getInputStream() throws IOException(Code) | | Returns an input stream over the index.
an input stream over the index. |
| golombModulus | | public static int golombModulus(int p, int q)(Code) | | Computes the Golomb modulus for a given fraction using
fixed-point arithmetic and a precomputed table for
small values. This gives results that are
extremely close to ⌈ log( 2 - p/q ) / log( 1 - p/q ) ⌉,
but the computation is orders of magnitude quicker.
Parameters:
p - the numerator.
Parameters:
q - the denominator (larger than or equal to p). the Golomb modulus for p/q. |
| quantumSigma | | public static long quantumSigma(int frequency, int numberOfDocuments, int quantum)(Code) | | Computes the standard deviation associated to a given quantum and document frequency.
Parameters:
frequency - the document frequency.
Parameters:
numberOfDocuments - the overall number of documents.
Parameters:
quantum - the quantum. a long representing in fixed-point arithmetic the value Math.sqrt( quantum * ( 1 - p ) ) / p, wherep is the relative frequency. |
| Methods inherited from it.unimi.dsi.mg4j.index.Index | public IndexIterator documents(int term) throws IOException(Code)(Java Doc)
public IndexIterator documents(CharSequence term) throws IOException(Code)(Java Doc)
abstract public IndexIterator documents(CharSequence prefix, int limit) throws IOException, TooManyTermsException(Code)(Java Doc)
public static Index getInstance(CharSequence uri, boolean randomAccess, boolean documentSizes, boolean maps) throws IOException, ConfigurationException, URISyntaxException, ClassNotFoundException, SecurityException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException(Code)(Java Doc)
public static Index getInstance(CharSequence uri, boolean randomAccess, boolean documentSizes) throws IOException, ConfigurationException, URISyntaxException, ClassNotFoundException, SecurityException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException(Code)(Java Doc)
public static Index getInstance(CharSequence uri, boolean randomAccess) throws ConfigurationException, IOException, URISyntaxException, ClassNotFoundException, SecurityException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException(Code)(Java Doc)
public static Index getInstance(CharSequence uri) throws ConfigurationException, IOException, URISyntaxException, ClassNotFoundException, SecurityException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException(Code)(Java Doc)
public IndexReader getReader() throws IOException(Code)(Java Doc)
abstract public IndexReader getReader(int bufferSize) throws IOException(Code)(Java Doc)
protected static TermProcessor getTermProcessor(Properties properties)(Code)(Java Doc)
public void keyIndex(Index newKeyIndex)(Code)(Java Doc)
|
|
|