| java.lang.Object
 org.archive.io.WriterPoolMember
All known Subclasses: org.archive.io.arc.ARCWriter, org.archive.io.warc.v10.ExperimentalWARCWriter, org.archive.io.warc.ExperimentalWARCWriter,
| WriterPoolMember | | abstract public class WriterPoolMember implements ArchiveFileConstants(Code) | | Member of
WriterPool .
Implements rotating off files, file naming with some guarantee of
uniqueness, and position in file. Subclass to pick up functionality for a
particular Writer type.
author:
stack
version:
$Date: 2007-04-02 22:02:14 +0000 (Mon, 02 Apr 2007) $ $Revision: 5032 $ |
| Constructor Summary | |
| protected | WriterPoolMember(AtomicInteger serialNo, OutputStream out, File file, boolean cmprs, String a14DigitDate)
Constructor.
Takes a stream. | | public | WriterPoolMember(AtomicInteger serialNo, List<File> dirs, String prefix, boolean cmprs, long maxSize, String extension)
Constructor.
Parameters:
serialNo - used to create unique filename sequences
Parameters:
dirs - Where to drop files.
Parameters:
prefix - File prefix to use.
Parameters:
cmprs - Compress the records written. | | public | WriterPoolMember(AtomicInteger serialNo, List<File> dirs, String prefix, String suffix, boolean cmprs, long maxSize, String extension)
Constructor.
Parameters:
serialNo - used to create unique filename sequences
Parameters:
dirs - Where to drop files.
Parameters:
prefix - File prefix to use.
Parameters:
cmprs - Compress the records written. |
| DEFAULT_PREFIX | | final public static String DEFAULT_PREFIX(Code) | | Default file prefix.
Stands for Internet Archive Heritrix.
|
| DEFAULT_SUFFIX | | final public static String DEFAULT_SUFFIX(Code) | | Default for file suffix.
|
| HOSTNAME_VARIABLE | | final public static String HOSTNAME_VARIABLE(Code) | | Value to interpolate with actual hostname.
|
| WriterPoolMember | | protected WriterPoolMember(AtomicInteger serialNo, OutputStream out, File file, boolean cmprs, String a14DigitDate) throws IOException(Code) | | Constructor.
Takes a stream. Use with caution. There is no upperbound check on size.
Will just keep writing.
Parameters:
serialNo - used to create unique filename sequences
Parameters:
out - Where to write.
Parameters:
file - File the out is connected to.
Parameters:
cmprs - Compress the content written.
Parameters:
a14DigitDate - If null, we'll write current time.
throws:
IOException - |
| WriterPoolMember | | public WriterPoolMember(AtomicInteger serialNo, List<File> dirs, String prefix, boolean cmprs, long maxSize, String extension)(Code) | | Constructor.
Parameters:
serialNo - used to create unique filename sequences
Parameters:
dirs - Where to drop files.
Parameters:
prefix - File prefix to use.
Parameters:
cmprs - Compress the records written.
Parameters:
maxSize - Maximum size for ARC files written.
Parameters:
extension - Extension to give file. |
| WriterPoolMember | | public WriterPoolMember(AtomicInteger serialNo, List<File> dirs, String prefix, String suffix, boolean cmprs, long maxSize, String extension)(Code) | | Constructor.
Parameters:
serialNo - used to create unique filename sequences
Parameters:
dirs - Where to drop files.
Parameters:
prefix - File prefix to use.
Parameters:
cmprs - Compress the records written.
Parameters:
maxSize - Maximum size for ARC files written.
Parameters:
suffix - File tail to use. If null, unused.
Parameters:
extension - Extension to give file. |
| checkSize | | public void checkSize() throws IOException(Code) | | Call this method just before/after any significant write.
Call at the end of the writing of a record or just before we start
writing a new record. Will close current file and open a new file
if file size has passed out maxSize.
Creates and opens a file if none already open. One use of this method
then is after construction, call this method to add the metadata, then
call
WriterPoolMember.getPosition() to find offset of first record.
exception:
IOException - |
| getBaseFilename | | protected String getBaseFilename()(Code) | | Get the file name
the filename, as if uncompressed |
| getCreateTimestamp | | protected String getCreateTimestamp()(Code) | | |
| getFile | | public File getFile()(Code) | | Get this file.
Used by junit test to test for creation and when
WriterPool wants
to invalidate a file.
The current file. |
| getNextDirectory | | protected File getNextDirectory(List<File> dirs) throws IOException(Code) | |
Parameters:
dirs - List of File objects that point at directories. Find next directory to write an arc too. If morethan one, it tries to round-robin through each in turn.
throws:
IOException - |
| getPosition | | public long getPosition() throws IOException(Code) | | Postion in current physical file.
Used making accounting of bytes written.
Position in underlying file. Call before or after writingrecords *only* to be safe.
throws:
IOException - |
| getTimestampSerialNo | | protected synchronized TimestampSerialno getTimestampSerialNo(String timestamp)(Code) | | Do static synchronization around getting of counter and timestamp so
no chance of a thread getting in between the getting of timestamp and
allocation of serial number throwing the two out of alignment.
Parameters:
timestamp - If non-null, use passed timestamp (must be 14 digitARC format), else if null, timestamp with now. Instance of data structure that has timestamp and serial no. |
| isCompressed | | public boolean isCompressed()(Code) | | |
| postWriteRecordTasks | | protected void postWriteRecordTasks() throws IOException(Code) | | Post file write tasks.
If compressed, finishes up compression and flushes stream so any
subsequent checks get good reading.
exception:
IOException - |
| preWriteRecordTasks | | protected void preWriteRecordTasks() throws IOException(Code) | | Post write tasks.
Has side effects. Will open new file if we're at the upperbound.
If we're writing compressed files, it will wrap output stream with a
GZIP writer with side effect that GZIP header is written out on the
stream.
exception:
IOException - |
|
|