Java Doc for LogToFile.java in  » Database-DBMS » db-derby-10.2 » org » apache » derby » impl » store » raw » log » Java Source Code / Java DocumentationJava Source Code and Java Documentation

The 'log' is a stream of log records. The 'log' is implemented as a series of numbered log files. These numbered log files are logically continuous so a transaction can have log records that span multiple log files. A single log record cannot span more then one log file. The log file number is monotonically increasing.

The log belongs to a log factory of a RawStore. In the current implementation, each RawStore only has one log factory, so each RawStore only has one log (which composed of multiple log files). At any given time, a log factory only writes new log records to one log file, this log file is called the 'current log file'.

A log file is named loglogNumber.dat

Everytime a checkpoint is taken, a new log file is created and all subsequent log records will go to the new log file. After a checkpoint is taken, old and useless log files will be deleted.

RawStore exposes a checkpoint method which clients can call, or a checkpoint is taken automatically by the RawStore when

1. the log file grows beyond a certain size (configurable, default 100K bytes)
2. RawStore is shutdown and a checkpoint hasn't been done "for a while"
3. RawStore is recovered and a checkpoint hasn't been done "for a while"

This LogFactory is responsible for the formats of 2 kinds of file: the log file and the log control file. And it is responsible for the format of the log record wrapper.

 Format of log control file 

This method is synchronized to ensure log records are added sequentially to the end of the log.

MT- single threaded through this log factory.

MT- Only one checkpoint is to be taking place at any given time.

The steps of a checkpoint are

1. switch to a new log file if possible

    freeze the log (for the transition to a new log file)
    flush current log file
    create and flush the new log file (with file number 1 higher
    than the previous log file).

MT - This method is synchronized to ensure that it always points to the end of a log record, not the middle of one.

MT- read only
Parameters:
  startAt - - if startAt == INVALID_LOG_INSTANT,start from the beginning of the log.

In this implementation, the log is a stream of log records stored in one or more flat files.

This method is synchronized to ensure log records are added sequentially to the end of the log.

MT- single threaded through this log factory. Log records are appended one at a time.
exception:
  StandardException - Log Full.

MT- caller provide synchronization
exception:
  StandardException - log factory cannot start up

MT- Only one checkpoint is to be taking place at any given time.

The steps of a checkpoint are

1. switch to a new log file if possible

    freeze the log (for the transition to a new log file)
    flush current log file
    create and flush the new log file (with file number 1 higher
    than the previous log file). The new log file becomes the
    current log file.
    unfreeze the log
    

2. start checkpoint transaction
3. gather interesting information about the rawStore: the current log instant (redoLWM) the earliest active transaction begin tran log record instant (undoLWM), all the truncation LWM set by clients of raw store (replication)
4. clean the buffer cache
5. log the next checkpoint log record, which contains (repPoint, undoLWM, redoLWM) and commit checkpoint transaction.
6. synchronously write the control file containing the next checkpoint log record log instant
7. the new checkpoint becomes the current checkpoint. Somewhere near the beginning of each log file should be a checkpoint log record (not guarenteed to be there)
8. see if the log can be truncated

   The earliest useful log record is determined by the repPoint and the undoLWM, whichever is earlier.

   Every log file whose log file number is smaller than the earliest useful log record's log file number can be deleted.

    Transactions can be at the following states w/r to a checkpoint -
    consider the log as a continous stream and not as series of log 
    files for the sake of clarity.  
    |(BT)-------(ET)| marks the begin and end of a transaction.
    .                          checkpoint started
    .       |__undoLWM          |
    .       V                   |___redoLWM
    .                           |___TruncationLWM
    .                           |
    .                           V
    1 |-----------------|
    2       |--------------------------------|
    3           |-------|
    4               |--------------------------------------(end of log)
    5                                       |-^-|
    .                                   Checkpoint Log Record
    ---A--->|<-------B--------->|<-------------C-----------
    

   There are only 3 periods of interest :
   A) before undoLWM, B) between undo and redo LWM, C) after redoLWM.

   Transaction 1 started in A and terminates in B.
   During redo, we should only see log records and endXact from this transaction in the first phase (between undoLWM and redoLWM). No beginXact log record for this transaction will be seen.

   Transaction 2 started in B (right on the undoLWM) and terminated in C.
   Any transaction that terminates in C must have a beginXact at or after undoLWM. In other words, no transaction can span A, B and C. During redo, we will see beginXact, other log records and endXact for this transaction.

   Transaction 3 started in B and ended in B.
   During redo, we will see beginXact, other log records and endXact for this transaction.

   Transaction 4 begins in B and never ends.
   During redo, we will see beginXact, other log records. In undo, this loser transaction will be rolled back.

   Transaction 5 is the transaction taking the checkpoint.
   The checkpoint action started way back in time but the checkpoint log record is only written after the buffer cache has been flushed.

   Note that if any time elapse between taking the undoLWM and the redoLWM, then it will create a 4th period of interest.
   exception:
     StandardException - - encounter exception while doing checkpoint.

MT - This method is synchronized to ensure that it always points to the end of a log record, not the middle of one.

MT - not needed, wrapper method
Parameters:
  where - flush log up to here
exception:
  StandardException - Standard Cloudscape error policy

MT - only one flush is allowed to be taking place at any given time (RESOLVE: right now it single thread thru the log factory while the log is frozen)
exception:
  StandardException - cannot sync log file

MT - not needed, wrapper method
exception:
  StandardException - Standard Cloudscape error policy

This only works after running recovery the first time.

MT - RESOLVE:

MT- read only
exception:
  StandardException - Standard Cloudscape error policy
exception:
  IOException - cannot access the log at the new position.

MT- read only null if file does not exist or of the wrong format
exception:
  IOException - cannot access the log at the new position.
exception:
  StandardException - Standard Cloudscape error policy

MT- read only
exception:
  IOException - cannot access the log
exception:
  StandardException - Standard Cloudscape error policy

MT- read only
exception:
  IOException - cannot access the log
exception:
  StandardException - Standard Cloudscape error policy

MT- read only
exception:
  StandardException - Standard cloudscape exception policy

MT- read only
Parameters:
  startAt - - if startAt == INVALID_LOG_INSTANT,start from the beginning of the log. Otherwise, start scan from startAt.
Parameters:
  stopAt - - if not null, stop at this log instant (inclusive).Otherwise, stop at the end of the log
exception:
  IOException - cannot access the log
exception:
  StandardException - Standard Cloudscape error policy

In this implementation, the log is a stream of log records stored in one or more flat files. Recovery is done in 2 passes: redo and undo.
Redo pass
In the redo pass, reconstruct the state of the rawstore by repeating exactly what happened before as recorded in the log.
Undo pass
In the undo pass, all incomplete transactions are rolled back in the order from the most recently started to the oldest.

MT - synchronization provided by caller - RawStore boot. This method is guaranteed to be the only method being called and can assume single thread access on all fields.
See Also:   Loggable.needsRedo
See Also:   FileLogger.redo
exception:
  StandardException - Standard Cloudscape error policy

MT- caller provide synchronization (RESOLVE: this should be called AFTER dataFactory and transFactory are stopped)

MT- synchronized by caller