Package Specification

This package provides a key/data pair model of a database record. Databases and database cursors are the key objects used to access data. An alternative collections based API is available through com.sleepycat.collections.

The Environment class embodies the database environment and is the starting point for the application. Databases and transaction objects are created through the Environment class.

Data can be inserted and retrieved directly through the Database object, or through a Cursor obtained from the Database. A database record consist of a key/data pair, where key and data are each individually represented by a DatabaseEntry object. Classes in com.sleepycat.bind provide optional support for mapping a Java object to a DatabaseEntry.

Configuration classes are used to specify the attributes of particular operations. For example the attributes of a database environment are specified in the EnvironmentConfig class. An instance of that class is required for Environment construction. Likewise, the attributes of a database are described in DatabaseConfig, which is a parameter to the Environment.openDatabase() method. @see [Getting Started Guide]

Package Specification

Occasionally the application will choose to set a less frequently used attribute through EnvironmentConfig.setConfigParam(). The classes in this package are used with that method.

Package Specification

Package Specification

Users who want to run JE within a J2EE Application Server can use the JCA Resource Adapter to connect to JE through a standard API. The JE Resource Adapter supports all three J2EE application server transaction types:

• No transaction.
• Local transactions.
• XA transactions.

JCA also includes the Java Transaction API (JTA), which means that JE supports 2 phase commit (XA). Therefore, JE can participate in distributed transactions managed by either a J2EE server or the applications direct use of the JTA API.

Package Specification

Package Specification

Package Specification

Package Specification

Package Specification

Package Specification

• Introduction
• The Entity Model
• A brief example
• Which API to use?
• Java 1.5 dependencies
  • Generic Types
  • Annotations
• Bytecode Enhancement

The Direct Persistence Layer (DPL) was designed to meet the following requirements.

• A type safe and convenient API is provided for accessing persistent objects. The use of Java generic types, although optional, is fully exploited to provide type safety. For example:

  {@literal PrimaryIndex employerById = ...;}
  long employerId = ...;
  Employer employer = employerById.get(employerId);

• All Java types are allowed to be persistent without requiring that they implement special interfaces. Persistent fields may be {@code private}, package-private (default access), {@code protected}, or {@code public}. No hand-coding of bindings is required. However, each persistent class must have a default constructor. For example:

  {@literal @Persistent}
  class Address {
      String street;
      String city;
      String state;
      int zipCode;
      private Address() {}
  }

• Bytecode enhancement provides fully optimized bindings that do not use Java reflection.
• It is easy to define primary and secondary keys. No external schema is required and Java annotations may be used for defining all metadata. Extensions may derive metadata from other sources. For example, the following Employer class is defined as a persistent entity with a primary key field {@code id} and the secondary key field {@code name}:

{@literal @Entity}
class Employer {

    {@literal @PrimaryKey(sequence="ID")}
    long id;

    {@literal @SecondaryKey(relate=ONE_TO_ONE)}
    String name;

    Address address;

    private Employer() {}
}

• Interoperability with external components is supported via the Java collections framework. Any primary or secondary index can be accessed using a standard java.util collection. For example:

  {@literal java.util.SortedMap map = employerByName.sortedMap();}

• Class evolution is explicitly supported. Compatible changes (adding fields and type widening) are performed automatically and transparently. For example, without any special configuration a {@code street2} field may be added to the {@code Address} class and the type of the {@code zipCode} field may be changed from {@code int} to {@code long}:

  {@literal @Persistent}
  class Address {
      String street;
      String street2;
      String city;
      String state;
      long zipCode;
      private Address() {}
  }

  Many incompatible class changes, such as renaming fields or refactoring a single class, can be performed using {@link com.sleepycat.persist.evolve.Mutations Mutations}. Mutations are automatically applied lazily as data is accessed, avoiding downtime to convert large databases during a software upgrade.

  Complex refactoring involving multiple classes may be performed using the a store conversion. The DPL always provides access to your data via a {@code RawStore}, no matter what changes have been made to persistent classes.



• The performance of the Berkeley DB transactional engine is not compromised. Operations are internally mapped directly to the engine API, object bindings are lightweight, and all engine tuning parameters are available. For example, a "dirty read" may be performed using an optional {@link com.sleepycat.je.LockMode LockMode} parameter:

  Employer employer = employerByName.get(null, "Gizmo Inc", LockMode.READ_UNCOMMITTED);

  For high performance applications, {@link com.sleepycat.je.DatabaseConfig DatabaseConfig} parameters may be used to tune the performance of the Berkeley DB engine. For example, the size of an internal Btree node can be specified as follows:

  DatabaseConfig config = store.getPrimaryConfig(Employer.class);
  config.setNodeMaxEntries(64);
  store.setPrimaryConfig(config);

The DPL is intended for applications that represent persistent domain objects using Java classes. An entity class is an ordinary Java class that has a primary key and is stored and accessed using a primary index. It may also have any number of secondary keys, and entities may be accessed by secondary key using a secondary index.

An entity class may be defined with the {@link com.sleepycat.persist.model.Entity Entity} annotation. For each entity class, its primary key may be defined using the {@link com.sleepycat.persist.model.PrimaryKey PrimaryKey} annotation and any number of secondary keys may be defined using the {@link com.sleepycat.persist.model.SecondaryKey SecondaryKey} annotation.

In the following example, the {@code Person.ssn} (social security number) field is the primary key and the {@code Person.employerIds} field is a many-to-many secondary key.

{@literal @Entity}
class Person {

    {@literal @PrimaryKey}
    String ssn;

    String name;
    Address address;

    {@literal @SecondaryKey(relate=MANY_TO_MANY, relatedEntity=Employer.class)}
    {@literal Set employerIds = new HashSet();}

    private Person() {} // For bindings
}

A set of entity classes constitutes an entity model. In addition to isolated entity classes, an entity model may contain relationships between entities. Relationships may be defined using the {@link com.sleepycat.persist.model.SecondaryKey SecondaryKey} annotation. Many-to-one, one-to-many, many-to-many and one-to-one relationships are supported, as well as foreign key constraints.

In the example above, a relationship between the {@code Person} and {@code Employer} entities is defined via the {@code Person.employerIds} field. The {@code relatedEntity=Employer.class} annotation property establishes foreign key constraints to guarantee that every element of the {@code employerIds} set is a valid {@code Employer} primary key.

For more information on the entity model, see the {@link com.sleepycat.persist.model.AnnotationModel AnnotationModel} and the {@link com.sleepycat.persist.model.Entity Entity} annotation.

The root object in the DPL is the {@link com.sleepycat.persist.EntityStore EntityStore}. An entity store manages any number of objects for each entity class defined in the model. The store provides access to the primary and secondary indices for each entity class, for example:

EntityStore store = new EntityStore(...);

{@literal PrimaryIndex personBySsn} =
    store.getPrimaryIndex(String.class, Person.class);

The following example shows how to define an entity model and how to store and access persistent objects. Exception handling is omitted for brevity.

import java.io.File;
import java.util.HashSet;
import java.util.Set;

import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.persist.EntityCursor;
import com.sleepycat.persist.EntityIndex;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;
import com.sleepycat.persist.SecondaryIndex;
import com.sleepycat.persist.StoreConfig;
import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.Persistent;
import com.sleepycat.persist.model.PrimaryKey;
import com.sleepycat.persist.model.SecondaryKey;
import static com.sleepycat.persist.model.DeleteAction.NULLIFY;
import static com.sleepycat.persist.model.Relationship.ONE_TO_ONE;
import static com.sleepycat.persist.model.Relationship.ONE_TO_MANY;
import static com.sleepycat.persist.model.Relationship.MANY_TO_ONE;
import static com.sleepycat.persist.model.Relationship.MANY_TO_MANY;

// An entity class.
//
{@literal @Entity}
class Person {

    {@literal @PrimaryKey}
    String ssn;

    String name;
    Address address;

    {@literal @SecondaryKey(relate=MANY_TO_ONE, relatedEntity=Person.class)}
    String parentSsn;

    {@literal @SecondaryKey(relate=ONE_TO_MANY)}
    {@literal Set emailAddresses = new HashSet();}

    {@code @SecondaryKey(relate=MANY_TO_MANY, relatedEntity=Employer.class,
                                       onRelatedEntityDelete=NULLIFY)}
    {@code Set employerIds = new HashSet();}

    Person(String name, String ssn, String parentSsn) {
        this.name = name;
        this.ssn = ssn;
        this.parentSsn = parentSsn;
    }

    private Person() {} // For bindings
}

// Another entity class.
//
{@literal @Entity}
class Employer {

    {@literal @PrimaryKey(sequence="ID")}
    long id;

    {@literal @SecondaryKey(relate=ONE_TO_ONE)}
    String name;

    Address address;

    Employer(String name) {
        this.name = name;
    }

    private Employer() {} // For bindings
}

// A persistent class used in other classes.
//
{@literal @Persistent}
class Address {
    String street;
    String city;
    String state;
    int zipCode;
    private Address() {} // For bindings
}

// The data accessor class for the entity model.
//
class PersonAccessor {

    // Person accessors
    //
    {@literal PrimaryIndex personBySsn;}
    {@literal SecondaryIndex personByParentSsn;}
    {@literal SecondaryIndex personByEmailAddresses;}
    {@literal SecondaryIndex personByEmployerIds;}

    // Employer accessors
    //
    {@literal PrimaryIndex employerById;}
    {@literal SecondaryIndex employerByName;}

    // Opens all primary and secondary indices.
    //
    public PersonAccessor(EntityStore store)
        throws DatabaseException {

        personBySsn = store.getPrimaryIndex(
            String.class, Person.class);

        personByParentSsn = store.getSecondaryIndex(
            personBySsn, String.class, "parentSsn");

        personByEmailAddresses = store.getSecondaryIndex(
            personBySsn, String.class, "emailAddresses");

        personByEmployerIds = store.getSecondaryIndex(
            personBySsn, Long.class, "employerIds");

        employerById = store.getPrimaryIndex(
            Long.class, Employer.class);

        employerByName = store.getSecondaryIndex(
            employerById, String.class, "name"); 
    }
}

// Open a transactional Berkeley DB engine environment.
//
EnvironmentConfig envConfig = new EnvironmentConfig();
envConfig.setAllowCreate(true);
envConfig.setTransactional(true);
Environment env = new Environment(new File("/my/data"), envConfig);

// Open a transactional entity store.
//
StoreConfig storeConfig = new StoreConfig();
storeConfig.setAllowCreate(true);
storeConfig.setTransactional(true);
EntityStore store = new EntityStore(env, "PersonStore", storeConfig);

// Initialize the data access object.
//
PersonAccessor dao = new PersonAccessor(store);

// Add a parent and two children using the Person primary index.  Specifying a
// non-null parentSsn adds the child Person to the sub-index of children for
// that parent key.
//
dao.personBySsn.put(new Person("Bob Smith", "111-11-1111", null));
dao.personBySsn.put(new Person("Mary Smith", "333-33-3333", "111-11-1111"));
dao.personBySsn.put(new Person("Jack Smith", "222-22-2222", "111-11-1111"));

// Print the children of a parent using a sub-index and a cursor.
//
{@literal EntityCursor children =}
    dao.personByParentSsn.subIndex("111-11-1111").entities();
try {
    for (Person child : children) {
        System.out.println(child.ssn + ' ' + child.name);
    }
} finally {
    children.close();
}

// Get Bob by primary key using the primary index.
//
Person bob = dao.personBySsn.get("111-11-1111");
assert bob != null;

// Create two employers.  Their primary keys are assigned from a sequence.
//
Employer gizmoInc = new Employer("Gizmo Inc");
Employer gadgetInc = new Employer("Gadget Inc");
dao.employerById.put(gizmoInc);
dao.employerById.put(gadgetInc);

// Bob has two jobs and two email addresses.
//
bob.employerIds.add(gizmoInc.id);
bob.employerIds.add(gadgetInc.id);
bob.emailAddresses.add("bob@bob.com");
bob.emailAddresses.add("bob@gmail.com");

// Update Bob's record.
//
dao.personBySsn.put(bob);

// Bob can now be found by both email addresses.
//
bob = dao.personByEmailAddresses.get("bob@bob.com");
assert bob != null;
bob = dao.personByEmailAddresses.get("bob@gmail.com");
assert bob != null;

// Bob can also be found as an employee of both employers.
//
{@literal EntityIndex employees;}
employees = dao.personByEmployerIds.subIndex(gizmoInc.id);
assert employees.contains("111-11-1111");
employees = dao.personByEmployerIds.subIndex(gadgetInc.id);
assert employees.contains("111-11-1111");

// When an employer is deleted, the onRelatedEntityDelete=NULLIFY for the
// employerIds key causes the deleted ID to be removed from Bob's employerIds.
//
dao.employerById.delete(gizmoInc.id);
bob = dao.personBySsn.get("111-11-1111");
assert !bob.employerIds.contains(gizmoInc.id);

store.close();
env.close();

The example illustrates several characteristics of the DPL:

• Persistent data and keys are defined in terms of instance fields. For brevity the example does not show getter and setter methods, although these would normally exist to provide encapsulation. The DPL accesses fields during object serialization and deserialization, rather than calling getter/setter methods, leaving business methods free to enforce arbitrary validation rules. For example:

  {@literal @Persistent}
  public class ConstrainedValue {
  
      private int min;
      private int max;
      private int value;
  
      private ConstrainedValue() {} // For bindings
  
      public ConstrainedValue(int min, int max) {
          this.min = min;
          this.max = max;
          value = min;
      }
  
      public setValue(int value) {
          if (value < min || value > max) {
              throw new IllegalArgumentException("out of range");
          }
          this.value = value;
      }
  }
  

  The above {@code setValue} method would not work if it were called during object deserialization, since the order of setting fields is arbitrary. The {@code min} and {@code max} fields may not be set before the {@code value} is set.


• The example creates a transactional store and therefore all operations are transaction protected. Because no explicit transactions are used, auto-commit is used implicitly.

  Explicit transactions may also be used to group multiple operations in a single transaction, and all access methods have optional transaction parameters. For example, the following two operations are performed atomically in a transaction:

  Transaction txn = env.beginTransaction(null, null);
  dao.employerById.put(txn, gizmoInc);
  dao.employerById.put(txn, gadgetInc);
  txn.commit();
  

• To provide maximum performance, the DPL operations map directly to the Btree operations of the Berkeley DB engine. Unlike other persistence approaches, keys and indices are exposed for direct access and performance tuning.

  Queries are implemented by calling methods of the primary and secondary indices. An {@link com.sleepycat.persist.EntityJoin EntityJoin} class is also available for performing equality joins. For example, the following code queries all of Bob's children that work for Gizmo Inc:

  {@literal EntityJoin join = new EntityJoin(dao.personBySsn);}
  
  join.addCondition(dao.personByParentSsn, "111-11-1111");
  join.addCondition(dao.personByEmployerIds, gizmoInc.id);
  
  {@literal ForwardCursor results = join.entities();}
  try {
      for (Person person : results) {
          System.out.println(person.ssn + ' ' + person.name);
      }
  } finally {
      results.close();
  }
  

• Object relationships are based on keys. When a {@code Person} with a given employer ID in its {@code employerIds} set is stored, the {@code Person} object becomes part of the collection of employees for that employer. This collection of employees is accessed using a {@link com.sleepycat.persist.SecondaryIndex#subIndex SecondaryIndex.subIndex} for the employer ID, as shown below:

  {@literal EntityCursor employees =}
      dao.personByEmployerIds.subIndex(gizmoInc.id).entities();
  try {
      for (Person employee : employees) {
          System.out.println(employee.ssn + ' ' + employee.name);
      }
  } finally {
      employees.close();
  }
  

• Note that when Bob's employer is deleted in the example, the {@code Person} object for Bob is refetched to see the change to its {@code employerIds}. This is because objects are accessed by value, not by reference. In other words, no object cache or "persistence context" is maintained by the DPL. The low level caching of the embedded Berkeley DB engine, combined with lightweight object bindings, provides maximum performance.

The Berkeley DB engine has a {@link com.sleepycat.je Base API}, a {@link com.sleepycat.collections Collections API} and a {@link com.sleepycat.persist Direct Persistence Layer (DPL)}. Follow these guidelines if you are not sure which API to use:

• When Java classes are used to represent domain objects in an application, meaning that the schema is relatively static, the DPL is recommended. The more domain classes, the more value there is in using annotations to define your schema.


• When porting an application between Berkeley DB and Berkeley DB Java Edition, or when implementing your own dynamic schema (the schema of an LDAP server, for example) then the Base API is recommended. You may also prefer to use this API if you have very few domain classes.


• The Collections API is useful for interoperating with external components because it conforms to the standard Java Collections Framework. It is therefore useful in combination with both the Base API and the DPL. You may prefer this API because it provides the familiar Java Collections interface.

NOTE: The current release of the DPL requires compiling and deploying with Java 1.5 or greater. Support for Java 1.4 may be added in a future release, based on user demand.

The DPL uses two features of Java 1.5: generic types and annotations. If you wish to avoid using these two Java 1.5 features, the DPL provides options for doing so.

Generic types are used to provide type safety, especially for the {@link com.sleepycat.persist.PrimaryIndex PrimaryIndex}, {@link com.sleepycat.persist.SecondaryIndex SecondaryIndex}, and {@link com.sleepycat.persist.EntityCursor EntityCursor} classes. If you don't wish to use generic types, you can simply not declare your index and cursor objects using generic type parameters. This is the same as using the Java 1.5 Collections Framework without using generic types.

If you don't wish to use annotations, you can provide another source of metadata by implementing an {@link com.sleepycat.persist.model.EntityModel EntityModel} class. For example, naming conventions, static members, or an XML configuration file might be used as a source of metadata. However, if you don't use annotations then you won't be able to use bytecode enhancement, which is described next.

The persistent fields of a class may be private, package-private, protected or public. The DPL can access persistent fields either by bytecode enhancement or by reflection.

Bytecode enhancement may be used to fully optimize binding performance and to avoid the use of Java reflection. In applications that are CPU bound, avoiding Java reflection can have a significant performance impact.

Bytecode enhancement may be performed either at runtime or at build time (offline). When enhancement is performed at runtime, persistent classes are enhanced as they are loaded. When enhancement is performed offline, class files are enhanced during a post-compilation step; both a main program and an Ant task are provided for performing offline enhancement. In either case, enhanced classes are used to efficiently access all fields and default constructors, including non-public members.

See {@link com.sleepycat.persist.model.ClassEnhancer ClassEnhancer} for bytecode enhancement configuration details.

If bytecode enhancement is not used as described above, the DPL will use reflection for accessing persistent fields and the default constructor. The {@link java.lang.reflect.AccessibleObject#setAccessible AccessibleObject.setAccessible} method is called by the DPL to enable access to non-public fields and constructors. If you are running under a Java security manager you must configure your security policy to allow the following permission:

{@code permission java.lang.reflect.ReflectPermission "suppressAccessChecks";}

There are three cases where setting the above permission is not required:

1. If you are not running under a Java Security Manager, then access to non-public members via reflection is not restricted. This is the default for J2SE.


2. If all persistent fields and default constructors are {@code public} then they can be accessed via reflection without special permissions, even when running under a Java Security Manager. However, declaring {@code public} instance fields is not recommended because it discourages encapsulation.


3. If bytecode enhancement is used as described above, then reflection will not be used.

It is well known that executing generated code is faster than reflection. However, this performance difference may or may not impact a given application since it may be overshadowed by other factors. Performance testing in a realistic usage scenario is the best way to determine the impact. If you are determined to avoid the use of reflection then option 3 above is recommended.

Class Evolution

For persistent data that is not short lived, changes to persistent classes are almost inevitable. Some changes are compatible with existing types, and data conversion for these changes is performed automatically and transparently. Other changes are not compatible with existing types. Mutations can be used to explicitly manage many types of incompatible changes.

Not all incompatible class changes can be handled via mutations. For example, complex refactoring may require a transformation that manipulates multiple entity instances at once. Such changes are not possible with mutations but can be made by performing a store conversion.

The different categories of type changes are described below.

Key Field Changes

Unlike entity data, key data is not versioned. Therefore, the physical key format for an index is fixed once the index has been opened, and the changes allowed for key fields are very limited. The only changes allowed for key fields are:

• The name of a key field may be changed, as long as this change is accompanied by a {@link com.sleepycat.persist.evolve.Renamer} mutation.
• A primitive type may be changed to its corresponding primitive wrapper type. This is a compatible change.
• For primary key fields and fields of a composite key class, a primitive wrapper type may be changed to its corresponding primitive type. This is allowed because these key fields with reference types may never have null values. This is a compatible change.

Any other changes to a key field are incompatible and may be made only by performing a store conversion.

Key ordering, including the behavior of a custom {@link java.lang.Comparable}, is also fixed, since keys are stored in order in the index. The specifications for key ordering may not be changed, and the developer is responsible for not changing the behavior of a {@code Comparable} key class. WARNING:: Changing the behavior of a {@code Comparable} key class is likely to make the index unusable.

Compatible Type Changes

Entity data, unlike key data, is versioned. Therefore, some changes can be made compatibly and other changes can be handled via mutations. Compatible changes are defined below. To make a compatible class change, a mutation is not required; however, the class version must be assigned a new (greater) integer value.

Changes to a class hierarchy are compatible in some cases. A new class may be inserted in the hierarchy. A class may be deleted from the hierarchy as long as one of the following is true: 1) it contains no persistent fields, 2) any persistent fields are deleted with field Deleter mutations, or 3) the class is deleted with a class Deleter mutation. Classes in an existing hierarchy may not be reordered compatibly, and fields may not moved from one class to another compatibly; for such changes a class Converter mutation is required.

Changes to field types in entity class definitions are compatible when they conform to the Java Language Specification definitions for Widening Primitive Conversions and Widening Reference Conversions. For example, a smaller integer type may be changed to a larger integer type, and a reference type may be changed to one of its supertypes. Automatic widening conversions are performed as described in the Java Language Specification.

Primitive types may also be compatibly changed to their corresponding primitive wrapper types, or to the wrapper type for a widened primitive type. However, changing from a primitive wrapper type to a primitive type is not a compatible change since existing null values could not be represented.

Integer primitive types (byte, short, char, int, long) and their primitive wrapper types may be compatibly changed to the BigInteger type.

In addition, adding fields to a class is a compatible change. When a persistent instance of a class is read that does not contain the new field, the new field is initialized by the default constructor.

All other changes to instance fields are considered incompatible. Incompatible changes may be handled via mutations, as described next.

Note that whenever a class is changed, either compatibly or incompatibly, a new (higher) class version number must be assigned. See {@link com.sleepycat.persist.model.Entity#version} and {@link com.sleepycat.persist.model.Persistent#version} for information on assigning class version numbers.

Mutations

There are three types of mutations: {@link com.sleepycat.persist.evolve.Renamer}, {@link com.sleepycat.persist.evolve.Deleter} and {@link com.sleepycat.persist.evolve.Converter}.

A class or field can be renamed using a {@link com.sleepycat.persist.evolve.Renamer}. Renaming is not expensive, since it does not involve conversion of instance data.

A class or field can be deleted using a {@link com.sleepycat.persist.evolve.Deleter}.

• Deleting an entity class causes removal of the primary and secondary indices for the store, on other words, removal of all store entities for that class and its subclasses. Removal is performed when the store is opened. A {@link com.sleepycat.persist.evolve.Deleter} should be used for an entity class in all of the following circumstances:
  • When removing the entity class itself.
  • When removing {@link com.sleepycat.persist.model.Entity} from the class to make it non-persistent.
  • When removing {@link com.sleepycat.persist.model.Entity} from the class and adding {@link com.sleepycat.persist.model.Persistent}, to use it as an embedded persistent class but not an entity class. The version of the class must be incremented in this case.
• Deleting a non-entity class does not itself cause deletion of instance data, but is needed to inform DPL that the deleted class will not be used. Instances of the deleted class must be handled (discarded or converted to another class) by {@link com.sleepycat.persist.evolve.Deleter} or {@link com.sleepycat.persist.evolve.Converter} mutations for the field or enclosing class that contain embedded instances of the deleted class. A {@link com.sleepycat.persist.evolve.Deleter} should be used for a non-entity class in all of the following circumstances:
  • When removing the persistent class itself.
  • When removing {@link com.sleepycat.persist.model.Persistent} from the class to make it non-persistent.
  • When removing {@link com.sleepycat.persist.model.Persistent} from the class and adding {@link com.sleepycat.persist.model.Entity}, to use it as an entity class but not an embedded persistent class. The version of the class must be incremented in this case.
• Deleting a field causes automatic conversion of the instances containing that field, in order to discard the field values.

Other incompatible changes are handled by creating a {@link com.sleepycat.persist.evolve.Converter} mutation and implementing a {@link com.sleepycat.persist.evolve.Conversion#convert Conversion.convert} method that manipulates the raw objects and/or simple values directly. The {@code convert} method is passed an object of the old incompatible type and it returns an object of a current type.

Conversions can be specified in two ways: for specific fields or for all instances of a class. A different {@link com.sleepycat.persist.evolve.Converter} constructor is used in each case. Field-specific conversions are used instead of class conversions when both are applicable.

Note that each mutation is applied to a specific class version number. The class version must be explicitly specified in a mutation for two reasons:

1. This provides safety in the face of multiple unconverted versions of a given type. Without a version, a single conversion method would have to handle multiple input types, and would have to distinguish between them by examining the data or type information.
2. This allows arbitrary changes to be made. For example, a series of name changes may reuse a given name for more than one version. To identify the specific type being converted or renamed, a version number is needed.

See {@link com.sleepycat.persist.model.Entity#version} and {@link com.sleepycat.persist.model.Persistent#version} for information on assigning class version numbers.

Mutations are therefore responsible for converting each existing incompatible class version to the current version as defined by a current class definition. For example, consider that class-version A-1 is initially changed to A-2 and a mutation is added for converting A-1 to A-2. If later changes in version A-3 occur before converting all A-1 instances to version A-2, the converter for A-1 will have to be changed. Instead of converting from A-1 to A-2 it will need to convert from A-1 to A-3. In addition, a mutation converting A-2 to A-3 will be needed.

When a {@link com.sleepycat.persist.evolve.Converter} mutation applies to a given object, other mutations that may apply to that object are not automatically performed. It is the responsibility of the {@link com.sleepycat.persist.evolve.Converter} to return an object that conforms to the current class definition, including renaming fields and classes. If the input object has nested objects or superclasses that also need conversion, the converter must perform these nested conversions before returning the final converted object. This rule avoids the complexity and potential errors that could result if a converter mutation were automatically combined with other mutations in an arbitrary manner.

The {@link com.sleepycat.persist.EntityStore#evolve EntityStore.evolve} method may optionally be used to ensure that all instances of an old class version are converted to the current version.

Other Metadata Changes

When a class that happens to be an entity class is renamed, it remains an entity class. When a field that happens to be a primary or secondary key field is renamed, its metadata remains intact as well.

When the {@link com.sleepycat.persist.model.SecondaryKey} annotation is added to an existing field, a new index is created automatically. The new index will be populated by reading the entire primary index when the primary index is opened.

When the {@link com.sleepycat.persist.model.SecondaryKey} annotation is included with a new field, a new index is created automatically. The new field is required to be a reference type (not a primitive) and must be initialized to null (the default behavior) in the default constructor. Entities will be indexed by the field when they are stored with a non-null key value.

When a field with the {@link com.sleepycat.persist.model.SecondaryKey} annotation is deleted, or when the {@link com.sleepycat.persist.model.SecondaryKey} annotation is removed from a field without deleting it, the secondary index is removed (dropped). Removal occurs when the store is opened.

The {@link com.sleepycat.persist.model.SecondaryKey#relate SecondaryKey.relate} property may NOT be changed. All other properties of a {@link com.sleepycat.persist.model.SecondaryKey} may be changed, although avoiding changes that cause foreign key integrity errors is the responsibility of the application developer. For example, if the {@link com.sleepycat.persist.model.SecondaryKey#relatedEntity} property is added but not all existing secondary keys reference existing primary keys for the related entity, foreign key integrity errors may occur.

The {@link com.sleepycat.persist.model.PrimaryKey} annotation may NOT be removed from a field in an entity class.

The {@link com.sleepycat.persist.model.PrimaryKey#sequence} property may be added, removed, or changed to a different name.

The {@link com.sleepycat.persist.model.Persistent#proxyFor} property may be NOT be added, removed, or changed to a different class.

Warnings on Testing and Backups

The application developer is responsible for verifying that class evolution works properly before deploying with a changed set of persistent classes. The DPL will report errors when old class definitions cannot be evolved, for example, when a mutation is missing. To test that no such errors will occur, application test cases must include instances of all persistent classes.

Converter mutations require special testing. Since the application conversion method is allowed to return instances of any type, the DPL cannot check that the proper type is returned until the data is accessed. To avoid data access errors, application test cases must cover converter mutations for all potential input and output types.

When secondary keys are dropped or entity classes are deleted, the underlying databases are deleted and cannot be recovered from the store. This takes place when the store is opened. It is strongly recommended that a backup of the entire store is made before opening the store and causing class evolution to proceed.

Store Conversion