protomatter

The Protomatter Software License, Version 1.0
derived from The Apache Software License, Version 1.1

Copyright (c) 1998-2003 Nate Sammons.  All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in
   the documentation and/or other materials provided with the
   distribution.

3. The end-user documentation included with the redistribution,
   if any, must include the following acknowledgment:
      "This product includes software developed for the
       Protomatter Software Project
       (http://protomatter.sourceforge.net/)."
   Alternately, this acknowledgment may appear in the software itself,
   if and wherever such third-party acknowledgments normally appear.

4. The names "Protomatter" and "Protomatter Software Project" must
   not be used to endorse or promote products derived from this
   software without prior written permission. For written
   permission, please contact support@protomatter.com.

5. Products derived from this software may not be called "Protomatter",
   nor may "Protomatter" appear in their name, without prior written
   permission of the Protomatter Software Project
   (support@protomatter.com).

THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED.  IN NO EVENT SHALL THE PROTOMATTER SOFTWARE PROJECT OR
ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.

This project is hosted by SourceForge, who generously donate disk space and network bandwidth. Please see protomatter.sourceforge.net for the latest version of these libraries.

Architecture

The Protomatter JDBC Connection Pool Driver has three main components: the driver, the pool and the connection. The pool holds many open connections, and the driver simply asks the pool for a connection.

Driver Features

1. Dynamically grows based on use.
2. Does not automatically shrink, but you can shrink the pool manually.
3. Can automatically refresh connections that have become stale as a result of database errors (including database restarts, etc).
4. Can be configured to automatically close connections that have been idle for too long, and will log a stack trace of where the connection was first opened (this is useful for debugging).
5. Automatically blocks threads until a connection is available.
6. Drop-in replacement for "normal" drivers -- just change the driver class and URL that's being used.
7. Can be configured to log all method calls and SQL being executed.

Loading the Driver

To load the driver, simply execute this code:

Class.forName("com.protomatter.jdbc.pool.JdbcConnectionPoolDriver");

The driver automatically registers itself with the JDBC DriverManager when the class is loaded by the VM. Because the driver keeps a list of pools as a static (class) variable, and because of a bug in JDK 1.1.x's classloader, you may need to either keep a reference to an instance of the JdbcConnectionPoolDriver class somewhere where it won't be re-claimed by the garbage collector, or you may need to give java the -noclassgc flag.

Creating a new Connection Pool

To create a new JDBC Connection Pool, you need to give it a name and initialization parameters.

One example is:

// initialization params are kept in a Map
Map args = new HashMap();

// the underlying driver -- in this case, the Oracle thin driver.
args.put("jdbc.driver", "oracle.jdbc.driver.OracleDriver");

// the URL to connect the underlyng driver with the server
args.put("jdbc.URL", "jdbc:oracle:thin:@server:1521:ORCL");

// these are properties that get passed
// to DriverManager.getConnection(...)
Properties jdbcProperties = new Properties();
jdbcProperties.put("user", "admin");
jdbcProperties.put("password", "secret");
args.put("jdbc.properties", jdbcProperties);

// a statement that is guaranteed to work
// if the connection is working.
args.put("jdbc.validityCheckStatement", "SELECT 1 FROM DUAL");

// If this is specified, a low-priority thread will
// sit in the background and refresh this pool every
// N seconds.  In this case, it's refreshed every two minutes.
args.put("pool.refreshThreadCheckInterval", new Integer(120));

// the initial size of the pool.
args.put("pool.initialSize", new Integer(5));

// the maximum size the pool can grow to.
args.put("pool.maxSize", new Integer(10));

// each time the pool grows, it grows by this many connections
args.put("pool.growBlock", new Integer(2));

// between successive connections, wait this many milliseconds.
// Some database freak out if you try to open connections
// in succession too quickly.
args.put("pool.createWaitTime", new Integer(500));

// finally create the pool and we're ready to go!
JdbcConnectionPool oraclePool
  = new JdbcConnectionPool("oraclePool", args);

You can also simply use a properties file to configure pools, using the JdbcConnectionPool object.

Once a pool is created, it is automatically registered with the pool driver -- you can immediately start opening connections with it. And you don't have to keep a reference to the pool itself because the driver keeps track of it.

Getting a connection from a pool

Lets say that you have created a connection pool named "myConnectionPool" and you now want to get a connection from it. Simply do the following:

String url = "jdbc:protomatter:pool:oraclePool";
Connection c = DriverManager.getConnection(url);

The URL given to the DriverManager is just "jdbc:protomatter:pool:poolName" where poolName is the name of the pool you want a connection from.

If there is not a connection available, and the maximum size for the pool has been reached, the caller is placed in a FIFO queue of threads waiting for connections to become available, and is awakened after a connection is checked back in. If the maximum size for the queue has not been reached and there are no available connections, a new connection is created and handed out. Of course, if there is a connection available, it is handed out immediately.

You can also use the JDBC 2.x javax.sql.DataSource interface to get connections from a pool. Simply do the following:

DataSource ds = new JdbcConnectionPoolDataSource("oraclePool");
Connection c = ds.getConnection();

And then use the connection as you would normally.

Using a connection

Use a connection from the pool just like you would use a "normal" connection. The only difference is that when you call close() on it, it does not close the connection. Instead, the connection is marked closed and put back into the pool. When a connection is put back into the pool, the following is done to the underlying connecton:

1. If autocommit was set to true, commit() is called.
2. clearWarnings() is called.
3. The catalog is set back to it's original value (only if its value was changed).
4. The type map is set back to it's original value (only if its value was changed).
5. The transaction isolation level is set back to it's original value (only if its value was changed).
6. The value for the autocommit flag is set back to it's original value (only if its value was changed).
7. The value for the read-only flag is set back to it's original value (only if its value was changed).

As you might guess, it's very important to close a connection when you are done using it, or the pool will "leak" connections. We suggest using the pool within a try/catch/finally block, like this:

Connection c = null;
PreparedStatement s = null;
ResultSet r = null;
try
{
  String url = "jdbc:protomatter:pool:myConnectionPool";
  c = DriverManager.getConnection(url);
  s = c.prepareStatement("SELECT * FROM MYTABLE");
  r = s.executeQuery();
  while (r.next())
  {
    // do something with each row
  }
}
catch (SQLException x)
{
  // handle the exception
}
finally
{
  // it's important to enclose the calls to close()
  // in a try block to make sure all three get called.
  try { r.close(); } catch (SQLException x) { ; }
  try { s.close(); } catch (SQLException x) { ; }
  try { c.close(); } catch (SQLException x) { ; }
}

It's also important to make sure you close all Statement, PreparedStatement and ResultSet objects associated with a connection before you close the connection itself.

Debugging

Using the Debug class, you can enable logging of all "set" methods on the connection, and all methods on statements. This includes logging all SQL executed (and all parameters on PreparedStatements) and how long it takes to call prepareStatement(), etc.

To enable this, turn on the debug flag named "com.protomatter.jdbc.pool". If this is on, then the pool will return wrapped copies of the Statement, PreparedStatement, and CallableStatement classes. ResultSet objects are not wrapped. Messages will be logged to the com.protomatter.jdbc.pool channel using Syslog. See the JavaDoc for Syslog for more information. Logging is done on the "com.protomatter.jdbc.pool" channel in Syslog at the debug level.

Output in Syslog will look something like this:

JdbcConnectionPoolDriver        connect(jdbc:protomatter:pool:testPool, {})
ConnectionWrapper               ConnectionWrapper.createStatement() took 0ms
StatementWrapper                StatementWrapper.executeQuery("select * from test_table")
                                 = org.postgresql.jdbc2.ResultSet@40e45a call took 8ms
StatementWrapper                StatementWrapper.close() call took 0ms

ConnectionWrapper               ConnectionWrapper.prepareStatement()
                                 SQL="select * from test_table where some_column=?" took 0ms
PreparedStatementWrapper        PreparedStatementWrapper.setString(1, "foo") call took 0ms
PreparedStatementWrapper        PreparedStatementWrapper.executeQuery()
                                 = org.postgresql.jdbc2.ResultSet@4fec48 call took 10ms
PreparedStatementWrapper        PreparedStatementWrapper.close() call took 0ms

Internationalization

Every error message shown by the connection pool can displayed in a different language. Simply add a new resource (named com/protomatter/jdbc/pool/Resources) to your classpath for the new language. For instance, to add French error messages, you would copy the existing file com/protomatter/jdbc/pool/Resources.properties and make a new file called com/protomatter/jdbc/pool/Resources_fr.properties and place it in the CLASSPATH. Please see the JavaDoc for the java.util.ResourceBundle class for more information.

Licenseing

This package (as are all the {@link com.protomatter com.protomatter} packages) is licensed under the Protomatter Software License, which is the Apache Software License with appropriate name changes.

For More Information

For more information on the Protomatter JDBC Connection Pool Driver, please consult the documentation for the following classes:

{@link com.protomatter.jdbc.pool.JdbcConnectionPool com.protomatter.jdbc.pool.JdbcConnectionPool}
{@link com.protomatter.jdbc.pool.JdbcConnectionPoolDriver com.protomatter.jdbc.pool.JdbcConnectionPoolDriver}
{@link com.protomatter.jdbc.pool.JdbcConnectionPoolConnection com.protomatter.jdbc.pool.JdbcConnectionPoolConnection}
{@link com.protomatter.jdbc.pool.PoolSQLException com.protomatter.jdbc.pool.PoolSQLException}
{@link com.protomatter.pool.GrowingObjectPool com.protomatter.pool.GrowingObjectPool}
{@link com.protomatter.pool.SimpleObjectPool com.protomatter.pool.SimpleObjectPool}
{@link com.protomatter.pool.ObjectPool com.protomatter.pool.ObjectPool}
{@link com.protomatter.pool.ObjectPoolObject com.protomatter.pool.ObjectPoolObject}

The main class that other programs interact with is the {@link com.protomatter.syslog.Syslog Syslog} class. Everything on the class is static, making it easy to interact with as a service. The {@link com.protomatter.syslog.Channel Channel} class is a simplified, "federated namespace" interface for sending messages.

Messages sent to Syslog are handled by objects implementing the Syslogger interface. Syslog provides the concept of levels (DEBUG, INFO, WARNING, ERROR and FATAL). Individual loggers can have their own log policies for deciding if they should pay attention to given log message. There is a useful default policy that allows loggers to only listen to certain channels or messages at certain levels, etc.

Syslog provides startup classes for both the BEA WebLogic Server Application Server, and for Protomatter's own PAS Application Server. This allows for easy configuration of the Syslog package when each application server starts.

Please refer to the Syslog Whitepaper the JavaDoc for the {@link com.protomatter.syslog.Syslog Syslog} class for more information on usage.

For information on configuring Syslog via an XML file, please refer to JavaDoc for the {@link com.protomatter.syslog.xml.SyslogXML SyslogXML} class.

Every error message can displayed in a different language. Simply add a new resource (named com/protomatter/syslog/Syslog) to your classpath for the new language. For instance, to add French error messages, you would copy the existing file com/protomatter/syslog/Syslog.properties and make a new file called com/protomatter/syslog/Syslog_fr.properties and place it in the CLASSPATH. Please see the JavaDoc for the java.util.ResourceBundle class for more information.

Adapter classes for configuring the Commons Logging system to use Syslog as the log implementation.

You can configure the Commons Logging system by creating a file called "commons-logging.properties" in your CLASSPATH with the following properties:

## ## Tells the Apache Jakarta Commons Logging package ## to use the Syslog adapter. ## org.apache.commons.logging.LogFactory = com.protomatter.syslog.commons.SyslogChannelLogFactory   ## ## XML configuration file for syslog. Required. ## Syslog.config.xml = syslog.xml ## ## XML parser class name. Optional. You should only ## have to specify this if you get an exception loading ## the parser while configuring the adapter. ## Syslog.xml.parser = org.apache.xerces.parsers.SAXParser ## ## Display the classloader warning if we're not being ## loaded by the system classloader? Optional, ## default is "on" ## Syslog.classloader.warning = off

import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; ... Log log = LogFactory.getLog("foo.bar"); log.info("This is an info message"); log.error("This is an error message"); log.error("This is an error message", new Exception("Blah!"));   ... Log log = LogFactory.getLog(SomeClass.class); log.debug("This is an info message"); log.warn("This is an error message"); log.warn("This is an error message", new Exception("Blah!"));

Note that if you use these classes, and don't have Syslog configured to compute the caller class and method, then the caller name in your logs will be incorrect. This is because the Apache Jakarta Commons Logging API doesn't have a mechanism for directly getting a reference to the caller.

For JDK 1.2 and 1.3:

import com.protomatter.syslog.util.logging.Logger;   import com.protomatter.syslog.util.logging.Level; ... Logger logger = Logger.getLogger("logger-name"); logger.log(Level.WARNING, "warning message"); logger.log(Level.INFO, "info message"); logger.log(Level.CONFIG, "config message"); logger.severe("severe message"); logger.warning("warning message"); logger.info("info message");

You should get output in Syslog that looks like this:

05/04/2003 09:43:00 [WARN] [logger-name ] Test.main():59 warning message   05/04/2003 09:43:00 [INFO] [logger-name ] Test.main():60 info message 05/04/2003 09:43:00 [INFO] [logger-name ] Test.main():61 config message 05/04/2003 09:43:00 [EROR] [logger-name ] Test.main():58 severe message   05/04/2003 09:43:00 [WARN] [logger-name ] Test.main():59 warning message 05/04/2003 09:43:00 [INFO] [logger-name ] Test.main():60 info message

When you convert to JDK 1.4, change the import statements to:

import java.util.logging.Logger;   import java.util.logging.Level;

If you use the Syslog JDK 1.4 Handler implementation, you can get identical output.

Note that if you use these classes, and don't have Syslog configured to compute the caller class and method, then the caller name in your logs will be incorrect. This is because the JDK 1.4 logging API doesn't have a mechanism for directly getting a reference to the caller.