Philosophy of This State Design

The states are encapsulated in the State class. These can only be created by the StateSet or StateMachine classes. They are designed such that the outside managing class maintains the state objects in a well known place. This replaces the need for a String lookup, and creates a well-defined set of states which cause a better handling of in-bounds detection (compiler checked, not runtime checked).

StateSet vs. StateMachine

A StateSet is a simple mechanism for setting a state across multiple objects. It has no concept of transistions.

A StateMachine, on the other hand, has a notion of transitions. In a state machine, the current state is handled internally. Outside forces send the machine a transition, which moves the current state to another based on a matrix. This allows for control over valid transition checking, and a better real-life correspondence.

Therefore, one could say that a StateCategory is a StateMachine where the State to Transition relationship is 1-to-1, and there are no invalid transitions.

Overview

This package contains a bootstrap class (AutoDoc), and the interfaces that it delivers.

Other classes use this package by creating an instance of the AutoDoc class on a per-class basis. This AutoDoc instance then returns to the using class-specific instances for the owning class' Self-Documentation needs.

Generating Implementations of the AutoDoc Framework

This framework makes it easy to create your own implementation for each part. Here's how the framework loads in the classes.

The AutoDoc class

The AutoDoc class loads an instance of the AutoDocFactory interface. It first checks the system property "net.sourceforge.groboutils.autodoc.AutoDocFactory.implementation". The value which this is set to will be the fully-qualified class name to create as the factory (an instance of AutoDocFactory). If that is not set, then it defaults to net.sourceforge.groboutils.autodoc.defimpl.DefaultAutoDocFactory.

Then, AutoDoc requests for a new instance of each entry-point interface from the loaded factory.

As an implementation creator, you could create your own AutoDocFactory instance, and set the system property to point to your own. Or, you could leave the DefaultAutoDocFactory, and depend upon it to load your specific entry-point factory.

The DefaultAutoDocFactory loads entry-point factories in a similar fashion to how AutoDoc loads its factory: through the system properties. It uses for the implementation class name the fully-qualified class name specified in the system property of the form "factory interface fully-qualified class name.implementation

ToDo

• Right now, there can be only one factory for each section system-wide. This should be modified to allow for a hierarchy much like log4j allows, where the owning class can have its own factory set, or its package, or each package component, or finally it defaults to the system-wide setting (and if that isn't defined, then the default framework setting). Alternatively, it could load a property file from each of these namespaces. I'll leave that as a AutoDoc factory specific type to be created in the future.

Overview

This framework consists of six different classes which work together to allow for proper integration between JUnit and AutoDoc, and the implementation-specific "recorder", or server as it is referenced in this framework.

The Server interface is where the implementation-specific recorder receives data from the framework. It receives an event with framework-specific data, which has been collected over all the calls to the framework during a single JUnit test.

The Monitor interface handles the calls from both the JUnit and AutoDoc frameworks. It acts as a singleton for each JUnit test.

The MonitorFinder is a non-singleton which knows how to discover the pseudo-singleton Monitor, which is shared between the particular JUnit and AutoDoc framework instances.

The unit of test events is contained in the interface TestData. This is created by a TestDataFactory, a framework implementation specific factory in charge of creating TestData instances for the implementation classes to interact through.

The Monitor acts as a singleton for each individual test by having one TestInfo for each test, in order to uniquely identify the tests. The TestCorrelate abstract class provides functionality to create a TestInfo instance, and integrate it with the TestData through a MonitorFinder.

The Apache Ant 1.6.1 org.apache.tools.zip package. This was pulled from Ant 1.6.1 due to version inconsistencies between this package in Ant 1.5, 1.5.1, 1.5.3, and 1.6. By using a well-known version of this package, we avoid having to resort to reflection, which might cause the task to break in the future.

This package does not have any dependencies upon any other Ant package, and so can be included here without worry to its effect on Ant.

Note that this package is covered by the Apache v2 License, not the MIT license. The Apache v2 license reads:

/* * Copyright 2001-2002,2004 The Apache Software Foundation * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * */

Overview

This package has been designed to follow the JUnit extention patterns (see my article in the documentation). Instead of creating direct TestCase subclasses to add functionality to actual TestCases, test extentions have been refactored for ease-of-use.

Utility Testing Classes

The classes listed here follow the JUnit Utility Functionality pattern. They give more testing functionality to unit tests through a separate instance, rather than requiring the tests to subclass to gain the functionality.

AssertTestFactory & IntegrationTestCase

There are two kinds of "validations" a test can generate: soft and hard. A hard validation must be true - any inconsistency will result in the whole test to fail. A soft validation will still cause a test error, but the test may continue. Soft validations are less useful than hard validations for unit tests, since the point of unit tests is to make sure a small part of an application has the absolute correct behavior. However, for integration tests, this becomes less a necessity. An integration test may take great efforts setting up a scenario, then runs a battery of validations against the setup. A hard validation in these situations does not expose all the issues which may lie in the source.

Hard validations are supported in JUnit with the Assert class. The IntegrationTestCase class in this framework allows for better support of soft validations by executing the soft asserts as a seperate test. This allows for proper reporting of the failure without stopping the flow of the test.

MultiThreadedTestRunner

It can be very difficult to test code which requires inter-thread communication, or which may encounter synchronization problems. This class allows a TestCase to create runner inner classes to perform a bit of functionality for a thread, then pass those instances to a MultiThreadedTestRunner instance to allow them to run in parallel. In situations where a dead-lock may occur, a timer is provided which will interrupt (not kill) the running threads and report a failure.

Interface Testing

The Sun Java Tutorial indicates that classes which implement an interface "[sign] a contract" [1]. In some cases the contract has meta-data beyond the API declared in the interface source-code, such as "must never return null", usually declared in the JavaDoc. Forcing each implemented class to test this on its own is both poor coding practice and unenforcable. Instead, what we need is a way to write test cases for the interface, and each implemented class can execute an instance of that class against the tests. This framework eases this practice by extending the JUnit framework.

The interface creator creates a TestCase which extends InterfaceTest, commonly through InterfaceTestCase. The class to test need not be only an interface: in can be any kind of class.

Implemented classes need to use something like the following to test through an interface test:

import net.sourceforge.groboutils.junit.v1.iftc.*;
import junit.framework.*;

public MyClassTest extends TestCase {
    public MyClassTest( String name ) {
        super( name );
    }

    public static Test suite() {
        TestSuite suite = new TestSuite( MyClassTest.class )
        InterfaceTestSuite its = MyInterfaceTest.suite();
        its.addFactory( new CxFactory( "A" ) {
            public Object createImplObject() {
                return new MyClass( "string" );
            }
        } );
        its.addFactory( new CxFactory( "B" ) {
            public Object createImplObject() {
                return new MyClass( null );
            }
        } );
        suite.addTest( its );
        
        return suite;
    }
    ...
}

import net.sourceforge.groboutils.junit.v1.iftc.*;
import junit.framework.*;

public MyInterfaceTest extends InterfaceTestCase {
    public MyInterfaceTest( String name, ImplFactory f ) {
        super( name, MyInterface.class, f );
    }

    public static InterfaceTestSuite suite() {
        InterfaceTestSuite suite = new InterfaceTestSuite(
            MyInterfaceTest.class );
        
        return suite;
    }
    
    ...
}

References

1. Various. The Java Tutorial. Online at http://java.sun.com/docs/books/tutorial/java/interpack/usinginterface.html .

Overview

The TestClassParser combined with the TestClassCreator contains much of the logic that the JUnit class TestSuite contains. However, the parser is designed with extensibility in mind. Its sole purpose is to parse out test methods and create test instances.

The TestClassParser knows how to extract the test methods from a test class, while TestClassCreator knows how to create test instances from the parsed test class.

In order to support various means of instantiating test instances from a test class, the TestClassCreator depends upon instances of ITestCreator to perform the actual creation of the tests. The validation and TestSuite compilation is performed by the TestClassCreator.

Implementation

Model-Based Testing allows for testing the various routes of computation though a system to ensure system integrity. Traditional testing methodologies requires the tester to calculate and manually generate these paths.

The logic goes something like this: the tester models the system in a state-machine, using a minimalistic model where possible. At each state, the tester creates validation tests to ensure the system is indeed in that state. Also, each state has an associated set of transitions; each transition has both a validation (to ensure the transition can occur), and an action which moves the current state to another state. The MBTF then takes this model and generates combinations of paths through the states. At each state, the state itself is verified, and all transitions are verified. Any error discovered causes the path to be recorded and stamped with the error for future evaluation.

Useage

A direct user of the MBTF needs to first define an implementation of ISystem. This class contains the necessary information and actions to perform state validation and transition actions for the system in question. Also, the user needs to define an ISystemFactory, which knows how to create an ISystem in the correct initial state.

I'd recommend for your particular System, creating an abstract IVerifiy implementation which allows for reducing the casting problem inherit in a generic system such as this:

    public abstract class MyVerify implements IVerify
    {
        public final void verify( ISystem ss, IErrors errors )
        {
            myVerify( (MySystem)ss, errors );
        }
        
        public abstract void myVerify( MySystem mss, IErrors errors );
    }

Then, it's a matter of defining the system's set of states and transitions.

References

• http://www.model-based-testing.org

The primary owning task is <mbtf.stateiterator>, which contains all the states, and defines the System that will be tested. It can contain references to the states, so that the states can be defined outside the iterator.

The key part is that your ISystem instance must be registered as a Reference in the ant Project.

What the Tester Needs To Write

The Java class for the system, the validators, transitions, and so on.

What's Left To Do

There should be a validator and transition that contains an ant task. Note that the "refid" doesn't work right on tasks for our purposes: if a task has an id, then the task must have executed. So, we need to add task containers for these types.

The Issue Traceability Framework supports the framework for tracing tests to an issue tracking system. The framework uses the net.sourceforge.groboutils.pmti.v1.itf classes to define how to record and load the traceability items, while the sub-packages define specific implementations of these interfaces.

This is used by the net.sourceforge.groboutils.pmti.v1.autodoc package and sub-packages to interface the AutoDoc accumulated data with the PMTI framework proper.