001: /*
002: * Copyright 2007 Google Inc.
003: *
004: * Licensed under the Apache License, Version 2.0 (the "License"); you may not
005: * use this file except in compliance with the License. You may obtain a copy of
006: * the License at
007: *
008: * http://www.apache.org/licenses/LICENSE-2.0
009: *
010: * Unless required by applicable law or agreed to in writing, software
011: * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
012: * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
013: * License for the specific language governing permissions and limitations under
014: * the License.
015: */
016: package com.google.gwt.junit.client;
017:
018: /**
019: * A type of {@link com.google.gwt.junit.client.GWTTestCase} which specifically
020: * records performance results. {@link com.google.gwt.junit.client.Benchmark}s
021: * have additional functionality above and beyond GWT's JUnit support for
022: * standard <code>TestCases</code>.
023: *
024: * <ul>
025: * <li>In a single <code>JUnit</code> run, the results of all executed
026: * benchmarks are collected and stored in an XML report viewable with the
027: * <code>benchmarkViewer</code>.</li>
028: *
029: * <li>GWT automatically removes jitter from your benchmark methods by running
030: * them for a minimum period of time (150ms). GWT also optionally limits your
031: * benchmark execution to a maximum period of time (1000ms).</li>
032: *
033: * <li>GWT supports "begin" and "end" test methods that separate setup and
034: * teardown costs from the actual work being benchmarked. Simply name your
035: * functions "begin[TestMethodName]" and "end[TestMethodName]" and they will
036: * be executed before and after every execution of your test method. The
037: * timings of these setup methods are not included in the test results.</li>
038: *
039: * <li>GWT supports test methods that have parameters. GWT will execute each
040: * benchmark method multiple times in order to exhaustively test all the possible
041: * combinations of parameter values. For each parameter that your test method
042: * accepts, it should document it with the annotation,
043: * <code>@gwt.benchmark.param</code>.
044: *
045: * <p>The syntax for gwt.benchmark.param is
046: * <code><param name> = <Iterable></code>. For example,
047: *
048: * <pre>
049: * @gwt.benchmark.param where = java.util.Arrays.asList(
050: * new Position[] { Position.BEGIN, Position.END, Position.VARIED } )
051: * @gwt.benchmark.param size -limit = insertRemoveRange
052: * public void testArrayListRemoves(Position where, Integer size) { ... }
053: * </pre></p>
054: *
055: * <p>In this example, the annotated function is executed with all the possible
056: * permutations of <code>Position = (BEGIN, END, and VARIED)</code> and
057: * <code>insertRemoveRange = IntRange( 64, Integer.MAX_VALUE, "*", 2 )</code>.
058: * </p>
059: *
060: * <p>This particular example also demonstrates how GWT can automatically limit
061: * the number of executions of your test. Your final parameter (in this example,
062: * size) can optionally be decorated with -limit to indicate to GWT that
063: * it should stop executing additional permutations of the test when the
064: * execution time becomes too long (over 1000ms). So, in this example,
065: * for each value of <code>Position</code>, <code>testArrayListRemoves</code>
066: * will be executed for increasing values of <code>size</code> (beginning with
067: * 64 and increasing in steps of 2), until either it reaches
068: * <code>Integer.MAX_VALUE</code> or the execution time for the last
069: * permutation is > 1000ms.</p>
070: * </li>
071: * </ul>
072: *
073: * <p>{@link Benchmark}s support the following annotations on each test method
074: * in order to decorate each test with additional information useful for
075: * reporting.</p>
076: *
077: * <ul>
078: * <li><code>@gwt.benchmark.category</code> - The class name of the {@link Category} the
079: * benchmark belongs to. This property may also be set at the
080: * {@link com.google.gwt.junit.client.Benchmark} class level.</li>
081: * </ul>
082: *
083: * <p>Please note that {@link Benchmark}s do not currently support asynchronous
084: * testing mode. Calling
085: * {@link com.google.gwt.junit.client.GWTTestCase#delayTestFinish(int)}
086: * or {@link com.google.gwt.junit.client.GWTTestCase#finishTest()} will result
087: * in an UnsupportedOperationException.</p>
088: *
089: * <h2>Examples of benchmarking in action</h2>
090: *
091: * <h3>A simple benchmark example</h3>
092: * {@link com.google.gwt.examples.benchmarks.AllocBenchmark} is a simple example
093: * of a basic benchmark that doesn't take advantage of most of benchmarking's
094: * advanced features.
095: *
096: * {@example com.google.gwt.examples.benchmarks.AllocBenchmark}
097: *
098: * <h3>An advanced benchmark example</h3>
099: * {@link com.google.gwt.examples.benchmarks.ArrayListAndVectorBenchmark} is a more
100: * sophisticated example of benchmarking. It demonstrates the use of "begin"
101: * and "end" test methods, parameterized test methods, and automatic
102: * test execution limits.
103: *
104: * {@example com.google.gwt.examples.benchmarks.ArrayListAndVectorBenchmark}
105: */
106: public abstract class Benchmark extends GWTTestCase {
107:
108: /**
109: * The name of the system property that specifies the location
110: * where benchmark reports are both written to and read from.
111: * Its value is <code>com.google.gwt.junit.reportPath</code>.
112: *
113: * If this system property is not set, the path defaults to the user's
114: * current working directory.
115: */
116: public static final String REPORT_PATH = "com.google.gwt.junit.reportPath";
117: }
|