au.id.jericho.lib.html

A simple but powerful java library allowing analysis and manipulation of parts of an HTML document, including some common server-side tags, while reproducing verbatim any unrecognised or invalid HTML. Also provides high-level HTML form manipulation functions.

For an introduction to the API, the documentation of the {@link au.id.jericho.lib.html.Source} class is the best place to start.

For a summary of features and sample applications, visit the homepage at http://jerichohtml.sourceforge.net

For downloads, support and updates visit the SourceForge.net project page at http://sourceforge.net/projects/jerichohtml/

The Jericho HTML Parser is an open source library released under the GNU Lesser General Public License (LGPL). You are therefore free to use it in commercial applications subject to the terms detailed in the licence document.

This class extends the java.util.logging.Formatter class, allowing it to be specified as a formatter for the java.util.logging system.

The static BasicLogFormatter.format(String level,String message,String loggerName) method provides a means of using the same formatting outside of the java.util.logging framework.

Take the following HTML segment as an example:

<p>This is a sample paragraph.</p>

The whole segment is represented by an Element object.

A FormControl consists of a single that matches one of the .

The term output element is used to describe the element that is if this form control is in an OutputDocument .

A predefined value control is a form control for which FormControl.getFormControlType() . FormControlType.hasPredefinedValue hasPredefinedValue() returns true.

The FormField.getFormControls() method can be used to obtain the collection of this field's constituent FormControl objects.

The FormFields class, which represents a collection of FormField objects, provides the highest level interface for dealing with form fields and controls.

An HTML element is a normal element with a that matches one of the (ignoring case). This type of element spans the logical HTML element as described in the HTML 4.01 specification section 3.2.1, which may be implicitly terminated if it specifies an .

The term Non-HTML element refers to a normal element with a that does not match one of the . This type of element must be either a single tag element or explicitly terminated.

All of the sets returned by the methods in this class may be modified to customise the behaviour of the parser. Care must be taken however to ensure that the sets only contain tag names in lower case.

Below is a table summarising the default characteristics of each HTML element.

An OutputDocument represents an original source document that has been modified by substituting segments of it with other text. Each of these substitutions must be registered in the output document, which is most commonly done using the various replace, remove or insert methods in this class. These methods internally one or more OutputSegment objects to define each substitution. After all of the substitutions have been registered, the modified text can be retrieved using the OutputDocument.writeTo(Writer) or OutputDocument.toString() methods.

The registered may be adjacent, and as of version 2.5 may also overlap. In most cases only output segments that have been or legitimately overlap each other.

The first step in parsing an HTML document is always to construct a Source object from the source data, which can be a String, Reader, InputStream or URL. Each constructor uses all the evidence available to determine the original of the data.

Once the Source object has been created, you can immediately start searching for or within the document using the tag search methods.

In certain circumstances you may be able to improve performance by calling the Source.fullSequentialParse() method before calling any tag search methods.

Any indentation present in the original source text is removed.

Use one of the following methods to obtain the output:

• SourceFormatter.writeTo(Writer)
• SourceFormatter.toString()
• CharStreamSourceUtil.getReader(CharStreamSource) CharStreamSourceUtil.getReader(this)

The output text is functionally equivalent to the original source and should be rendered identically unless specified below.

The following points describe the process in general terms. Any aspect of the algorithm not specifically mentioned here is subject to change without notice in future versions.

• Every element that is not an appears on a new line with an indent corresponding to its in the document element hierarchy.
• The indent is formed by writing n repetitions of the string specified in the SourceFormatter.setIndentString(String) IndentString property, where n is the depth of the indentation.
• The of an indented element starts on a new line and is indented at a depth one greater than that of the element, with the end tag appearing on a new line at the same depth as the start tag. If the content contains only text and , it may continue on the same line as the start tag.

A start tag type is any TagType that with the character '<' (as with all tag types), but whose second character is not '/'.

This includes types for many tags which stand alone, without a , and would not intuitively be categorised as a "start tag".

Take the following HTML segment as an example:

<p>This is a sample paragraph.</p>

The "<p>" is represented by a StartTag object, and the "</p>" is represented by an EndTag object, both of which are subclasses of the Tag class. The whole segment, including the start tag, its corresponding end tag and all of the content in between, is represented by an Element object.

Tag Parsing Process

1. Every '<' character found in the source document is considered to be the start of a tag. The characters following it are compared with the of all the , and a list of matching tag types is determined.
2. A more detailed analysis of the source is performed according to the features of each matching tag type from the first step, in order of precedence, until a valid tag is able to be constructed.

   The analysis performed in relation to each candidate tag type is a two-stage process:

   1. The position of the tag is checked to determine whether it is . In theory, a is valid in any position, but a non-server tag is not valid inside another non-server tag.

      The TagType#isValidPosition(Source, int pos, int[] fullSequentialParseData) method is responsible for this check and has a common default implementation for all tag types (although custom tag types can override it if necessary). Its behaviour differs depending on whether or not a is peformed. See the documentation of the TagType.isValidPosition(Sourceintint[]) isValidPosition method for full details.

   2. A final analysis is performed by the TagType#constructTagAt(Source, int pos) method of the candidate tag type. This method returns a valid Tag object if all conditions of the candidate tag type are met, otherwise it returns null and the process continues with the next candidate tag type.
3. If the source does not match the start delimiter or syntax of any registered tag type, the segment spanning it and the next '>' character is taken to be an tag. Some tag search methods ignore unregistered tags.

This class is the root abstract class common to all tag types, and contains methods to and tag types as well as various methods to aid in their implementation.

Every tag type is represented by an instance of a class (usually a singleton) that must be a subclass of either StartTagType or EndTagType .