JXUnit

Data-Centric Testing: Advantage of Separating Test Data from Test Code

Test data plays a central role in testing. From the lowest level (method tests) to the highest (acceptance tests), a test involves input data, expected output data, and a comparison between the expected and actual output data.

There are several advantages to separating test data from test code:

• Test data can be edited, making it easy to add additional test cases;
• Test data can be validated, as a means of reducing "false failures"; and
• Test data can be externally generated or captured from a production process.

JXUnit

JXUnit is a directory-driven test scripting system which builds on JUnit. The operation of JXUnit has three phases:

1. Search the current directory, and all subordinate directories, for test.jxu files. Each test.jxu file describes the steps needed to perform one kind of test.
2. Check for a test.jxuc file in the same directory where each test.jxu file was found. If present, the test.jxuc file identifies the directory tree and file filters for the test data to be used.
3. Finally, the tests are run.

Changes

This is the first production release of JXUnit. It has been repackaged to simplify installation. (All the necessary jar files are now included.)

New features:

• main and arg: Call the main method with a String array; and
• absolute: Convert a file name to the absolute name of a file relative to a selected directory.

The jxu.qiml and jxuc.qiml files have now been added to the JXUnit jar file--you should no longer copy these files into your test directories.

Additional documentation can now be found in the Quick Wiki Wiki. (A Wiki Wiki is a set of web pages that can be modified by any viewer.)

Test Steps and Test Properties

We can break a test into a series of small steps, where each test step does one of three things:

• Create test data,
• Run test code, or
• Compare results against expected values.

Test properties are used to pass data between the test steps. Each property has a name and a value.

OK, lets look at a first example to get an idea of where we are going with all this:

	<jxu>
		<set name="input" 
			value="123"
		<eval stepClass="DummyTestStep"/>
		<isEqual name="output" 
			value="123" 
			message="Output was not 123"/>
	</jxu>

We are using an XML markup language, Java XML Unit (JXU), to describe a series of test steps:

1. Create a property named input with a value of "123";
2. Run DummyTestStep; and
3. Check that the property named output has been created and is equal to "123". If this is not true, then the test fails with the message "Output was not 123".

The Test Framework

The class net.sourceforge.jxunit.JXTestCase provides the overall framework for JXUnit. This class extends junit.framework.TestCase and has a suit class which is called by JUnit.

A second class, net.sourceforge.jxunit.JXProperties is the container class for the test properties. This class extends java.util.HashMap.

JXU test scripts, like the example above, are conterted into a collection of objects. The classes used for set, eval, and isEqual are as follows:

These classes, as well as the DummyTestStep class, all implement the net.sourceforge.jxunit.JXTestStep interface:

	package net.sourceforge.jxunit;

	public interface JXTestStep
	{
		public void eval(JXTestCase testCase)
			throws Throwable;
	}

Now, lets look at the code for DummyTestStep:

	import net.sourceforge.jxunit.*;

	public class DummyTestStep implements JXTestStep
	{
		public void eval(JXTestCase testCase)
			throws Throwable
		{
			JXProperties properties=testCase.getProperties();
			Object data=properties.get("input");
			properties.put("output",data);
		}
	}

Note the getProperties method on JXTestCase used in the above code. This method provides access to the data being passed between test steps.

JXU

The JXU markup language is used to create a test from a series of test steps. The relationship between the JXU markup language and the JXUnit classes is described using a QJML document. For example, the following QJML fragment describes the eval element and how it relates to the JXEval class:

	<bean tag="eval">
		<rem>Create and run a test step.</rem>
		<implements>testStep</implements>
		<targetClass>net.sourceforge.jxunit.JXEval</targetClass>
		<attributes>
			<item coin="stepClass">
				<field name="stepClass"/>
			</item>
		</attributes>
	</bean>

	<text tag="stepClass">
		<rem>The fully qualified class name of the test step</rem>
	</text>

Here's the same information in tabular form:

You can easily extend JXU to include your own classes, so long as they implement JXTestStep. Here's how:

1. Update the jxu.qjml file with a description of your own classes and how they are to be accessed from XML. (Be sure to have public constructors with no arguments and make all the appropriate variables public as well.)
2. Use the Quick4 tool, qjml2qiml, to compile the revised jxu.qjml file. Name the output file jxu.qiml.
3. Expand the JXUnit jar file. (It's really just a zip file.) The contents of the jar file should be put in a classes directory.
4. Replace the jxu.qiml file in the classes\net\sourceforge\jxunit directory.
5. Drop the JXUnit jar file from the classpath, replacing it with the classes directory. (Or rebuild the jar file.)

Working with Data Files

When relative path names are used in a JXU script, the names are always relative to the directory containing the active test.jxu file. Here's an example:

	<jxu>
		<set name="input" 
			file="myData.txt"/>
		<eval stepClass="DummyTestStep"/>
		<isEqual name="output" file="myData.txt"
			message="Dummy Test Failure: output does not match myData.txt"/>
	</jxu>

Test Objects

When creating a unit test, you almost always need to create test objects to pass to the method or sub-system under test. Now, you could create your own test step classes to convert strings of text into the necessary objects. But that gets awkward as the complexity of the objects increases.

XML can be used to represent complex data structures. JXUnit can then transform these structures into test objects. Lets look at an example:

	<dataList>
		<dataItem>abc<dataItem>
		<dataItem>def<dataItem>
		<dataItem>g<dataItem>
		<dataItem>hi<dataItem>
	</dataList>

What we want to do is to convert this structure into a List containing several String objects. Here's how we would map the XML into Java:

Here then is a QJML file which defines this mapping:

	<qjml root="dataList">
		<bean tag="dataList">
			<rem>A list of String objects</rem>
			<targetClass>java.util.ArrayList</targetClass>
			<elements>
				<item coin="dataItem" repeating="true">
					<identity kind="list"/>
				</item>
			</elements>
		</bean>
		<text tag="dataItem">
			<rem>A simple text value</rem>
		</text>
	</qjml>

As before, we need to use the Quick4 utility, qjml2qiml, to convert the QJML file into a useable form. Lets name the output file mySchema.qiml. Using this file, we can now create test objects from an XML file:

	<jxu>
		<set name="input" 
			file="myData.xml" schema="mySchema.qiml"/>
		<eval stepClass="DummyTestStep"/>
		<isEqual name="output" schema="mySchema.qiml"
			file="myData.xml"
			message="Dummy Test Failure: myData.xml"/>
	</jxu>

1. The set step uses mySchema.qiml to convert the content of myData.xml into an object named input.
2. As before, the eval step invokes DummyTestStep to perform the actual test.
3. The isEqual step first converts the output object into an XML string using the mySchema.qiml binding schema. Then it compares that string to the content of the myData.xml file.

Woops! If you actually ran the above test, chances are it failed. This is because the content of myData.xml must exactly match the way the dataList object is converted into XML. Here's a quick fix to make it work:

	<jxu>
		<set name="input" 
			file="myData.xml" schema="mySchema.qiml"/>
		<save name="input" 
			file="genData.xml" schema="mySchema.qiml"/>
		<eval stepClass="DummyTestStep"/>
		<isEqual name="output" schema="mySchema.qiml"
			file="genData.xml"
			message="Dummy Test Failure: genData.xml"/>
	</jxu>

We've added another step, save, which creates an XML file from the input object. Now the string comparison of isEqual should work fine!

Creating Test Data

When a test fails, it would be great if we could keep a copy of the data, and even better if the data is saved in a human-readable format. Here's how:

	<jxu>
	    <set name="input" 
		 file="myData.xml" schema="mySchema.qiml"/>
	    <eval stepClass="DummyTestStep"/>
	    <ifEqual converse="true"
		     name="output" schema="mySchema.qiml"
		     file="myData.xml">
		<save name="output" schema="mySchema.qiml"
		      file="myData_.xml"/>
		<fail>Dummy Test Failure: myData.xml</fail>
	    </ifEqual>
	</jxu>

1. The set step creates an object named input.
2. The eval step performs the actual test.
3. The ifEqual step converts the output object to XML and compares it to the content of myData.xml. If not equal (converse="true"), then the nested steps are performed:
1. The save step converts the output object into XML and writes it to the myData_.xml file.
2. The fail step then ends the test as a failure.

Testing Context: Running the Same Test with Multiple Data Files

It is often not sufficient to just run a test. You may want to run the test more than once, using multiple threads, and/or with multiple test files. Lets call this the context of the test. The test context is defined by the test.jxuc file, which must be in the same directory which holds the active test.jxu file.

As mentioned above, there are many things that could be done here. For now, we support the use of multiple test files, as this is the most central to data-centric testing. (Here is the table describing the currently-supported JXUC elements.) But lets begin by looking at a sample JXUC document:

	<jxuc>
		<directoryScan dir="testDirectory">
			<includeFiles regexp=".txt$"/>
			<excludeFiles regexp="_.txt$"/>
		</directoryScan>
	</jxuc>

DirectoryScan gives us the option of selecting files from a directory. Any number of includeFiles and excludeFiles elements can be used to make the selection. (If there are no includeFiles, then all the files in the selcted directory are included by default.)

Regular Expressions are used to filter files based on file name. In the above example, all files ending in .txt, except for files ending in _.txt, are selected.

The test defined by the test.jxu file is run once for each selected file. The absolute path name of the selected file is used as the test name and is passed to the test steps using a property named absDataFileName. The simple file name is also passed, in a property named dataFileName. Here's an example of a test.jxu file which can be applied to more than one test file:

	<jxu>
		<set name="data" file="absDataFileName" indirect="true"/>
		<ifEqual converse="true" name="data" value="dataFileName" indirect="true">
			<save name="data" file="badFileName_.txt"/>
			<fail>Each test file must contain only its own name!</fail>
		</ifEqual>
	</jxu>

1. The data property is set to the content of the file named by the absDataFileName property. (The indirect flag here indicates that the file attribute names a property which holds the file name.)
2. The value of the data property is then compared to the file name held by the dataFileName property. (Here the indirect flag indicates that the value of the property named by the value attribute should be used in the comparison.) If the two values are NOT equal (converse="true"), then
1. The value of the data property is saved in a file named badFileName_.txt; and
2. The test fails.

Testing Context: Framework

The JXUC elements are mapped into Java classes, as follows:

All of the classes used in defining the testing context must implement the JXTestSetup interface:

	package net.sourceforge.jxunit;

	public interface JXTestSetup
	{
		public void setup(String cwd, JXProperties properties)
			throws Throwable;
	}

• The cwd parameter is used to pass the name of the directory where the active test.jxu and test.jxuc files are located.
• The properties parameter is used to pass the container which holds all the test properties.

Once the setup is complete for the testing context, the property named candidateFiles, if not null, is expected to hold a list of file names. A test is run with each file named in the list. Each test is run with its own copy of the JXProperties object that was initialized during setup.

Like JXU, you can extend JXUC to include your own classes, so long as they implement JXTestSetup:

1. Update the jxuc.qjml file with a description of your own classes and how they are to be accessed from XML. (Be sure to have public constructors with no arguments and make all the appropriate variables public as well.)
2. Use the Quick4 tool, qjml2qiml, to compile the revised jxuc.qjml file. Name the output file jxuc.qiml.
3. Expand the JXUnit jar file. (It's really just a zip file.) The contents of the jar file should be put in a classes directory.
4. Replace the jxuc.qiml file in the classes\net\sourceforge\jxunit directory.
5. Drop the JXUnit jar file from the classpath, replacing it with the classes directory. (Or rebuild the jar file.)

Getting Started

Everything you need is now included in the download file. Depending on where you unzip this file, you may need to modify the setup.bat file.

Just run setup each time before using JXUnit. It modifies the PATH and CLASSPATH variables.

Running the Tests

Running a test is pretty straight forward:

        java junit.textui.TestRunner net.sourceforge.jxunit.JXTestCase

The command line above can be run from any directory. JXTestCase searches the current directory and all child directories for test.jxu files. Each file found becomes a test case, with the name of the directory used as the test name. This gives you control over which tests are run simply by changing directories.

Links

• Download JXUnit
• JXUnit topic page in the Quick Wiki Wiki.
• JUnit Home Page
• Quick Home Page
• JXUnit Project Summary Page
• JXUnit Forum
• License (Modified BSD)
• JXU DTD and reference