Apache™ FOP Development: Testing
"Build" Testing
Apache™ projects use an automated build tool called "jenkins" to create nightly builds from the Git repository. Jenkins sends "nag" messages if the build fails. This can be considered a sort of basic test of the build process. To view the most recent logs of the jenkins builds:
Basic/API Testing
There is a group of basic API tests that are included in the build process. For these tests to occur, Apache PDFBox jar (same version as the fontbox jar) and fop-hyph.jar should be copied to the fop/lib directory. Also JUnit must be available to Ant (simply copy junit.jar into fop's lib directory). The build will then report error(s) if the high-level APIs for Driver and the Transcoders fail. The tests do not check the output, but only ensure that something is generated and without exceptions.
Layout Engine Testing
The http://svn.apache.org/viewcvs.cgi/xmlgraphics/fop/trunk/fop/test/layoutengine/ directory in the repository contains a test suite for checking the functionality of Apache™ FOP's layout engine. For information on how to create test cases for the layout engine, please visit the Wiki page.
Functional Testing
Running and Using Tests
Testing is an important part of getting FOP to operate correctly and conform to the necessary standards.
A testing system has been set up that works with as a build target when developing with FOP. A developer can run the tests after making changes to the code, the aim is to have the tests run to verfiy that nothing working has been broken.
To setup the testing the developer must place a reference fop.jar in the "
W3C TestSuite
The testing is set up so that you can download the testsuite from http://www.w3.org/Style/XSL/TestSuite/, unzip the file into the base directory of FOP. Then you can uncomment the lines in the build.xml file in the test target and itwill run through all the tests in the testsuite distribution.
Writing a Test
A test belongs to one of a few catagories. A basic test should excercise one element in a number of situations such as changing a property. This should have at least one normal value, one border value and one invalid value. If the property can be of different types then this should also be included.
A bug test is a test that is specifically aimed at a problem with FOP. That is, the test is not excercising the specification but rather a problem with FOP in handling a particular situation that is not exposed with the other testing.
A system test is one that tests the abitlity of FOP to handle a number of different elements together.
A test can consist of a complete fo document or a part of the document such as some elements that will be placed into the flow of a standard document.
Submitting a Test
If you have a test which you think would be useful you should supply the test and a diff to the appropriate test suite xml file. Make sure that the test works as would be expected against the current build.
How Testing Works
The tests are stored in the "
You can run the tests by specifying the build target "test" ie:
ant.sh test (Unix)
ant test (Windows)
This will then compare the current code in the local src directory to a specified release of FOP. Any differences between the current code and the output from the reference version will be reported. If the test previously passed then the test run will have failed.
The testing is done by reading a test suite xml file, which corresponds to the standard testsuite.dtd supplied from w3c. This xml file contains a test xml file and an xsl file (which may simply copy the file). It also contains information such as if the test has passed and any comments.
For FOP the testing is done by rendering all the testing documents using the XML renderer. The XML files are then compared to see if there are any differences.
SVG Testing
The testing of SVG is not part of this testing system. SVG is tested for its rendering accuracy by using the transcoding mechanism via Apache Batik. So that the only part that needs testing is how the SVG image is embedded inside the flow of the fo document.