Code Coverage - xspec/xspec GitHub Wiki

Introduction

According to Wikipedia, code coverage is a measure (usually expressed in percentage) of the amount of source code tested by a test suite. XSpec supports code coverage for XSLT and provides a report highlighting the lines of code tested by an XSpec test.

Tutorial

The code coverage report is generated when the -c option is included when running XSpec command line or the xspec.coverage.enabled property is set to true when running XSpec with Ant.

This is an example of code coverage with the provided tutorial example included in XSpec:

$ bin/xspec.sh -c tutorial/coverage/demo.xspec
Creating Test Stylesheet...

Running Tests...
Collecting test coverage data; suppressing progress report...
****************************************
controller=net.sf.saxon.Controller@d29f28

Formatting Report...
passed: 1 / pending: 0 / failed: 0 / total: 1
Report available at tutorial/coverage/xspec/demo-coverage.html
Done.

This generates the following HTML report:

Code Coverage Report

Formatting in Report

The coverage report formats the XML nodes of the XSLT stylesheet to indicate coverage status:

ignored

hit

missed

unknown

Limitations

  • Code coverage is currently supported only for XSLT. Code coverage for XQuery and Schematron calls for development.
  • In XSpec v1.6 and v2.x, code coverage requires Saxon 9.9.
  • Starting in XSpec v3.0: Code coverage requires Saxon 12.4 or later.
  • Starting in XSpec v3.1: Nodes are marked as unknown in situations where XSpec has insufficient data to judge whether the node was hit or missed.

Using v3.0 Code Coverage in Oxygen

If you use the XSpec v3.0 XSLT Code Coverage transformation scenario in Oxygen 26.0 or 26.1, please configure it to run with Saxon 12.4 instead of 12.3. Follow these steps:

  1. Download and unzip SaxonJ 12.4 to a location of your choice.

  2. In the Transformation Scenarios pane, duplicate XSLT Code Coverage.

  3. In the Options tab of Edit Ant Scenario dialog, click Libraries.

  4. In the Libraries dialog, click Add, browse to your copy of saxon-he-12.4.jar or other Saxon 12.4 .jar file, and click OK.

  5. In the Libraries dialog, click the "Move Up" (up arrow) button repeatedly until the listing you added is above the ${oxygenHome}/lib/*saxon*12*.jar listing. Then click OK twice to finish editing your new transformation scenario.

  6. Use your duplicate transformation scenario instead of the original one.