Getting Started - xspec/xspec GitHub Wiki

Introduction

This page demonstrates how to get started with XSpec. It runs through a tutorial that tests an XSLT stylesheet with XSpec. The examples are in the tutorial directory.

Requirements

Make sure that XSpec is installed correctly by running the help message at the end of the installation process:

Your First Test Suite

The tutorial directory contains an sample of XSLT stylesheet escape-for-regex.xsl to test and its sample XSpec test suite escape-for-regex.xspec.

The stylesheet file to test is specified in the @stylesheet attribute at the beginning of the XSpec file:

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
               xmlns:functx="http://www.functx.com"
               stylesheet="escape-for-regex.xsl">

Running a Test Suite

Navigate to your XSpec directory (e.g. ~/xspec/ for Mac/Linux or C:\xspec\ for Windows) and run the XSpec test suite with this command:

For Mac/Linux:

bin/xspec.sh tutorial/escape-for-regex.xspec

For Windows:

bin\xspec.bat tutorial\escape-for-regex.xspec

Note that you invoke the XSpec script against the XSpec test file (*.xspec), not the stylesheet file (*.xsl).

The output in the shell should be similar to this:

Creating XSpec Directory at tutorial/xspec...

Creating Test Stylesheet...
Checking for deprecated Saxon versions: Passed

Running Tests...
Testing with SAXON HE 10.9
No escaping
Must not be escaped at all
Test simple patterns
..When encountering parentheses
escape them.
..When encountering a whitespace character class
escape the backslash
result should have one more character than source
When processing a list of phrases
All phrase elements should remain
Strings should be escaped and status attributes should be added
      FAILED

Formatting Report...
passed: 5 / pending: 0 / failed: 1 / total: 6
Report available at tutorial/xspec/escape-for-regex-result.html
Done.

If everything worked, there should be a new xspec subdirectory in ~/xspec/tutorial (or C:\xspec\tutorial). It contains three files generated by XSpec:

escape-for-regex-compiled: the compiled XSpec test.
escape-for-regex-result.xml: the test results in XML format.
escape-for-regex-result.html: the human-readable HTML test report. Most of the time, this is the only file out of these three that you'll care about.

Open the HTML report at tutorial/xspec/escape-for-regex-result.html with your favorite web browser to see the test results. The HTML report should look like this:

XSpec HTML Report

As you can see, one of the tests has failed.

Dive into the XSpec tests

Open the fully commented escape-for-regex.xsl and escape-for-regex.xspec in your favorite XML editor to follow the rest of the tutorial.

The XSLT stylesheet contains a function functx:escape-for-regex and a matching template match="phrase".

The XSpec file has three top-level scenarios, which are contained in x:scenario elements. The first two test the function, and the third tests the matching templates.

The first scenario is simple, it just calls the function functx:escape-for-regex with the given parameter value and directly compares its result to our expected result. A function is invoked by using an x:call element, passing it a parameter using an x:param element. Our expectations about how the functions should behave are captured by x:expect elements:

<x:scenario label="No escaping">
   <!-- call the function with the string 'Hello' -->
   <x:call function="functx:escape-for-regex">
      <x:param select="'Hello'"/>
   </x:call>

   <!-- check the result -->
   <x:expect label="Must not be escaped at all" select="'Hello'"/>
</x:scenario>

Scenarios can be nested and the second top-level scenario is an example as it contains two scenarios that test two aspects of the function functx:escape-for-regex. The first nested scenario is an equality test, just like the first top-level scenario. The second nested scenario is more interesting as it tests the output of a function against an XPath expression given in @test. Note that we can have as many expectations as we like within a scenario:

<x:scenario label="Test simple patterns">
   <!-- call the function -->
   <x:call function="functx:escape-for-regex" />

   <!-- first sub-scenario -->
   <x:scenario label="When encountering parentheses">
      <!-- with a parameter -->
      <x:call>
         <x:param select="'(Hello)'"/>
      </x:call>

      <!-- check the result -->
      <x:expect label="escape them." select="'\(Hello\)'"/>
   </x:scenario>

   <!-- second sub-scenario -->
   <x:scenario label="When encountering a whitespace character class">
      <!-- with another parameter -->
      <x:call>
         <x:param select="'\sHello'"/>
      </x:call>

      <!-- check the result -->
      <x:expect label="escape the backslash" select="'\\sHello'"/>

      <!-- we can have several checks on the same result -->
      <x:expect label="result should have one more character than source"
                test="string-length(.) = 8"/>
   </x:scenario>
</x:scenario>

The third top-level scenario tests the templates in the transform. An x:context element contains nodes we want to apply templates to. It has two expectations: the first a simple XPath test and the second demonstrates how we can directly compare the expected nodes to the actual result nodes by embedding the expected nodes as descendants of an x:expect element:

<x:scenario label="When processing a list of phrases">
   <!-- apply template rules to this element -->
   <x:context>
      <phrases>
         <phrase>Hello!</phrase>
         <phrase>Goodbye!</phrase>
         <phrase>(So long!)</phrase>
      </phrases>
   </x:context>

   <!-- check the result -->
   <x:expect label="All phrase elements should remain"
             test="count(phrases/phrase) = 3"/>
   <x:expect label="Strings should be escaped and status attributes should be added">
      <phrases>
         <phrase status="same">Hello!</phrase>
         <phrase status="same">Goodbye!</phrase>
         <phrase status="changed">\(So long!\)</phrase>
      </phrases>
   </x:expect>
</x:scenario>

Although it's not shown here, both x:context and x:call elements have optional @href and @select attributes, which can be a very powerful way to define tests. @href is used to point to an external document, and @select is used to select certain nodes to test against.

One of the tests failed, so let's fix it. When a test fails in XSpec, the report shows both the actual result and the expected result side-by-side, with differences highlighted in pink. From this report, we can see that the failed scenario is labelled "When processing a list of phrases" and the specific expectation that failed is labelled "Strings should be escaped and status attributes should be added." The difference is that the @status attributes contain the wrong values. Change the @status attribute constructor to read:

<xsl:attribute name="status" select="if (. = $escaped-text) then 'same' else 'changed'" />

Save the XSLT stylesheet and re-run the XSpec test. All tests should be green:

XSpec HTML Report

Tips

There are many ways to integrate XSpec into your workflow. Here are a few tips:

Add the XSpec executable directory to your system path. For example, if XSpec is installed in ~/xspec, add this to your ~/.bashrc:
```
PATH=$PATH:~/xspec/bin
```
In Windows:
```
PATH %PATH%;C:\xspec\bin
```
By default, XSpec stores its result documents in the subdirectory xspec of the folder where the particular XSpec file that you are testing resides. You may want to change the variable TEST_DIR to the system temporary folder:
```
export TEST_DIR=/tmp/xspec
```
In Windows:
```
SET TEST_DIR=%TEMP%\xspec
```
This works well when you don't want to clutter up your project folders with XSpec results.
Running XSpec with Ant is far faster and more feature-rich than the shell/batch scripts.
XSpec is integrated in the Oxygen XML editor.