Writing Scenarios - xspec/xspec GitHub Wiki

• Introduction
• Matching Scenarios
• Function Scenarios
• Named Template Scenarios
• Expectations
• Global Parameters
• XSpec Variables

The description of the behaviour of a stylesheet lives within an XSpec document, which should adhere to the XSpec RELAX NG schema. All elements are in the http://www.jenitennison.com/xslt/xspec namespace, which is bound to x in these examples.

The document element is a x:description element, whose stylesheet attribute holds a relative URI pointing to the stylesheet that the XSpec document describes.

The x:description element contains a number of x:scenario elements, each of which describes a particular scenario that's being tested. Each x:scenario element has a label attribute that describes the scenario in human language. For example:

<x:scenario label="when processing a para element">
   ...
</x:scenario>

Scenarios fall into four main types:

• matching scenarios describe the result of applying templates to a node in a particular mode (and with particular parameters)
• function scenarios describe the results of calling a particular function with particular arguments
• named template scenarios describe the results of calling a particular named template with particular parameters
• Schematron scenarios, covered in Writing Scenarios for Schematron

Matching scenarios hold a x:context element that describes a node to apply templates to.

The context can be supplied in two main ways:

• you can point to a node in an existing document by giving the document URI in the href attribute and, if you want, selecting a particular node by putting a path in the select attribute
• you can embed XML within the x:context element; the content becomes the context node, although you can also select a node within that XML using the select attribute

The first method is useful if you already have example XML documents that you want to use as the basis of your testing. For example:

<x:scenario label="when processing a para element">
   <x:context href="source/test.xml" select="/doc/body/p[1]" />
   ...
</x:scenario>

The second method is related to the concept of a mock object: It is an example of some XML which you have created simply for testing purposes. The XML might not be legal; it only needs to have the attributes or content necessary for the particular behaviour that needs to be tested. For example:

<x:scenario label="when processing a para element">
   <x:context>
      <para>...</para>
   </x:context>
   ...
</x:scenario>

A big difference between the methods lies in how whitespace-only text nodes are handled. With the first method (x:context/@href), whitespace-only text nodes are kept. With the second method (x:context/node()), whitespace-only text nodes are discarded.

An additional difference is in the data type produced when x:context has no select attribute; see Selecting Nodes for details.

The x:context element can also have a mode attribute that supplies the mode to apply templates in:

<x:scenario label="when processing a para element in 'shortdesc' mode">
   <x:context mode="shortdesc">
      <para>...</para>
   </x:context>
   ...
</x:scenario>

If you want to test a template rule that inherits its mode from a @default-mode attribute at the top level of the stylesheet, you must either

• Specify the @mode attribute on x:context, or
• Use the external transformation feature, if it suits your needs. With external transformations, x:context without a @mode attribute uses the mode specified in the stylesheet's @default-mode attribute. Note that the external transformation feature has some limitations and is considered experimental; see External Transformations.

The x:param element in x:context supplies a parameter to apply templates with:

<x:scenario label="when processing a para element with an 'indent' parameter">
   <x:context>
      <x:param name="indent" select="'2'" />
      <para>...</para>
   </x:context>
   ...
</x:scenario>

x:param can also have a tunnel attribute to indicate a tunnel parameter:

<x:scenario label="when processing a para element with a tunnel 'indent' parameter">
   <x:context>
      <x:param name="indent" select="'2'" tunnel="yes" />
      <para>...</para>
   </x:context>
   ...
</x:scenario>

Function scenarios hold a x:call element with a function attribute whose content is a qualified name of the function you want to call. The x:call element should hold x:param elements, one for each of the arguments to the function.

The x:param elements can specify node values in the same way as the x:context element gets set, or simply by giving a select attribute which holds an XPath that specifies the value. You can specify a position attribute for each of the x:param elements; if you don't, the order in which they're specified will determine the order in which they're given in the function call. For example:

<x:scenario label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="'an example string'" />
      <x:param select="true()" />
   </x:call>
   ...
</x:scenario>

will result in the function call eg:capital-case('an example string', true()), as will the following:

<x:scenario label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="true()" position="2" />
      <x:param select="'an example string'" position="1" />
   </x:call>
   ...
</x:scenario>

Like x:context element, you can use @href and @select attributes and embed XML within x:param.

Starting in XSpec v3.0.3: To call an XPath function that is stored in an XSLT variable or XQuery variable, set call-as="variable" on the x:call element.

Named template scenarios are similar to function scenarios except that

• the x:call element takes a template attribute rather than a function attribute
• the x:param elements within x:call must have a name attribute that supplies the name of the parameter

For example:

<x:scenario label="when capitalising a string">
   <x:call template="capital-case">
      <x:param name="input-string" select="'an example string'" />
      <x:param name="each-word" select="true()" />
   </x:call>
   ...
</x:scenario>

Optionally you can provide a context item (x:context element) in which the named template is called:

<x:scenario label="Creating a two-column table when the context node is a 'data' element containing three 'value' elements">
   <x:context>
      <data>
         <value>A</value>
         <value>B</value>
         <value>C</value>
      </data>
   </x:context>
   <x:call template="createTable" />
      <x:param name="cols" select="2" />
   </x:call>
   ...
</x:scenario>

If you do not want to provide a context item, omit x:context instead of making it an empty element.

Each scenario can have one or more "expectations": things that should be true of the result of the function or template invocation described by the scenario. Each expectation is specified with an x:expect element. The label attribute on the x:expect element gives a human-readable description of the expectation.

There are three ways of describing expectations in x:expect:

• Describe only the expected result.
  • XSpec compares it with the actual result.
• Describe the expected result and filter the actual result.
  • XSpec compares the described expected result with the filtered actual result.
• Describe an xs:boolean XPath expression.
  • Your XPath expression determines whether the test is Success (xs:boolean true) or Failure (xs:boolean false).

If you describe only the expected result, XSpec compares it with the actual result. If they are deep-equal, the test is Success.

You can use @select, embedded XML and @href (external XML) to describe the expected result:

• Only @select

  <x:scenario label="when calling a template">
     <x:call template="generate-strings" />
     <x:expect
        label="the result should be a sequence of two strings"
        select="'foo', 'bar'" />
  </x:scenario>

• Embedded XML

  <x:scenario label="when calling a template">
     <x:call template="generate-element" />
     <x:expect label="the result should be a foo element containing a bar element">
        <foo>
           <bar />
        </foo>
     </x:expect>
  </x:scenario>

• @href

  <x:scenario label="when calling a template">
     <x:call template="generate-doc" />
     <x:expect
        label="the result should be a document node equal to expected.xml"
        href="expected.xml" />
  </x:scenario>

• Embedded XML and @select

  <x:scenario label="when calling a template">
     <x:call template="generate-attribute" />
     <x:expect
        label="the result should be a foo attribute whose value is bar"
        select="e/@*"
        as="attribute(foo)">
        <e foo="bar" />
     </x:expect>
  </x:scenario>

  Note that you must be careful writing @select. In this example, if you write select="e/@bar" (an inadvertent error in writing select="e/@foo") without @as and the tested template generates an empty sequence, then the test is Success because @select is also an empty sequence.

  Also note that @select matches elements not in any namespace. To work with XML in a namespace, see Namespaces in XPath expressions.

• @href and @select

  <x:scenario label="when calling a template">
     <x:call template="generate-element" />
     <x:expect
        label="the result should be equal to the bar element in expected.xml"
        href="expected.xml"
        select="foo/bar"
        as="element(bar)" />
  </x:scenario>

  Note that you must be careful writing @select. In this example, if you omit @as when expected.xml inadvertently does not have the specified bar element and the tested template generates an empty sequence, then the test is Success because @select is also an empty sequence.

  Also note that @select matches elements not in any namespace. To work with XML in a namespace, see Namespaces in XPath expressions.

For additional information about nodes as expected results, including data types, see Selecting Nodes.

If you want to test only some portions of the actual result, you can filter the actual result with @test. XSpec evaluates @test and compares its evaluation result with the expected result. If they are deep-equal, the test is Success.

Even when @test is used, the way of describing the expected result is still the same as describing only the expected result.

For example:

<x:scenario label="when creating a table with two columns containing three values">
   <x:call template="createTable">
      <x:param name="nodes">
         <value>A</value>
         <value>B</value>
         <value>C</value>
      </x:param>
      <x:param name="cols" select="2" />
   </x:call>
   <x:expect
      label="the table should have two columns"
      test="/table/colgroup/col => count()"
      select="2" />
   <x:expect
      label="the first row should contain the first two values as described in this embedded XML"
      test="/table/tbody/tr[1]">
      <tr>
         <td>A</td>
         <td>B</td>
      </tr>
   </x:expect>
   <x:expect
      label="the second row should be equal to the third row in expected.xml"
      test="/table/tbody/tr[2]"
      href="expected.xml"
      select="/html/body/table/tbody/tr[3]" />
</x:scenario>

Note that in this case,

• @test must not be an instance of xs:boolean. If @test is an instance of xs:boolean, then x:expect is considered to be expressing the boolean test result.
• x:expect must have @as, @href, @select or a child node.

Sometimes you may want to determine Success or Failure by yourself instead of letting XSpec compare the expected result with the actual result. In that case, you should describe an xs:boolean XPath expression in @test and do not describe the expected result anywhere else in x:expect. Then your XPath expression determines whether the test is Success (xs:boolean true) or Failure (xs:boolean false).

For example:

<x:scenario label="when creating a table">
   <x:call template="createTable" />
   <x:expect
      label="its width should be greater than 100"
      test="/table/@width > 100" />
</x:scenario>

Note that in this case,

• @test must be an instance of xs:boolean. If @test is not an instance of xs:boolean, then x:expect is considered to be filtering the actual result.
• x:expect must not have @as, @href, @select or a child node.

The path /table expressed in @test matches elements <table> that are not in any namespace. To work with XML in a namespace, @test must indicate the namespace using a prefix or Q{...} notation. Details are analogous to the discussion of @select in Namespaces in XPath expressions.

Do not confuse xs:boolean with the effective boolean value. XSpec's @test does not work like XSLT's xsl:if/@test. XSLT's @test takes the effective boolean value.

When comparing the actual result with the expected result, ... (three dots) in an element or attribute value within the expected XML means that the corresponding portions aren't compared.

For example, if the actual result is

<p>A sample para</p>

and the expected result is given as

<p>...</p>

then they match. If the expected result is

<p>Some other para</p>

then they don't.

Starting in XSpec v3.0.3: In addition, the attribute x:attrs="..."on an element in the expected result makes the comparison ignore any attributes you don't specify explicitly. For example, if the actual result is

<p date="2000-01-01" id="e103">A sample para</p>

then <p x:attrs="...">A sample para</p> matches. An alternative form <p date="..." id="...">A sample para</p> also matches, but it goes further by checking that the element has those two attributes and no others.

Not your use case? If you need to test that an element's content is literally three dots, the three dot feature of node comparisons is not suitable. Instead, compare strings. For instance:

<x:expect label="The 2nd p has a text node of ..." select="'...'"
   test="div/p[2]/text()/string()" />

For more examples of the three dot feature, see the test scenarios and their results.

In @test, you can use the variable $x:result to access the actual result as is (i.e. the raw result of calling the function or the template, or of applying the template rule).

• If the actual result is a sequence of nodes except attribute and namespace nodes, it is wrapped in a document node and this document node is set as the context item (.) of the XPath expression in @test.
• Else if the actual result is a single item, the raw actual result ($x:result) is set as the context item (.) of @test.
• Else the context item (.) in @test is absent.

• Nodes in the wrapped . always have a tree relationship. Items in $x:result do not always have a common root.
• The wrapped . merges adjacent text nodes into one text node. Such merging doesn't happen in $x:result unless it happened within the tested stylesheet.

For example, with this stylesheet and XSpec, all x:expect tests are Success:

Stylesheet

<xsl:template name="multiple-elements" as="element()+">
   <foo />
   <bar />
</xsl:template>

<xsl:template name="multiple-text-nodes" as="text()+">
   <xsl:text>foo</xsl:text>
   <xsl:value-of select="'bar'" />
</xsl:template>

XSpec

<x:scenario label="generate multiple elements">
   <x:call template="multiple-elements" />
   <x:expect
      label="the first child element is followed by a bar element, when the actual result is wrapped in a document node"
      test="element()[1]/following-sibling::element()">
      <bar />
   </x:expect>
   <x:expect
      label="the first item in the raw actual result is an element followed by no nodes"
      test="
         ($x:result[1] treat as element())/following-sibling::node()
         => empty()" />
</x:scenario>

<x:scenario label="generate multiple text nodes">
   <x:call template="multiple-text-nodes" />
   <x:expect
      label="only one child text node exists, when the actual result is wrapped in a document node"
      test="text() => count()"
      select="1" />
   <x:expect
      label="Two text nodes exist in the raw actual result"
      test="$x:result/self::text() => count()"
      select="2" />
</x:scenario>

Depending on the actual result, the context item (.) in @test may be absent or may be different from what you expect. In @test, if you like to inspect the actual result as is, you must use $x:result instead of the context item (.).

The context item (.) in @test is available only on XSLT. On XQuery, the context item (.) in @test is always absent.

You can put x:param elements at the top level of the XSpec description document (as a child of the x:description element) or at the scenario level (as a child of the x:scenario element). These effectively override any global parameters or variables that you have declared in your stylesheet. They are set in just the same way as setting parameters when testing named templates or functions.

Since the scenario-level x:param is not available by default, different scenarios cannot use different global parameters by default. Testing is made easier if you declare local parameters on any templates or functions that use global parameters; these can default to the value of the global parameter, but be set explicitly when testing. For example, if $tableClass is a global parameter, you might do the following to enable the full testing of the createTable template:

<xsl:template name="createTable">
   <xsl:param name="nodes" as="node()+" required="yes" />
   <xsl:param name="cols" as="xs:integer" required="yes" />
   <xsl:param name="tableClass" as="xs:string" select="$tableClass" />
   ...
</xsl:template>

The scenario-level x:param is available only when /x:description/@run-as is external and some restrictions are applied. See its page for details.

You can define variables that are specific to your XSpec description document, as opposed to overrides of variables in the code you are testing. Reasons for using XSpec variables can include reusing data within the XSpec file and naming intermediate results for clarity.

To define an XSpec variable, use the x:variable element as a child of x:description or x:scenario. Each variable has a required name attribute. Consider putting XSpec variable names in your own namespace, although doing so is not required. To specify a data type, use the optional as attribute.

Define the value of the variable using one of these approaches:

• Use the select attribute alone to provide an XPath expression that specifies the value
• Use attributes href and, optionally, select to point to a node in an existing document
• Embed XML within the x:variable element
• Embed XML within the x:variable element and select a node within that XML using the select attribute

For additional information about values that are nodes, including data types, see Selecting Nodes.

Here are some examples of variable definitions, assuming you have bound a namespace to the prefix myv.

<x:variable name="myv:mystring" select="'text'" as="xs:string" />
<x:variable name="myv:mydoc" href="mydoc.xml" as="document-node()?" />
<x:variable name="myv:mysections" href="mydoc.xml" select="//section" as="element()*" />
<x:variable name="myv:mypara" as="element(p)">
   <p><span>text</span></p>
</x:variable>
<x:variable name="myv:myspan" select="//span" as="element(span)">
   <p><span>text</span></p>
</x:variable>

After defining a variable, you can refer to it by name (prefixed with $) in XPath expressions in the same XSpec file. Valid locations where you can refer to XSpec variables include:

• select attribute in x:context, x:param, x:expect, and x:variable
• test attribute in x:expect
• An attribute value template in embedded XML, such as <x:context><mycontext role="{$myv:myvariable}" /></x:context>
• A text value template in embedded XML, such as <x:context><x:text expand-text="yes">{$myv:myvariable}</x:text></x:context>

Here is an example showing several references to XSpec variables. The example assumes you have declared namespaces (with prefixes my for code, myv for XSpec variables, and db for elements in the XML content), defined the my:select-figure function, and created a suitable mydoc.xml XML document with a section containing several child figure elements.

<x:scenario label="select-figure function">
   <x:variable name="myv:topic" as="element(db:section)"
      href="mydoc.xml" select="//db:section[@xml:id = 'topicwithimages']" />
   <x:scenario label="with one nonempty argument">
      <x:call function="my:select-figure">
         <x:param select="$myv:topic" />
         <x:param select="''" />
      </x:call>
      <x:variable name="myv:last-figure" as="element(db:figure)"
         select="($myv:topic/db:figure)[last()]" />
      <x:expect label="selects the last figure in the section"
         test="deep-equal($x:result, $myv:last-figure)" />
   </x:scenario>
   <x:scenario label="with two nonempty arguments">
      <x:variable name="myv:sample-id" select="'scatterplot'" as="xs:string" />
      <x:call function="my:select-figure">
         <x:param select="$myv:topic" />
         <x:param select="$myv:sample-id" />
      </x:call>
      <x:expect label="selects the figure with specified ID"
         select="$myv:topic/db:figure[@xml:id = $myv:sample-id]" />
   </x:scenario>
</x:scenario>

Due to inheritance, you can refer to any XSpec variables that are defined in the same scenario, an ancestor scenario, or at the top level.