Writing Scenarios for Schematron - xspec/xspec GitHub Wiki

XSpec tests for Schematron are written in an XML document that adheres to the RelaxNG schema xspec.rnc.

The root element of a test file is x:description.

To test a Schematron schema, the x:description element must have @schematron attribute which specifies the path to the Schematron file to test. For example:

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
               schematron="filename.sch">

Parameters that are specified in the global x:param elements are passed into the Schematron compilation.

If you are testing a Schematron that has multiple phases, you can specify a particular Schematron phase to be tested by adding a x:param element with @name="phase" and provide the name of the phase. For example:

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
               schematron="filename.sch">
  <x:param name="phase">thephase</x:param>

Likewise, you may want to add other global x:param elements such as allow-foreign, only-child-elements and visit-text.

Parameters that are specified in the scenario-level x:param elements are passed into the compiled XSLT, not into the Schematron compilation.

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
               schematron="filename.sch"
               run-as="external">
  <x:scenario label="Scenario-level x:param">
    <!-- This $p (//x:scenario/x:param) takes effect only in the compiled XSLT,
      not while compiling Schematron to XSLT. -->
    <x:param name="p" select="'value'" />

Tests are defined using x:scenario elements. Each scenario should have a label that describes it.

<x:scenario label="description">

Scenarios can be nested, shared, focused, or pending.

When the test file is executed by XSpec, each scenario will be evaluated by using the Schematron to validate the context XML and then checking the validation result to find out if the result matches the expectations.

The x:context element is used to provide XML to be validated by the Schematron for a particular test scenario. The context XML can be provided in a separate file that is specified by an @href attribute:

<x:context href="filename.xml" />

Alternatively, the context XML can be given as an XML fragment within the x:context element:

<x:context>
  <example>any XML</example>
</x:context>

Expectations specify the results that are expected when the context XML is validated by the Schematron. Expectations are written using any combination of x:expect-* elements to specify expected results of running the Schematron on the context XML. In particular:

• x:expect-valid verifies that the Schematron is executed and passes validation. In the Schematron, an assert or report can have a @role attribute specifying that it is an error (value error or fatal), a warning (value warn or warning), an informational message (value info or information), or potentially other user-specified values. A @role attribute not equal to error or fatal is considered to be allowed for a passing validation. These role values are all considered as case-insensitive.
• x:expect-assert verifies that an assert is thrown (sch:assert is evaluated as failure).
• x:expect-not-assert verifies that an assert is not thrown (sch:assert is not evaluated as failure).
• x:expect-report verifies that a report is thrown.
• x:expect-not-report verifies that a report is not thrown.

With the exception of x:expect-valid, the x:expect-* elements all have attributes to identify an expected assert or report. These attributes can be used in any combination to specify the expected results of running the Schematron on the context XML. In particular:

• @id: identify a specific assert or report using its @id attribute or match the @id attribute of the ancestor rule or pattern. The nearest of these @id attributes is considered for matching.
• @role: match a specific @role attribute value of an assert or report or the parent rule. @role attribute values are often used to specify error, fatal, warn, warning, info, information. The parent rule's @role attribute is considered for matching only if assert or report has no @role attribute.
• @location: XPath of a location in the context XML that the assert or report is expected to find. Namespace prefixes that are defined in Schematron using ns elements can be used.

For example, say you expect an assert with @id="a0001" to be thrown (sch:assert[@id="a0001"] is evaluated as failure) at XPath location /html[1]/body[1]/div[5]. This expectation would be written as:

<x:expect-assert id="a0001"
                 location="/html[1]/body[1]/div[5]" />

Starting in XSpec v3.0.3, the x:expect-assert and x:expect-report elements can also have text content that identifies an expected assert or report by its message or diagnostic. Any content of these elements must be literal text with no markup, variables, or text value templates. XSpec normalizes space before comparing messages and diagnostics with the content of these elements.

Although not required, it is helpful if the assert and report elements in the Schematron being tested have an @id attribute. Test scenarios can be more specific by using the @id attribute to match expectations to assert and report elements in the Schematron. The file quickfix.sch included with the demos is a Schematron Quick Fix that can be used in oXygen to quickly add the @id attribute to assert and report elements.

It is good practice to test each Schematron assert and report at least twice: once using XML that is valid according to the rule and once using XML that should cause the assert or report to be thrown (XML that is invalid according to the rule):

• Use x:expect-not-assert and x:expect-not-report to verify that a Schematron passes validation on XML that is valid according to the assert or report.
• Use x:expect-assert and x:expect-report to verify that a Schematron identifies XML that causes the assert or report to be thrown (XML that is invalid according to the rule).

@count can be used to describe the number of times an assert or report is expected to be thrown. For example, if you expect one specific assert and no other assert to be thrown (only the specific sch:assert is evaluated as failure), you would write:

<x:scenario label="Only unknown-pi assert should be thrown">
  <x:expect-assert id="unknown-pi" />
  <x:expect-assert count="1" />
</x:scenario>

Scenarios can be imported from a separate file. This allows more flexibility for organizing your test cases. To import scenarios from another file use the x:import element and specify the XSpec file to import using the @href attribute:

<x:import href="filename.xspec" />

Anything that can be done in an XSpec test for XSLT can be done in an XSpec test for Schematron. It is possible to test XSLT function definitions (xsl:function) that are embedded in a Schematron. A test scenario can also check the SVRL output of Schematron for specific conditions.