Tests are structured in story files, each file having one or more scenarios. Story files should each have a unique file name, ideally describing the context they're testing.

Test steps within a scenario follow the BDD naming approach using simple phrases.

Steps allow setting up of pipeline variables establishing mocks, invoking services, validating mock invocation and verifying pipeline contents. Tests are executed with Junit which runs all of the stories, with a green tick shown for each successful story step and overall green bar if all test scenarios pass

Each sequential step is a phrase based on:

• Given - This is the set up of the test where there can be one or more steps
• When - A single step to invoke the service you want to test
• Then - Multiple steps to verify that the Given and Invoke have the expected returns.

Taking a standard service, string concatenation, this service can be tested:

Scenario: Concatenate two strings
Given pipeline values inString1 = "hello "; inString2 = "world"
When invoke pub.string:concat without idata
Then pipeline has value == "hello world"

The Given is setting up two pipeline values which correspond to the inputs required by the service. The When is invoking a service without passing any additional values to the pipeline. Values are passed into services in the form of IData files. The Then performs a test on the pipeline looking for value to be the concatenated result. Running this test, it should pass with a green bar and green ticks against all the steps.

If the test fails then the unit test will fail, red bar and the step in error highlighted with a cross. It also shows what the problem could be in the Failure Trace. In the example below, the Then has failed when testing the string match:

The word And can be used as a synonym in place of Then to make the scenario more readable:

Scenario:
Given pipeline values inString1 = "hello "; inString2 = "world"
When invoke pub.string:concat without idata
Then pipeline has value == "hello world"
And pipeline has inString1 == "hello "

Within a Then, expected outcome is described as an expression to test the pipeline that something has happened. The expression is based on the JEXL syntax with full details of how to use it on the Pipeline Expressions - Using Jexl page

The scenarios rely on the setting up of data into the pipeline then testing that the pipeline has values. To do this there are two options: Direct set up using a Given pipeline values phrase or supplying data from a file.

A phrase exists that allows one or more simple values such as strings numbers, etc to be set in to the pipeline before the service is called. Using the phrase Given pipeline values followed by one or more name = value assignments sets those into the pipeline. Where there is more than one required, separate each on the same line with a semicolon followed by a space, e.g Given pipeline values inString1 = "hello"; inString2 = " world"

This method of assignment is not suitable for setting long strings of complex XML, lots of values, etc and where that is required, use an IData file instead.

The internal pipeline format of webMethods is known as IData and is a key-value map hierarchy of the pipeline and its documents. You can see this format if using the pub.flow:savePipeline to capture pipeline contents during a service execution. The format is well defined and can be easily edited with an XML tool. Using savePipeline will give examples and if you want to capture data without modifying any flow, the RESTful pipeline capture functionality in wmaop can do that.

Once you have IData files they can be used to set the pipeline contents when the service under test is invoked with the phrase When invoke xxx with yyy This phrase will load the IData from the yyy specified file and set it in the pipeline before the service xxx is called. An example might be When invoke org.wmaop.test.services:rootSvc with data/lorem.xml This is loading the file lorem.xml from the data directory within the project (See Test project set up and options for more information on how the files are located and loaded) then the rootSvc is invoked.

When testing multiple conditions with many varied inputs, its sometimes required to alter some input parameters to a service. Creating lots of IData files where one value has changed is tedious and verbose. It is possible to create one IData file and have values overridden by using the Given pipeline values step in conjunction with the When invoke .. with .. step. An example of this is where we have a simple IData content:

<IDataXMLCoder version="1.0">
    <record javaclass="com.wm.data.ISMemDataImpl">
        <value name="lorem">ipsum</value>
        <value name="dolor">sit</value>
    </record>
</IDataXMLCoder>

And we invoke a service but first override lorem to have a different value. The test is looking for lorem being amet and not the ipsum value from the IData file

Given pipeline values lorem = "amet"
When invoke org.wmaop.test.services:rootSvc with data/lorem.xml
Then pipeline has lorem == "amet"

A story always has a When invoke line which indicates which service to call. There is a choice of whether to call the service and supply some IData to set up the pipeline or just to call the service with an empty pipeline. Using the 'with ...' suffix which indicates a file containg data in the IData format (The same XML format as that when you capture a pipeline). File references are relative from the src/test/resources directory so, for example, specifying data/pubresponse.xml as the name would pull the file from src/test/resources/data/pubresponse.xml

Alternatively the service can be invoked with no inputs. This is useful if simple pipeline data is already set with the Given pipeline values or no pipeline content is required.

Special attention must be made to the service and whether caching is enabled. Any service specified in the When invoke line that has caching enabled will still return cached values and therefore give unexpected values, particularly if you're trying to mock services. You may have to switch off the caching (Not recommended) or alternatively refactor your service to separate the caching from the service behaviour (recommended). Note that cached services elsewhere in the call stack (Ie called from within the 'invoke' service, etc) will always respond correctly to mocking and should need changing.]

Injecting data into your service and testing the output form the basis of the tests but an important part is eliminating service dependencies so that the test and the package it is deployed in becomes independent. Mocking out dependant services allows you to control what's returned so that the service under test is exercised without having to set up other systems or complex data flows.

Using Given mock, a service can be replaced with a mock such that it returns some pipeline values instead of calling the real service. There are two alternatives to this approach, always returning data or conditionally returning it.

When a fixed response is required where the same data is always supplied, the syntax looks like: Given mock org.wmaop.test:aMockedService always returning data/mydata.xml where the service and the IData files are specified.

In some cases, the data needs to be varied, depending on the pipeline or presented a particular condition applies. The syntax in this case is Given mock org.wmaop.test:aMockedService returning data/option-a.xml when selection == "a" This mock will only fire when selection is set to 'a' otherwise the real service is invoked. This allows for a service to be overridden only when needed

Taking the conditional approach one step further by overloading the mock definition, the data can be varied depending on the pipeline content and by using an always returning mock as the last definition, a default can be provided:

Given mock org.wmaop.test.services:mockedOption returning data/option-a.xml when selection == "a"
Given mock org.wmaop.test.services:mockedOption returning data/option-b.xml when selection == "b"
Given mock org.wmaop.test.services:mockedOption always returning data/option-c.xml
When invoke org.wmaop.test.services:concatOptions without idata
Then pipeline has result == "abc"

Sometimes its necessary to return more than one value from a mock, for instance in a loop. You could define multiple Given mock statement checking for some counter but that's not easy, particularly when nothing indicates the position. Fortunately its possible to specify more than one file as a comma separated list:

Given mock org.wmaop.test.services:mockedOption always returning data/option-c.xml,option-a.xml,option-b.xml

In this example when the mock is called it first returns the option-c.xml content from the data directory. On the next call it returns the option-a.xml file, from the implicit data directory used by the first, then the option-b.xml file. If called again it loops back around to serve the option-c.xml file, etc.

If the directory is specified anywhere that becomes the default for that and subsequent files so:

Given mock org.wmaop.test.services:foo always returning data/a.xml,bar/b.xml,c.xml

Would server a.xml from data then both b.xml and c.xml from the bar directory.

Mocks are a form of assertion (See below for more details) and this allows for checking on the number of times the mock has been called. In the Then section of the scenario the syntax looks like: Then mock org.wmaop.test.services:aMock was invoked 2 times Because data is being supplied and the outcome should be known, the invoke check is for the exact number of times its been actioned.

Key to a number of statements is the use of expressions to specify the condition to be met before the mock, assertion, etc is actioned. If you haven't already, take a look at defining pipeline expressions using JEXL for more information about the syntax and options available.

In some cases, matching complex data structures with multiple Then pipeline may be overly verbose and some form of direct xml matching is required. While the idea of capturing a document and matching against the pipeline seems like the easiest way to match pipeline contents, it's not the most declarative and can lead to lazy matching - Ie, just dump and compare without thinking about or declaring what's important within the document. To this end, document matching should only be used where there's a lot of data to compare and not as the default means of testing the pipeline.

Documents can be matched with either XML or IData outputs of the document. The following two examples are the same document but in XML format:

<?xml version="1.0" encoding="UTF-8"?>
<locality>
    <pcounty id="30" name="North Yorkshire">
        <parea id="HG">
            <ptown name="Harrogate"/>
            <ptown name="Ripon"/>
            <ptown name="Knaresborough"/>
        </parea>
    </pcounty>
    <pcounty id="46" name="West Yorkshire">
        <parea>
            <ptown name="Bradford"/>
            <ptown name="Shipley"/>
            <ptown name="Skipton"/>
        </parea>
    </pcounty>
</locality>

And IData format as the pipleline document called 'document':

<?xml version="1.0" encoding="UTF-8"?>
<IDataXMLCoder version="1.0">
    <record javaclass="com.wm.data.ISMemDataImpl">
        <record name="document" javaclass="com.wm.data.BasicData">
            <value name="@version">1.0</value>
            <value name="@encoding">UTF-8</value>
            <record name="locality" javaclass="com.wm.data.BasicData">
                <array name="pcounty" type="record" depth="1">
                    <record javaclass="com.wm.data.BasicData">
                        <value name="@id">30</value>
                        <value name="@name">North Yorkshire</value>
                        <record name="parea" javaclass="com.wm.data.BasicData">
                            <value name="@id">HG</value>
                            <array name="ptown" type="record" depth="1">
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Harrogate</value>
                                </record>
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Ripon</value>
                                </record>
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Knaresborough</value>
                                </record>
                            </array>
                        </record>
                    </record>
                    <record javaclass="com.wm.data.BasicData">
                        <value name="@id">46</value>
                        <value name="@name">West Yorkshire</value>
                        <record name="parea" javaclass="com.wm.data.BasicData">
                            <array name="ptown" type="record" depth="1">
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Bradford</value>
                                </record>
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Shipley</value>
                                </record>
                                <record javaclass="com.wm.data.BasicData">
                                    <value name="@name">Skipton</value>
                                </record>
                            </array>
                        </record>
                    </record>
                </array>
            </record>
        </record>
    </record>
</IDataXMLCoder>

To make a document match, the following syntax is looking for myDocument in the pipeline and is being matched against the snippet.xml: Then pipeline document myDocument matches data/snippet.xml

The data being offered for match can be either an exact match or partial. For an exact match, the example above where the document and the data are one for one in their elements and attributes.

A partial match takes the data being offered and as long as there's a corresponding element and attribute where specified it assumes a match. In this case you dont have to specify every element, just the ones of interest that are to be validated. So, for the IData shown above we could partially match using:

<locality>
    <pcounty id="30">
        <parea id="HG">
            <ptown name="Ripon"/>
        </parea>
    </pcounty>
    <pcounty id="46">
        <parea>
            <ptown name="Skipton"/>
        </parea>
    </pcounty>
</locality>

Note that key attributes are specified to identify the elements, such as id and that not all child elements are specified, ie only one ptown element per parea. Where more than one child element exists, say two ptown elements, then as long as both exist, in any order, the match is successful.

When using an IData file as the data to match, the same principal applies in that the data can be trimmed to match just key elements (Represented as record elements). It will also perform partial match where the IData is represented as an array of records. In this case, the match is successful if the records match within the array, regardless of order.

Firing mocks and testing pipeline values might be enough to satisfy you that the test has worked, but sometimes its useful to know that a particular service has been invoked and how many times (Eg, if the service has iterated some data, did it action the correct number?). Its also useful to know if a service has produced the expected output but it might not be possible to check the eventual pipeline contents due to values being dropped, etc.

Using Assertions allows you to 'peek' at the pipeline and check that something has happend.

Given MyAssertionName assertion before service org.wmaop.test.services:svcB always
When invoke org.wmaop.test.services:rootSvc with data/lorem.xml
Then assertion MyAssertionName was invoked 1 times

In this case, before svcB is actioned, the assertion called MyAssertion is triggered to prove that svcB was called. The Then checks for the assertion name and the number of invocations.

Just like mocks, assertions can have conditions applied to them so that the assertion only registers as invoked if the condition is met.

In the above examples, the invocation point has been before a service. By specifying after instead its possible to verify that a service has returned data or check at the end of a flow. If you specify invoke as the intercept point this will effectively replace the service with the assertion, effectively creating a blank, featureless mock.

Always ensure there is a Then assertion... statement for every Given ... assertion to verify the assertion - Simply creating an assertion without checking whether its fired wont give you a meaningful result!

Note that you only need assertions when checking flow through a service, usually before or after a particular service of interest is invoked. If you want to check that your mocks have fired, use the Then mock syntax as shown above in Did my mock fire?

The assertion (And mock) invoke count lookup uses the serviceName as a prefix rather than an exact match. This allows a combined count for all mocks or assertions within a package or a combined count where they begin with the name. For example, a serviceName of "pub.foo:ba" will give a count for "pub.foo:bar" and "pub.foo:bat". Likewise just using "pub.foo" gives a combined count for all mocks or assertion in that package. Note that the combined count is only for the mocks and assertions declared in the Given for the scenario, not the whole story (ie just that particular test)

Testing of behaviour should include negative paths where exceptional circumstances need handling and exeception handling requires execution. Instead of a service executing, its possible to raise an exception instead thereby simulating a failure. An example of the syntax is Given exception java.lang.RuntimeException thrown calling service foo:bar always The exception class name must be any java class that is Throwable and with a String constructor (Ie one where the exception message is passed in; This is the norm for most exceptions).

When testing within a loop you may want an exception to be thrown when a pipeline variable has a particular value (Eg, a piece of data from a service or a point within the loop counter). Like the mocking of a service, an exception can have a condition applied: Given exception java.lang.RuntimeException thrown calling service pub.foo:bar when loopCounter == 5

• Defining exceptions to be thrown in place of a service or before/after to interrupt flow

When an exception is raised in a test, be it from using a Given exception step or from having a service throw an exception, the exception must have a check applied to catch it and verify its existence. Without a check, the exception is treated as unexpected and the test will fail. To check for an exception use the syntax of:

Then exception com.wm.app.b2b.server.ServiceException was thrown

Its also possible to check the message and match if it contains a value, etc. After a Then exception... statement the pipeline has the message inserted as exceptionMessage which allows for a regular pipeline check such as:

And pipeline has exceptionMessage.contains("Service interface name required")

A variety test options can be seen in the stories used to test the framework. Browse through https://github.com/wmaop/OrgWmAOPTest/tree/master/src/test/resources/stories for examples of how to use the framework. Its also possible to deploy the OrgWmAOPTest package and run all of the tests if you want to experiment with what's there.

Each line in the story is made up from a step statement based on set phrases. The phrases are shown below in the table. Each step must appear on its own line so ignore the word wrapping in the table due to formatting limitations within the wiki space.

• Parameter place holders are shown as $...$ and should be replaced with actual values
  • $jexVariableExpression$ one or more expressions to set a pipeline value in the form of name = value such as myVar = "hello" Where there is the need to set more than one variable, separated with a semicolon, e.g: a = 1; b = "two"
  • $serviceName$ The fully qualified webMethods namespace of a service, eg: org.wmaop.pub.connectivity:doSomething
  • $idataFile$ A file containing data in the IData format (The same XML format as that when you capture a pipeline) File references to IData are relative from the src/test/resources directory so, for example, specifying data/pubresponse.xml as the name would pull the file from src/test/resources/data/pubresponse.xml
  • $jexlPipelinExpression$ An expression in JEXL format that tests the contents of the pipeline and returns a boolean value of true to indicate the expression has passed its test. See the page detailing JEXL Expressions for more information about the format.
  • $assertionId$ A name used to identify an assertion. This name is used to define and refer to the assertion and will be displayed at test failure. Should be something meaningful to the test, unique, camelcase for easy reading and be alphanumeric with no spaces, such as BeforeSendInvoice or hasExpiredPolicy
  • $interceptPoint$ The point at which the mock or assertion will be triggered. Values can be either: before to be triggered before a service; after to trigger after a service; or invoke to be triggered in-place of a service.
  • $exception$ The fully qualified Java class name of an exception that needs to be raised or tested for, such as java.lang.RuntimeExcpetion