Test Specification Grammar - capn-freako/PyAMI GitHub Wiki

Tests are specified, using EmPy, a Python embedding/templating package. Test definition files must have a '*.em' extension. Tests should be designed to be generic, having applicability to many different models. They should NOT contain model specific parameter definitions, for instance. These should, instead, go into parameter sweep definition files. (See the Test Parameter Sweeping page of this WiKi.)

Test definition files are, essentially, XML containing some embedded Python. A typical test file might look like this:

@# Example input file for `run_tests.py'.
@# 
@# Original Author: David Banas
@# Original Date:   July 20, 2012
@# 
@# Copyright (c) 2012 David Banas; All rights reserved World wide.

<test>
    <name>@(name)</name>
    <result>Pass</result>
    <description>Model impulse response for: @(description)</description>
    <output>
@{
from pylab import *
#import amimodel as ami           # Use this one for developing/testing.
import pyibisami.amimodel as ami # Use this one for distribution.
cla()
for cfg in data:
    cfg_name = cfg[0]
    params = cfg[1]
    if(len(cfg) > 2):
        reference = ref_dir + '/' + name.split()[0] + '/' + cfg[2]
    else:
        reference = None
    initializer = ami.AMIModelInitializer(params[0])
    items = params[1].items()
    items.sort(reverse=True)
    for item in items:
        exec ('initializer.' + item[0] + ' = ' + repr(item[1]))
    model.initialize(initializer)
    print '        <block name="Model Initialization (' + cfg_name + ')" type="text">'
    print model.msg
    print model.ami_params_out
    print '        </block>'
    h = model.initOut
    T = model.sample_interval
    t = [i * T for i in range(len(h))]
    rgb_main, rgb_ref = plot_colors.next()
    color_main = "#%02X%02X%02X" % (rgb_main[0] * 0xFF, rgb_main[1] * 0xFF, rgb_main[2] * 0xFF)
    color_ref = "#%02X%02X%02X" % (rgb_ref[0] * 0xFF, rgb_ref[1] * 0xFF, rgb_ref[2] * 0xFF)
    plot(t, h, label=cfg_name, color=color_main)
    if(reference):
        r = ami.getImpulse(reference, T)
        plot(t, r, label=cfg_name+'_ref', color=color_ref)
title('Model Impulse Response')
xlabel('Time (sec.)')
legend()
filename = plot_names.next()
savefig(filename)
}
        <block name="Model Impulse Response" type="image">
            @(filename)
        </block>
    </output>
</test>

The embedded Python is identified, by the '@' prefix. And anything not so prefixed is passed unaltered. Note that the embedded Python can contribute to the structure, as well as the content, of the final XML.

Note the following features of the XML used in the PyAMI framework:

1. With the exception of the obvious header tags, the <block> tag is the only container that can provide content to the final output report. That is, the XSLT style sheet, which governs the final output report formatting, only looks in <block> tags for content (again, with the exception of the obvious header tags). This means that, if your embedded Python prints something outside of a <block> / </block> pair, it will not be seen in the final report. (This was, actually, a handy feature, during project debugging.)
2. There are, currently, two types of <block>s:

• text - This is the usual type and is used to capture all print output from the embedded Python. It is formatted as <code> in the final XHTML report.
• image - This type of block is used to display an image in the final report. There must be exactly one text string in between the <block> / </block> pair, and it must name an image file in the local directory.

The following global variables are initialized by the calling system and provided to the embedded Python:

• name - The name of the test run, formatted as: <test name> (<param. config. id>).
• model - The uninitialized AMIModel instance. The DLL has been loaded and the 3 AMI functions bound.
• data - The list of parameter configuration blocks for this test. Each configuration block consists of a tuple containing, at least, a string label and a pair of dictionaries, which contain the Model Specific and Reserved AMI parameters, respectively. (See the Test Parameter Sweeping page for more detail.) Optionally, a third item may be added to the tuple, which is the name of a reference waveform file to be included in the plot, color-coded so as to match this particular test output waveform. (See plot_colors, below.)
• plot_names - A generator of unique plot file names, guaranteed not to cause an overwrite of any other saved plot file needed by the final report. In order to generate the next unique plot file name, call 'plot_names.next()'.
• plot_colors - A generator of unique plot color pairs, which evenly span the color hue space and provide one fully bright, fully saturated color, for model output, and another of the same hue, but half saturated, half bright, for the reference waveform. In order to generate the next unique plot color pair, call 'plot_colors.next()'.
• description - A description of this test.