DSP_Blockset_Testing - david-macmahon/wiki_convert_test GitHub Wiki

The library testing framework is meant to allow consistency checks between library versions and to ease future porting efforts. This verification relies on regression testing, guaranteeing only that the new block gives the same output as what was observed in the old block. This means that it does not test correctness but sameness relative to some standard.

Creating a new test case

  • Each test case should be used for testing a single block under a specific configuration.
  • Please ensure that the simulation time is as short as is necessary.
  • A test case should be created to ensure that all paths through a block's initialisation script are exercised.
  • A new test case consisting of a Simulink model should be created in the tests directory that is in the casper_library directory. The name of the test case model should begin with the name of the block it is intended to test, for example for the bit_reverse block we have created a test model called bit_reverse_test.mdl.
  • The library block to be tested in the Simulink model should have the default name e.g the bit_reverse block is called bit_reverse in the Simulink test model.
  • A reference file which will be used to compare against data generated by the block to be tested needs to be created using a To File Simulink block with the parameters set as follows (note that <file_name> is the name of the test file e.g bit_reverse_test.mdl has a corresponding bit_reverse_test_reference.mat file):
    • Filename: <file_name>_reference.mat
    • Variable name: reference
    • Decimation: 1
    • Sample time: -1
  • The rest of the test case can be created in any way, as long as it can be simulated.
  • Once the output has been checked and is known to be good, the test case must be modified to save the output to a different file which will be compared against the reference file. Change the To File parameters to the following:
    • Filename: <file_name>_output.mat
    • Variable name: output
    • Decimation: 1
    • Sample time: -1

Adding the test case to the framework

  • Once the block has been added to the library, the test case can be added to the tester_input.txt file.
  • Each line should be formatted as follows:

block_name block_location test_file_name parameter1_name parameter1_value parameter2_name parameter2_value ... After the block name (e.g bit_reverse), location of the block in the library (e.g casper_library/Misc/bit_reverse) and test file name (e.g bit_reverse_test), any number of parameters can follow. If the file doesn't exist or the block cannot be found within the file it will skip that test and report an error. Otherwise the test will proceed, applying the parameters to a fresh copy of the block and comparing the data in <file_name>_output.mat with <file_name>_reference.mat. Any differences in the data will be reported as a test failure.

  • Multiple tests for each block can be created by creating a new line for each test.
  • The first line in the file gives the library to be tested, blocks from different libraries should be tested using different tester files.
  • Lines begining with % followed by a space will be ignored so feel free to insert comments.
  • Lines beginning with % # followed by a space denote different sections which may be tested in isolation.

Please see the existing tester_input.txt which illustrates these formatting rules.

Running the tester

  • In Matlab, change directory to the directory containing the test cases e.g casper_library/Tests
  • Execute the command test_library <test_input_file> <test_section>. <test_input_file> is the file containing the test parameters (e.g tester_input.txt) and <test_section> is the section in the file containing the test parameters to execute (e.g PFBs). Use all as the test section to test all sections.
  • Please ensure that the test runs for the minimum time possible to test the relevant block.
  • A summary at the end of the test gives the number of test cases failed during the test (output results differed from results provided as standard) as well as the number of tests ignored for various reasons (block not found in the library specified etc).

Adding the test case to revision control

Before checking in files to version control, the test case should be run in the framework to ensure that the new or modified block passes.

When checking in the test case to version control, the following files must be updated or added:

  • The library file including the new block
  • The test case
  • The reference data file <file_name>_reference.mat
  • The input file for the tester, tester_input.txt

Do NOT check in the output file <file_name>_output.mat, it should be recreated every time the tester is run.

⚠️ **GitHub.com Fallback** ⚠️