Table of Contents generated with DocToc

• Introduction
• Existing tests
  • Compiling the tests
  • Running the test programs
• Writing your own tests
  • Test drivers
  • Test suites and cases
    • Organizing the test cases
    • Writing the test cases

The cisst libraries come with a fair amount of unit testing, aka test cases. The test programs are fairly easy to compile and run. It is also strongly suggested to add new tests whenever you add features to the libraries. If you want to setup a test site, follow these instructions.

The test programs are in the tests directory for each library (see for example cisstCommon/tests). They are organized by library, i.e. cisstCommon, cisstVector, ... C++ tests are in the sub-directory tests and the python tests are in testsPython.

To compile the C++ tests, you will need CppUnit. Make sure you read the Download and Installation FAQ first! Once you made sure you had CppUnit installed, you will need to set the CISST_BUILD_TESTS flags ON when configuring the cisst with CMake.

For the Python tests, we use unittest which is included in most Python distributions.

• Using the cisst test driver.

  • A test program is generated per library (actually, two if some python tests are available). For cisstCommon, the tests drivers are cisstCommonTests(.exe) and cisstCommonPython.py.
  • Both C++ and python test drivers use similar command line arguments:
    • cisstCommonTests -r runs all tests available
    • cisstCommonTests -l lists all the available tests
    • cisstCommonTests -r -t cmnPathTest runs all the tests in the test suite cmnPathTest
    • cisstCommonTests -r -t cmnPathTest::TestAdd runs a single test case (i.e. cmnPathTest::TestAdd)
    • -o N will create N instantiation(s) of the test suite. This is a good test for classes with static methods or data members
    • -i N will run N iteration(s) of the test suite. This is a good test for setup/cleanup and some randomization

• Using CTest. CTest is a utility to start some registered test programs. For the cisst libraries, it is important to note that all tests are executed 25 times (5 instantiations times 5 iterations, it uses the cisst test driver with the options -i 5 -o 5 ).

  • When you configured the build with the tests activated, a special target call ''Experimental'' has been generated by CMake. When you build this target, the test suite is executed. Using Makefiles, type make Experimental. Using a graphical IDE (Eclipse, Visual Studio, Xcode), select the target and build it. This will run all the tests (which can be very long) and submit the results to our CDash web dashboard.
  • You can also start the tests using CTest. In this case, you will need to open a shell (or a dos/command windows) in the build tree. You can then run the tests using ctest.

Each test driver (test executable), one per library (e.g. cisstCommontTests) needs the following:

• The main function which parses the command line options and run the tests selected. This is provided as part of the library cisstTestMain (code located in tests/code).
• Tests cases, these are provided by the library programmer (i.e. you!) and should be located in the directory <library_name>/tests (e.g. cisstCommon/tests).
• Description of the build process, i.e. a CMakeLists.txt file in the directory <library_name>/tests. The simplest solution to create a new test driver is to copy an existing CMakeLists.txt and edit it. Once the driver is in place, it is possible to add new tests cases fairly quickly.

By convention, we use a test suite per class to be tested and then use as many test cases as needed. A test suite is a set of test cases that can be run together using the -t option of our test driver. For example, all tests for the class cmnPath are organized in a single test suite called cmnPathTest. The declaration is placed in the file cisstCommon/tests/cmnPathTest.h. This file uses a few CppUnit macros:

class cmnPathTest : public CppUnit::TestFixture
{
    CPPUNIT_TEST_SUITE(cmnPathTest);
    CPPUNIT_TEST(TestAdd);
    CPPUNIT_TEST(TestFind);
    CPPUNIT_TEST(TestRemove);
    CPPUNIT_TEST_SUITE_END();
   
 public:
    void setUp(void) {}
    void tearDown(void) {}

    void TestAdd(void);
    void TestFind(void);
    void TestRemove(void);
};

The macros CPPUNIT_TEST_SUITE and CPPUNIT_TEST_SUITE_END are used to define a test suite and start/stop listing the test cases. Each test case is registered using the macro CPPUNIT_TEST. This macro requires a method name (e.g. TestAdd). The methods corresponding to test cases have to been declared in the same class scope. Finally, if the same dataset has to be setup for each case, one can use the setUp and tearDown. Since these methods are defined as pure virtual in the base class CppUnit::TestFixture, the user will have to redefine them.

Once the header file describing the test suite is created, we need to work on the corresponding code file (e.g. cisstCommon/tests/cmnPathTest.cpp). This file will contain the methods associated to the test cases. Each test case can contain multiple assertions but it is important to remember that for each test case, the first failed assertion will end the test case execution. Therefore the test case should start with assertions less likely to fail. CppUnit supports a number of assertions. For example, cmnPathTest::TestFind uses the macro CPPUNIT_ASSERT:

void cmnPathTest::TestFind(void) {
    cmnPath path("/this/directory/is/not/valid;/this/one/either/but/it/helps");
    path.Add(CISST_BUILD_ROOT, cmnPath::TAIL);
    // test a file that should exist
    std::string fullName(CISST_BUILD_ROOT);
    fullName += "/libs/include/cisstConfig.h";
    CPPUNIT_ASSERT(fullName == path.Find("libs/include/cisstConfig.h", cmnPath::READ));
    // test with a file that shouldn't exist
    std::string thisFileCantExist = "/aBadPath/QwErTy.hohoho";
    CPPUNIT_ASSERT("" == path.Find(thisFileCantExist, cmnPath::READ));
} 

Other CppUnit macros can be used to test assertions:

• CPPUNIT_ASSERT_EQUAL(expected, actual), not very different from CPPUNIT_ASSERT.
• CPPUNIT_ASSERT_DOUBLES_EQUAL(expected, actual, delta), very convenient for floating point. This should be used in conjunction with cmnTypeTraits to use a reasonable epsilon/delta (CPPUNIT_ASSERT_DOUBLES_EQUAL(expected, actual, cmnTypeTraits<double>::Tolerance()))
• CPPUNIT_ASSERT_NO_THROW(expression) to validate that no exception it thrown.
• CPPUNIT_ASSERT_THROW(expression, ExceptionType), very useful to test error handling.
• CPPUNIT_FAIL(message) can be used to force a test to fail (e.g. default case in a switch statement).