MLUNITEXT unit testing framework description - SystemAnalysisDpt-CMC-MSU/ellipsoids GitHub Wiki

Introduction

MLUNITEXT is a custom-made unit testing framework developed by Peter Gagarinov. At the moment in r25 of trunk the framework is located in https://github.com/SystemAnalysisDpt-CMC-MSU/ellipsoids/tree/master/lib/mlunitext. It consists of the following packages.

  1. mlunit - contains the classes designed for a direct use by a developers. When writing your tests you should inherit from mlunitext.test_case class.
  2. mlunit_samples - contain a sample test case for demo purposes. This should be the first place to look for a developer without any experience with MLUNITEXT.
  3. mlunit_test - contains the tests for MLUNITEXT itself (yes, the testing framework also needs to be tested!!!). This is the second place where developer should look for the test case examples. To run all the tests type mlunitext.test.run_tests``(``) from Matlab command line.

Typical use case for MLUNITEXT

  1. Inherit from mlunitext.test_case and write you tests in the methods that start with test. Using test in front of your test methods is obligatory as test prefix tells MLUNITEXT that the method in question is a test method.

        classdef BasicTestCase < mlunitext.test_case
        %
        properties (Access=private)
            someProperty
        end
        methods
            function self = BasicTestCase(varargin)
                self = [email protected]_case(varargin{:});
            end
            %
            function set_up(self)
               self.someProperty=1;
            end
            %
            function tear_down(self)
               self.someProperty=0;
            end
            %
            function testAlwaysPass(self)
                mlunitext.assert_equals(self.someProperty,1);
            end
            %
            function testNull(self)
                mlunitext.assert_equals(0, sin(0));
            end
            %
            function testSinCos(self)
                mlunitext.assert_equals(cos(0), sin(pi/2));
            end
            function testFailed(self)
                mlunitext.assert_equals(1,0);
            end
            function testSin(self)
                mlunitext.assert_equals(sin(0), 0);
            end
        end
    end

  2. When you move than one test case created, let's say BasicTestCase and AdvancedTestCase within mymainpackage.mychildpackage.test.mlunit the next step would be writing run_tests function within mymainpackage.mychildpackage.test\ for these two test cases. The following code snippet shows how this can be done.

        function result=run_tests(varargin)
    runner = mlunitext.text_test_runner(1, 1);
    loader = mlunitext.test_loader;
    suiteBasic = loader.load_tests_from_test_case(...
    'mymainpackage.mychildpackage.test.mlunit.BasicTestCase',varargin{:});
    %
    suiteAdvanced = loader.load_tests_from_test_case(...
    'mymainpackage.mychildpackage.test.mlunit.AdvancedTestCase',varargin{:});
    %
    suite = mlunitext.test_suite(horzcat(...
        suiteBasic.tests,...
        suiteAdvanced.tests));
    %
    resultVec=runner.run(suite);

Now you can just type mymainpackage.mychildpackage.test.run_tests to run the tests from both use cases.

Writing negative tests

A test that causes an exception in the called function/class is also a test aka negative test. mlunitext.test_case class has a method runAndCheckError designed for negative testing. Let's consider an example where a call to class method doSomethingBad1 causes an exception with 'wrongInput' identifier and to method doSomethingBad1 causes exception 'wrongInput' or 'complexResult'. The test for this case can be written in the following way.

        function testNegative(self)
        obj=...
        inpParamList={1,2};
        self.runAndCheckError('obj.doSomethingBad1(inpParamList{:})','wrongInput');
        errList = {'wrongInput','complexResult'};
        self.runAndCheckError('obj.doSomethingBad2(inpParamList{:})',errList);
        end

The same test can be written in a slightly different manner:

        function testNegative(self)
        obj=...
        inpParamList={1,2};
        self.runAndCheckError(@check,'wrongInput');
        %
          function check()
             obj.doSomethingBad(inpParamList{:});
          end
        end

Structuring the tests into packages

Let's assume that you have a few additional test cases implemented within a different package mymainpackage.mysecondpackage.test.mlunit along with 'mymainpackage.mysecondpackage.test.run_tests' function. Then you can (and actually should) create a higher-level run_tests function in mymainpackage.test package that call the lower-level 'run_test' functions (see the following code snippet)

        function result=run_tests(varargin)
        resList{2}=mymainpackage.mysecondpackage.test.run_tests();
        resList{1}=mymainpackage.mychildpackage.test.run_tests();
        %
        resultVec=[resList{:}];

The command 'mymainpackage.test.run_tests will run all the tests. Please note that 'mymainpackage.test.run_tests returns results of ALL the tests in both packages.

Running specific tests from a test suite

When you need to run tests from a specific test case (let's say mymainpackage.mychildpackage.test.mlunit.BasicTestCase) directly you can use mlunitext.runtestcase function (see the following snippet).

        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase') % runs all tests from the test case
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase','testSinCos') % runs only testSinCos test
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase\testSinCos') % does the same
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase.testSinCos') % does the same
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase','testSin') % runs only testSinCos and testSin tests
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase\testSin') % does the same
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase','testSin$') % runs only testSin test
        %
        mlunitext.runtestcase(...
        'mymainpackage.mychildpackage.test.mlunit.BasicTestCase.testSin$') % does the same

Sometimes the same test case is used multiple times with different markers. You can use getCopyFiltered method of mlunitext.test_suite class to filter the tests and run only those that match the filtering pattern:

testSuiteList=cell(1,3);
testSuiteList=loader.load_tests_from_test_case('mypackage.MyTestCase',firstParam1Obj,secParam1Obj,'marker','first1_sec1');
testSuiteList=loader.load_tests_from_test_case('mypackage.MyTestCase',firstParam1Obj,secParam2Obj,'marker','first1_sec2');
testSuiteList=loader.load_tests_from_test_case('mypackage.MyTestCase',firstParam2Obj,secParam1Obj,'marker','first2_sec1');

testLists = cellfun(@(x)x.tests,testSuiteList,'UniformOutput',false);
testList=horzcat(testLists{:});
suiteObj = mlunitext.test_suite(testList);	
   %
%run all tests from mypackage.MyTestCase for the first parameter equal to firstParam1Obj
suiteFilteredObj=suiteObj.getCopyFiltered('first1','mypackage.MyTestCase','.*');
resultVec = runner.run(suiteFilteredObj);
    %
%run all tests from mypackage.MyTestCase for first parameter equal to firstParam1Obj and second parameter equal to secParam2Obj
suiteFilteredObj=suiteObj.getCopyFiltered('first1_sec2','mypackage.MyTestCase','.*');
resultVec = runner.run(suiteFilteredObj);
    %
%run 'testA' test from mypackage.MyTestCase for second parameter equal to secParam2Obj
suiteFilteredObj=suiteObj.getCopyFiltered('sec2','mypackage.MyTestCase','testA');
resultVec = runner.run(suiteFilteredObj);

Writing parameterized tests

In addition to set_up method mlunitext.test_case class defines set_up_param method that can be used for passing external arguments into the test case i.e. making the test parameterized. Here is an example of such parameterized test case.

	classdef PrameterizedTC < mlunitext.test_case
		properties (Access=private)
			secretProperty
		end
		methods
			function self = PrameterizedTC(varargin)
				self = [email protected]_case(varargin{:});
			end
			%
			function set_up_param(self, varargin)
				import modgen.common.throwerror;
				nArgs = numel(varargin);
				if nArgs == 1
					self.secretProperty = varargin{1};
				elseif nArgs > 1
					throwerror('wrongInput','Too many parameters');
				end
			end
			%
			function testSecretProperty(self)
				SECRET_VALUE=247;
				if ~isequal(self.secretProperty,SECRET_VALUE)
					modgen.common.throwerror('wrongInput',...
						['wrong value of secret propety,\n',...
						'expected %d but received %d'],...
						SECRET_VALUE,self.secretProperty);
				end
			end
		end
	end

In PrameterizedTC test case class the property secretProperty is expected to be passed from outside via set_up_param method. This can be done using the following approach

            secretVal=22;
            suite1Obj=mlunitext.test_suite.fromTestCaseNameList(...
                'mlunitext.test.PrameterizedTC',...
                {secretVal,'marker','secretVal_22'});
            %
            secretVal=247;
            suite2Obj=mlunitext.test_suite.fromTestCaseNameList(...
                'mlunitext.test.PrameterizedTC',...
                {secretVal,'marker','secretVal_247'});
            suiteObj=mlunitext.test_suite.fromSuites(suite1Obj,...
                suite2Obj);
            runnerObj=mlunitext.text_test_runner(1,1);
            resultObj=runnerObj.run(suiteObj);

Variable resultObj will contain a result of 2 test runs (same tests but with different markers):

	>> resultObj.getNTestsRun

	ans =

		 2

	>> resultObj.get_error_list

	ans = 

		'PrameterizedTC[secretVal_22]/testSecretProperty'    [1x4191 char]

	>> resultObj.getNErrors

	ans =

		 1

And, of course, you can always run testSecretProperty test using mlunitext.runtestcase function like this

       mlunitext.runtestcase('PrameterizedTC','testSecretProperty','testParams',{1})