Unit Testing - mwgeurts/viewray_mlc GitHub Wiki

The principal features of the ViewRay MLC Position Check application have been tested between versions for a set of test suites to determine if regressions have been introduced which may effect the results. These results are summarized below, grouped by test suite. Note that pre-releases have not been included in this unit testing.

Unit testing was performed using an automated test harness developed to execute each unit test stored in UnitTest() and, where relevant, compare the results to pre-validated reference data. Refer to the documentation UnitTest() for a description and conditions of each unit test and UnitTestHarness() for details on how the unit test is executed for each software version and test suite. Each Unit Test is referenced to one or more Requirements through the Traceability Matrix using the Test ID.

Unit tests developed for performance requirements are separated from functional or interface tests (even if they execute the same code) to allow the performance metric, typically time to execute, to be reported in addition to the conventional Pass/Fail test outcome. The purpose of this, along with the cyclomatic complexity and code analyzer summary unit tests (which uniquely do not trace to a requirement) are to not only test unit test passing but to also help identify poor coding practices. Performance tests that pass requirement but are significantly worse than prior versions of the application are further analyzed for refactoring opportunities to maintain code simplicity.

Contents

Test System Configuration

Computation times are documented based on the following system configuration and reflect elapsed real time (as opposed to CPU time). Note, other hardware configurations may also be run during compatibility testing.

Specification Configuration
Operating System Mac OS X 10.10.2
Processor Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz
Frequency 3500 MHz
Number of Cores 4
L2 Cache (per core) 256 KB
Memory 15.99 GB (8.20 GB available)
Graphics Card GeForce GTX 780M
Graphics Memory 4096 MB (1413 MB available)
CUDA Version 3.0
MATLAB Version R2014b

AP Test Suite Results

Input Data Value
Gamma Criteria 3%/1mm
Input Data Head1_G90.txt
Gantry Angle 90
ID Test Case Version 1.0 Version 1.1.0 Version 1.1.1 Version 1.1.2
1 Application Loads Successfully Pass Pass Pass Pass
2 Application Load Time 2.0 sec 1.6 sec 1.6 sec 1.6 sec
3 Code Analyzer Messages 0 0 0 0
4 Cumulative Cyclomatic Complexity 313 569 569 580
5 Reference Data Loads Successfully Pass Pass Pass Pass
6 AP Reference Data within 1%/0.1 mm N/A Pass Pass Pass
7 PANC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
8 PATC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
9 H1 Browse 1 Loads Data Successfully Pass Pass Pass Pass
10 Browse Callback Load Time 2.8 sec 12.8 sec 12.6 sec 5.9 sec
11 Field Edges within 0.1 mm Pass Pass Pass Pass
12 Field Widths within 0.1 mm Pass Pass Pass Pass
13 Gamma within 0.1 Pass Pass Pass Pass
14 Statistics within 0.1 mm Pass Pass Pass Pass
15 Remaining Browse Buttons Load Data Successfully Pass Pass Pass Pass
16 H1 Figure Display Functional Pass Pass Pass Pass
17 H2 Figure Display Functional Pass Pass Pass Pass
18 H3 Figure Display Functional Pass Pass Pass Pass
19 Print Report Functional N/A Pass Pass Pass
20 Print Report Time N/A 3.0 sec 2.9 sec 2.9 sec
21 H1 Clear Button Functional Pass Pass Pass Pass
22 H2 Clear Button Functional Pass Pass Pass Pass
23 H3 Clear Button Functional Pass Pass Pass Pass
24 Documentation Exists Pass Pass Pass Pass

PA (thru couch) Test Suite Results

Input Data Value
Gamma Criteria 3%/1mm
Input Data Head2_G180.txt
Gantry Angle 180
ID Test Case Version 1.0 Version 1.1.0 Version 1.1.1 Version 1.1.2
1 Application Loads Successfully Pass Pass Pass Pass
2 Application Load Time 2.0 sec 1.6 sec 1.6 sec 1.8 sec
3 Code Analyzer Messages 0 0 0 0
4 Cumulative Cyclomatic Complexity 313 569 569 580
5 Reference Data Loads Successfully Pass Pass Pass Pass
6 AP Reference Data within 1%/0.1 mm N/A Pass Pass Pass
7 PANC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
8 PATC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
9 H1 Browse 1 Loads Data Successfully Pass Pass Pass Pass
10 Browse Callback Load Time 2.8 sec 13.2 sec 13.9 sec 5.8 sec
11 Field Edges within 0.1 mm Fail Pass Pass Pass
12 Field Widths within 0.1 mm Fail Pass Pass Pass
13 Gamma within 0.1 Fail Pass Pass Pass
14 Statistics within 0.1 mm Fail Fail Pass Pass
15 Remaining Browse Buttons Load Data Successfully Pass Pass Pass Pass
16 H1 Figure Display Functional Pass Pass Pass Pass
17 H2 Figure Display Functional Pass Pass Pass Pass
18 H3 Figure Display Functional Pass Pass Pass Pass
19 Print Report Functional N/A Pass Pass Pass
20 Print Report Time N/A 2.4 sec 2.5 sec 2.5 sec
21 H1 Clear Button Functional Pass Pass Pass Pass
22 H2 Clear Button Functional Pass Pass Pass Pass
23 H3 Clear Button Functional Pass Pass Pass Pass
24 Documentation Exists Pass Pass Pass Pass

PA (no couch) Test Suite Results

Input Data Value
Gamma Criteria 3%/1mm
Input Data Head3_G270_through_back_of_ICP.txt
Gantry Angle 270
ID Test Case Version 1.0 Version 1.1.0 Version 1.1.1 Version 1.1.2
1 Application Loads Successfully Pass Pass Pass Pass
2 Application Load Time 2.0 sec 1.7 sec 1.6 sec 1.6 sec
3 Code Analyzer Messages 0 0 0 0
4 Cumulative Cyclomatic Complexity 313 569 569 580
5 Reference Data Loads Successfully Pass Pass Pass Pass
6 AP Reference Data within 1%/0.1 mm N/A Pass Pass Pass
7 PANC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
8 PATC Reference Data within 1%/0.1 mm N/A Pass Pass Pass
9 H1 Browse 1 Loads Data Successfully Pass Pass Pass Pass
10 Browse Callback Load Time 2.7 sec 13.1 sec 13.2 sec 5.7 sec
11 Field Edges within 0.1 mm Fail Pass Pass Pass
12 Field Widths within 0.1 mm Fail Pass Pass Pass
13 Gamma within 0.1 Fail Pass Pass Pass
14 Statistics within 0.1 mm Fail Fail Pass Pass
15 Remaining Browse Buttons Load Data Successfully Pass Pass Pass Pass
16 H1 Figure Display Functional Pass Pass Pass Pass
17 H2 Figure Display Functional Pass Pass Pass Pass
18 H3 Figure Display Functional Pass Pass Pass Pass
19 Print Report Functional N/A Pass Pass Pass
20 Print Report Time N/A 2.5 sec 2.5 sec 2.5 sec
21 H1 Clear Button Functional Pass Pass Pass Pass
22 H2 Clear Button Functional Pass Pass Pass Pass
23 H3 Clear Button Functional Pass Pass Pass Pass
24 Documentation Exists Pass Pass Pass Pass

Code Coverage

Function Version 1.0 Version 1.1.0 Version 1.1.1 Version 1.1.2
AnalyzeMLCProfiles 80.8% 72.4% 72.4% 73.6%
AnalyzeProfilerFields 57.2% 57.2% 56.2%
BrowseCallback 75.5% 75.5% 75.5%
CalcGamma 37.7% 37.2% 37.2% 41.0%
ClearCallback 100.0% 100.0% 100.0%
Event 60.0% 60.0% 60.0% 60.0%
LoadProfilerDICOMReference 81.2% 81.2% 74.6%
LoadVersionInfo 84.6% 84.6% 84.6% 84.6%
ParseSNCprm
ParseSNCtxt 66.3% 66.3% 66.3%
PrintReport 82.7% 82.7% 82.7%
UpdateDisplay 96.0% 96.2% 96.2% 96.2%
UpdateMLCStatistics 97.5% 97.5% 97.5% 97.5%