Unit Testing - mwgeurts/viewray_mlc GitHub Wiki
The principal features of the ViewRay MLC Position Check application have been tested between versions for a set of test suites to determine if regressions have been introduced which may effect the results. These results are summarized below, grouped by test suite. Note that pre-releases have not been included in this unit testing.
Unit testing was performed using an automated test harness developed to execute each unit test stored in UnitTest()
and, where relevant, compare the results to pre-validated reference data. Refer to the documentation UnitTest()
for a description and conditions of each unit test and UnitTestHarness()
for details on how the unit test is executed for each software version and test suite. Each Unit Test is referenced to one or more Requirements through the Traceability Matrix using the Test ID.
Unit tests developed for performance requirements are separated from functional or interface tests (even if they execute the same code) to allow the performance metric, typically time to execute, to be reported in addition to the conventional Pass/Fail test outcome. The purpose of this, along with the cyclomatic complexity and code analyzer summary unit tests (which uniquely do not trace to a requirement) are to not only test unit test passing but to also help identify poor coding practices. Performance tests that pass requirement but are significantly worse than prior versions of the application are further analyzed for refactoring opportunities to maintain code simplicity.
Contents
Test System Configuration
Computation times are documented based on the following system configuration and reflect elapsed real time (as opposed to CPU time). Note, other hardware configurations may also be run during compatibility testing.
Specification |
Configuration |
Operating System |
Mac OS X 10.10.2 |
Processor |
Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz |
Frequency |
3500 MHz |
Number of Cores |
4 |
L2 Cache (per core) |
256 KB |
Memory |
15.99 GB (8.20 GB available) |
Graphics Card |
GeForce GTX 780M |
Graphics Memory |
4096 MB (1413 MB available) |
CUDA Version |
3.0 |
MATLAB Version |
R2014b |
AP Test Suite Results
Input Data |
Value |
Gamma Criteria |
3%/1mm |
Input Data |
Head1_G90.txt |
Gantry Angle |
90 |
ID |
Test Case |
Version 1.0 |
Version 1.1.0 |
Version 1.1.1 |
Version 1.1.2 |
1 |
Application Loads Successfully |
Pass |
Pass |
Pass |
Pass |
2 |
Application Load Time |
2.0 sec |
1.6 sec |
1.6 sec |
1.6 sec |
3 |
Code Analyzer Messages |
0 |
0 |
0 |
0 |
4 |
Cumulative Cyclomatic Complexity |
313 |
569 |
569 |
580 |
5 |
Reference Data Loads Successfully |
Pass |
Pass |
Pass |
Pass |
6 |
AP Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
7 |
PANC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
8 |
PATC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
9 |
H1 Browse 1 Loads Data Successfully |
Pass |
Pass |
Pass |
Pass |
10 |
Browse Callback Load Time |
2.8 sec |
12.8 sec |
12.6 sec |
5.9 sec |
11 |
Field Edges within 0.1 mm |
Pass |
Pass |
Pass |
Pass |
12 |
Field Widths within 0.1 mm |
Pass |
Pass |
Pass |
Pass |
13 |
Gamma within 0.1 |
Pass |
Pass |
Pass |
Pass |
14 |
Statistics within 0.1 mm |
Pass |
Pass |
Pass |
Pass |
15 |
Remaining Browse Buttons Load Data Successfully |
Pass |
Pass |
Pass |
Pass |
16 |
H1 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
17 |
H2 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
18 |
H3 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
19 |
Print Report Functional |
N/A |
Pass |
Pass |
Pass |
20 |
Print Report Time |
N/A |
3.0 sec |
2.9 sec |
2.9 sec |
21 |
H1 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
22 |
H2 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
23 |
H3 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
24 |
Documentation Exists |
Pass |
Pass |
Pass |
Pass |
PA (thru couch) Test Suite Results
Input Data |
Value |
Gamma Criteria |
3%/1mm |
Input Data |
Head2_G180.txt |
Gantry Angle |
180 |
ID |
Test Case |
Version 1.0 |
Version 1.1.0 |
Version 1.1.1 |
Version 1.1.2 |
1 |
Application Loads Successfully |
Pass |
Pass |
Pass |
Pass |
2 |
Application Load Time |
2.0 sec |
1.6 sec |
1.6 sec |
1.8 sec |
3 |
Code Analyzer Messages |
0 |
0 |
0 |
0 |
4 |
Cumulative Cyclomatic Complexity |
313 |
569 |
569 |
580 |
5 |
Reference Data Loads Successfully |
Pass |
Pass |
Pass |
Pass |
6 |
AP Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
7 |
PANC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
8 |
PATC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
9 |
H1 Browse 1 Loads Data Successfully |
Pass |
Pass |
Pass |
Pass |
10 |
Browse Callback Load Time |
2.8 sec |
13.2 sec |
13.9 sec |
5.8 sec |
11 |
Field Edges within 0.1 mm |
Fail |
Pass |
Pass |
Pass |
12 |
Field Widths within 0.1 mm |
Fail |
Pass |
Pass |
Pass |
13 |
Gamma within 0.1 |
Fail |
Pass |
Pass |
Pass |
14 |
Statistics within 0.1 mm |
Fail |
Fail |
Pass |
Pass |
15 |
Remaining Browse Buttons Load Data Successfully |
Pass |
Pass |
Pass |
Pass |
16 |
H1 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
17 |
H2 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
18 |
H3 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
19 |
Print Report Functional |
N/A |
Pass |
Pass |
Pass |
20 |
Print Report Time |
N/A |
2.4 sec |
2.5 sec |
2.5 sec |
21 |
H1 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
22 |
H2 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
23 |
H3 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
24 |
Documentation Exists |
Pass |
Pass |
Pass |
Pass |
PA (no couch) Test Suite Results
Input Data |
Value |
Gamma Criteria |
3%/1mm |
Input Data |
Head3_G270_through_back_of_ICP.txt |
Gantry Angle |
270 |
ID |
Test Case |
Version 1.0 |
Version 1.1.0 |
Version 1.1.1 |
Version 1.1.2 |
1 |
Application Loads Successfully |
Pass |
Pass |
Pass |
Pass |
2 |
Application Load Time |
2.0 sec |
1.7 sec |
1.6 sec |
1.6 sec |
3 |
Code Analyzer Messages |
0 |
0 |
0 |
0 |
4 |
Cumulative Cyclomatic Complexity |
313 |
569 |
569 |
580 |
5 |
Reference Data Loads Successfully |
Pass |
Pass |
Pass |
Pass |
6 |
AP Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
7 |
PANC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
8 |
PATC Reference Data within 1%/0.1 mm |
N/A |
Pass |
Pass |
Pass |
9 |
H1 Browse 1 Loads Data Successfully |
Pass |
Pass |
Pass |
Pass |
10 |
Browse Callback Load Time |
2.7 sec |
13.1 sec |
13.2 sec |
5.7 sec |
11 |
Field Edges within 0.1 mm |
Fail |
Pass |
Pass |
Pass |
12 |
Field Widths within 0.1 mm |
Fail |
Pass |
Pass |
Pass |
13 |
Gamma within 0.1 |
Fail |
Pass |
Pass |
Pass |
14 |
Statistics within 0.1 mm |
Fail |
Fail |
Pass |
Pass |
15 |
Remaining Browse Buttons Load Data Successfully |
Pass |
Pass |
Pass |
Pass |
16 |
H1 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
17 |
H2 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
18 |
H3 Figure Display Functional |
Pass |
Pass |
Pass |
Pass |
19 |
Print Report Functional |
N/A |
Pass |
Pass |
Pass |
20 |
Print Report Time |
N/A |
2.5 sec |
2.5 sec |
2.5 sec |
21 |
H1 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
22 |
H2 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
23 |
H3 Clear Button Functional |
Pass |
Pass |
Pass |
Pass |
24 |
Documentation Exists |
Pass |
Pass |
Pass |
Pass |
Code Coverage
Function |
Version 1.0 |
Version 1.1.0 |
Version 1.1.1 |
Version 1.1.2 |
AnalyzeMLCProfiles |
80.8% |
72.4% |
72.4% |
73.6% |
AnalyzeProfilerFields |
|
57.2% |
57.2% |
56.2% |
BrowseCallback |
|
75.5% |
75.5% |
75.5% |
CalcGamma |
37.7% |
37.2% |
37.2% |
41.0% |
ClearCallback |
|
100.0% |
100.0% |
100.0% |
Event |
60.0% |
60.0% |
60.0% |
60.0% |
LoadProfilerDICOMReference |
|
81.2% |
81.2% |
74.6% |
LoadVersionInfo |
84.6% |
84.6% |
84.6% |
84.6% |
ParseSNCprm |
|
|
|
|
ParseSNCtxt |
|
66.3% |
66.3% |
66.3% |
PrintReport |
|
82.7% |
82.7% |
82.7% |
UpdateDisplay |
96.0% |
96.2% |
96.2% |
96.2% |
UpdateMLCStatistics |
97.5% |
97.5% |
97.5% |
97.5% |