Unit Testing - mwgeurts/viewray_fielduniformity GitHub Wiki
The principal features of the ViewRay Field Uniformity-Timing Check application have been tested between versions for a set of test suites to determine if regressions have been introduced which may effect the results. These results are summarized below, grouped by test suite. Note that pre-releases have not been included in this unit testing.
Unit testing was performed using an automated test harness developed to execute each unit test stored in UnitTest() and, where relevant, compare the results to pre-validated reference data. Refer to the documentation UnitTest() for a description and conditions of each unit test and UnitTestHarness() for details on how the unit test is executed for each software version and test suite. Each Unit Test is referenced to one or more Requirements through the Traceability Matrix using the Test ID.
Unit tests developed for performance requirements are separated from functional or interface tests (even if they execute the same code) to allow the performance metric, typically time to execute, to be reported in addition to the conventional Pass/Fail test outcome. The purpose of this, along with the cyclomatic complexity and code analyzer summary unit tests (which uniquely do not trace to a requirement) are to not only test unit test passing but to also help identify poor coding practices. Performance tests that pass requirement but are significantly worse than prior versions of the application are further analyzed for refactoring opportunities to maintain code simplicity.
Computation times are documented based on the following system configuration and reflect elapsed real time (as opposed to CPU time). Note, other hardware configurations may also be run during compatibility testing.
| Specification | Configuration |
|---|---|
| Operating System | Mac OS X 10.10.2 |
| Processor | Intel(R) Core(TM) i5-3427U CPU @ 1.80GHz |
| Frequency | 1800 MHz |
| Number of Cores | 2 |
| L2 Cache (per core) | 256 KB |
| Memory | 4.00 GB (0.09 GB available) |
| Graphics Card | No compatible GPU device found |
| Graphics Total Memory | |
| Graphics Memory Available | |
| CUDA Version | |
| MATLAB Version | R2015a |
| Input Data | Value |
|---|---|
| Reference Data | ./reference/AP_10P5X10P5_PlaneDose_Vertical_Isocenter.dcm ./reference/AP_27P3X27P3_PlaneDose_Vertical_Isocenter.dcm |
| Gamma Criteria | 3%/1mm |
| Measured Data | Head1_G90_27p3.prm |
| ID | Test Case | Version 1.0 | Version 1.1.0 | Version 1.1.1 |
|---|---|---|---|---|
| 1 | Application Loads Successfully | Pass | Pass | Pass |
| 2 | Application Load Time1 | 0.7 sec | 0.9 sec | 1.0 sec |
| 3 | Code Analyzer Messages | 2 | 0 | 0 |
| 4 | Cumulative Cyclomatic Complexity | 261 | 395 | 403 |
| 5 | Reference Data Loads Successfully | Pass | Pass | Pass |
| 6 | Reference MLC X Data within 1%/0.1 mm | Pass | Pass | Pass |
| 7 | Reference MLC Y Data within 1%/0.1 mm2 | Fail | Pass | Pass |
| 8 | H1 Browse Loads Data Successfully | Pass | Pass | Pass |
| 9 | Browse Callback Load Time | 2.2 sec | 3.8 sec | 4.1 sec |
| 10 | MLC X Profile within 0.1% | Pass | Pass | Pass |
| 11 | MLC Y Profile within 0.1% | Pass | Pass | Pass |
| 12 | Positive Diagonal Profile within 0.1%3 | N/A | Pass | Pass |
| 13 | Negative Diagonal Profile within 0.1%3 | N/A | Pass | Pass |
| 14 | Timing Profile within 0.1% | Pass | Pass | Pass |
| 15 | MLC X Gamma within 0.11 | Pass | Pass | Pass |
| 16 | MLC Y Gamma within 0.11,2 | Fail | Pass | Pass |
| 17 | Positive Diagonal Gamma within 0.13 | N/A | Pass | Pass |
| 18 | Negative Diagonal Gamma within 0.13 | N/A | Pass | Pass |
| 19 | Statistics within 0.1 sec/mm/%1,4 | Pass | Fail | Pass |
| 20 | H1 Figure Display Functional | Pass | Pass | Pass |
| 21 | H2 Browse Loads Data Successfully | Pass | Pass | Pass |
| 22 | H3 Browse Loads Data Successfully | Pass | Pass | Pass |
| 23 | H2 Figure Display Functional | Pass | Pass | Pass |
| 24 | H3 Figure Display Functional | Pass | Pass | Pass |
| 25 | Print Report Functional | N/A | Pass | Pass |
| 26 | Print Report Time | N/A | 4.4 sec | 4.5 sec |
| 27 | H1 Clear Button Functional | Pass | Pass | Pass |
| 28 | H2 Clear Button Functional | Pass | Pass | Pass |
| 29 | H3 Clear Button Functional | Pass | Pass | Pass |
| 30 | Documentation Exists | Pass | Pass | Pass |
1Prior to Version 1.1 only the 27.3 cm x 27.3 cm reference profile existed
2#10 In Version 1.0 a bug was identified where MLC Y T&G effect was not accounted for in the reference data
3Prior to Version 1.1 diagonal profiles were not available
4#11 In Version 1.1.0 a bug was identified where flatness was computed incorrectly
| Input Data | Value |
|---|---|
| Reference Data | ./reference/AP_10P5X10P5_PlaneDose_Vertical_Isocenter.dcm ./reference/AP_27P3X27P3_PlaneDose_Vertical_Isocenter.dcm |
| Gamma Criteria | 3%/1mm |
| Measured Data | Head3_G90_10p5.prm |
| ID | Test Case | Version 1.0 | Version 1.1.0 | Version 1.1.1 |
|---|---|---|---|---|
| 1 | Application Loads Successfully | Pass | Pass | Pass |
| 2 | Application Load Time1 | 0.7 sec | 0.9 sec | 0.9 sec |
| 3 | Code Analyzer Messages | 2 | 0 | 0 |
| 4 | Cumulative Cyclomatic Complexity | 261 | 395 | 403 |
| 5 | Reference Data Loads Successfully | Pass | Pass | Pass |
| 6 | Reference MLC X Data within 1%/0.1 mm | Pass | Pass | Pass |
| 7 | Reference MLC Y Data within 1%/0.1 mm2 | Fail | Pass | Pass |
| 8 | H1 Browse Loads Data Successfully | Pass | Pass | Pass |
| 9 | Browse Callback Load Time | 2.0 sec | 3.7 sec | 3.6 sec |
| 10 | MLC X Profile within 0.1% | Pass | Pass | Pass |
| 11 | MLC Y Profile within 0.1% | Pass | Pass | Pass |
| 12 | Positive Diagonal Profile within 0.1%3 | N/A | Pass | Pass |
| 13 | Negative Diagonal Profile within 0.1%3 | N/A | Pass | Pass |
| 14 | Timing Profile within 0.1% | Pass | Pass | Pass |
| 15 | MLC X Gamma within 0.11 | Fail | Pass | Pass |
| 16 | MLC Y Gamma within 0.11,2 | Fail | Pass | Pass |
| 17 | Positive Diagonal Gamma within 0.13 | N/A | Pass | Pass |
| 18 | Negative Diagonal Gamma within 0.13 | N/A | Pass | Pass |
| 19 | Statistics within 0.1 sec/mm/%1,4 | Fail | Fail | Pass |
| 20 | H1 Figure Display Functional | Pass | Pass | Pass |
| 21 | H2 Browse Loads Data Successfully | Pass | Pass | Pass |
| 22 | H3 Browse Loads Data Successfully | Pass | Pass | Pass |
| 23 | H2 Figure Display Functional | Pass | Pass | Pass |
| 24 | H3 Figure Display Functional | Pass | Pass | Pass |
| 25 | Print Report Functional | N/A | Pass | Pass |
| 26 | Print Report Time | N/A | 4.4 sec | 4.1 sec |
| 27 | H1 Clear Button Functional | Pass | Pass | Pass |
| 28 | H2 Clear Button Functional | Pass | Pass | Pass |
| 29 | H3 Clear Button Functional | Pass | Pass | Pass |
| 30 | Documentation Exists | Pass | Pass | Pass |
1Prior to Version 1.1 only the 27.3 cm x 27.3 cm reference profile existed
2#10 In Version 1.0 a bug was identified where MLC Y T&G effect was not accounted for in the reference data
3Prior to Version 1.1 diagonal profiles were not available
4#11 In Version 1.1.0 a bug was identified where flatness was computed incorrectly
| Function | Version 1.0 | Version 1.1.0 | Version 1.1.1 |
|---|---|---|---|
| AnalyzeProfilerFields | 71.9% | 70.6% | |
| BrowseCallback | 91.2% | 91.2% | |
| CalcGamma | 41.8% | 39.8% | 39.8% |
| ClearCallback | 100.0% | 100.0% | |
| Event | 60.0% | 60.0% | 60.0% |
| FieldUniformity | 88.5% | 80.4% | 81.3% |
| LoadProfilerDICOMReference | 81.2% | 74.6% | |
| LoadVersionInfo | 88.5% | 88.5% | 88.5% |
| ParseSNCprm | 68.6% | 70.6% | |
| PrintReport | 76.3% | 74.7% | |
| UpdateDisplay | 93.0% | 94.7% | 94.7% |
| UpdateStatistics | 83.7% | 93.0% | 93.0% |