Testing - ISET/isetcam GitHub Wiki
Testing
ISETCam's automated tests help contributors check a focused change before sharing it. They are developer tools, not a prerequisite for installing or using ISETCam. Start MATLAB with ISETCam on the path, use a clean session for tutorial or example tests, and begin with the smallest relevant check.
Test levels
| Change | First check | Broader check |
|---|---|---|
| A function or object component | The nearby _tests_ runner, such as sceneUnitTest |
ieUnitTest |
A tutorial in tutorials/ |
ieTutorialTest('selection', 't_name') |
ieTutorialTest |
An example in examples/ |
ieExampleTest('selection', 's_name') |
ieExampleTest |
ieUnitTest discovers and runs the function-based MATLAB tests in ISETCam's
colocated _tests_ directories. Area runners, including sceneUnitTest,
sensorUnitTest, opticsUnitTest, and displayUnitTest, are useful while
developing because they limit the check to one component.
% Run the focused tests for a scene change.
results = sceneUnitTest;
% Run all ISETCam unit tests before a substantial shared change.
results = ieUnitTest;
The unit-test runner
returns a MATLAB TestResult array and prints a summary.
Tutorial and example smoke tests
Tutorials (t_*.m) and examples (s_*.m) are executable teaching and
workflow material. Their runners reset ISETCam state between scripts, so a
script must not depend on objects or variables left by an earlier one.
% Check one tutorial or one example while editing it.
tutorialRun = ieTutorialTest('selection', 't_cameraIntroduction');
exampleRun = ieExampleTest('selection', 's_metricsSPD');
% Run every tutorial or example.
tutorialRun = ieTutorialTest;
exampleRun = ieExampleTest;
% Start at a named tutorial and continue through the path-sorted suite.
tutorialRun = ieTutorialTest('start', 't_cameraIntroduction');
'selection' accepts a script stem, filename, path relative to the tutorial
or example directory, or full path. Use 'start' after correcting a failure
in a long suite; it begins a new run rather than resuming an old one.
Scripts that cannot run unattended
If a tutorial or example genuinely needs unavailable data or hardware, manual interaction, an optional toolbox, or an impractically long run, place this marker on its own line in the script:
% SkipFile
Use it sparingly and document why the script is unsuitable for automated
smoke testing. Scripts that create or refresh repository data should instead
use the data_*.m naming convention; they are not tutorial or example smoke
tests.
Read the results
All runners print a summary. ieTestReport can also report a unit-test result
or list specific statuses from a tutorial/example run:
ieTestReport(results, 'ieUnitTest');
ieTestReport(tutorialRun, 'List', {'failed', 'skipped'});
ieTestReport(exampleRun, 'List', 'all');
Tutorial and example runs save a timestamped checkpoint, progress log, and
planned-file list under ISETCam's local/ directory. If MATLAB closes before
a run returns, pass the checkpoint file or its containing run directory to
ieTestReport to inspect it.
Scope and further guidance
Use ISETCam's built-in runners for ordinary repository development. The separate ISETValidate project is for broader system and regression validation; it is not required for routine ISETCam use or a focused contribution.
For the current operational details, including authoring tests and diagnosing failures, see the source repository's testing workflow and the implementations of tutorial testing, example testing, and test reporting.