Automated testing - NetworkGradeLinux/mion-docs GitHub Wiki

Scripts, configuration files, and test results will be available in mion-testing (pull requests welcome!)

Package Tests, or ptests, run test suites that are included in a packages source, and outputs the results as <PASS/FAIL/SKIP>: <testname> Many recipes already have ptests enabled.

For those that do not, they can be written and included in a recipe. This is detailed in the Yocto Project wiki on ptests. This should be done whenever possible for packages that add important functionality; i.e. onlpdump provided by onlpv1.

To make things easy, there's ptest-runner, which runs all the ptests on the target.

Be sure that DISTRO_FEATURE_append += " ptest" is in your local.conf before building!

Log into the target and run ptest-runner. Individual ptests can be found in /usr/lib/<package>/ptest. Note that ptest-runner does not save the results. To do so, direct output into a file. For release candidate testing, run the release_candidate_ptest.sh (NI) to save and upload the results, or commit manually to mion-testing. Run Ptests on all targets.

Automated Runtime Testing is part of OpenEmbedded and provides a set of tests which check and log various runtime functionality of basic utilities such as ping, ssh, and dmesg. This can be run on a qemu image or on hardware, however it is generally sufficient to do so for qemu. Running these tests on hardware requires a number of additional steps and resources, see https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#performing-automated-runtime-testing.

It is also possible, using the python unittest framework, to create additional tests.

FIXME how are these run???????

TODO: Outline of our CI setup and links to GH CI information. Ideally tests can be triggered here, however this is yet to be implemented. For now CI is primarily to check that images build.

Test results should go into mion-testing. See the README in mion-testing for more information.%