TestPlan - PortAudio/portaudio Wiki

Test Plan and QA Strategy for PortAudio

Our goal is to have a set of tests that can be used to quickly verify PortAudio on multiple platforms.

Please see our test status for each platform.

We have two kinds of tests.

We recommend that developers periodically run the Self Checking Unit Tests and a few selected Listening Tests before checking in changes to PortAudio.

Self Checking Unit Tests

These tests can be run from the command line. They have a return code of zero if they pass, non-zero if they fail. This allows them to be run as part of an automated test suite.

qa/paqa_errs.c - This test deliberately generates errors then verifies that the appropriate error message is returned.

qa/paqa_devs.c - This test tries to output audio using many combinations of device, numChannels, format, interleave, etc.

qa/paqa_latency.c - This test verifies that the default latencies for each device are non-zero, and that suggestedLatency is honored.

Loopback Test - This test listens to its own output using a loopback cable. It plays sine waves then analyzes the audio input. It can detect pops, noise and phase errors. The loopback tests will ultimately be able to replace most of the Listening Tests. The loopback test can be built on Linux and Mac using "make loopback".

Please see this file for loopback instructions: https://github.com/PortAudio/portaudio/blob/master/qa/loopback/README.txt

Listening Tests

We highly recommend that the following tests and examples be run before checking in changes to portaudio.

A list of additional tests can be found here: http://www.portaudio.com/docs/v19-doxydocs-dev/group__test__src.html

Most of those additional tests are useful and are expected to build and run. But the following tests have problems.

Broken Build Tests

These tests do not build. They should be reviewed and probably deleted.

Failing Tests

Pointless Tests

These programs are either redundant or unnecessary as tests. We should delete them instead of maintaining them. This is only a partial list.

Platform Specific Tests