Specific RTT terms - crocs-muni/randomness-testing-toolkit GitHub Wiki

Following entries explain some expressions in context of RTT configuration and implementation.

Statistical battery

Collection of statistical tests implemented by a third party. Examples of statistical batteries are NIST STS, DIEHARDER or TestU01. RTT serves as unified interface to these statistical batteries.

Test

Single test in any statistical battery, e.g. Birthday Test in DIEHARDER. Test can be variously parametrized and can provide more than one resut.

Variant of a test

In RTT, you can parametrize tests. Not only that, but you can also run single test multiple times with different parameters. Then each executed instance of a test is called a variant of a test. From implementation point of view, each created test has at least single variant.

Subtest

When you execute some tests, they may sometimes provide multiple results. For example, Runs Test from DIEHARDER will measure two properties of data - runs up and runs down. Therefore, two results of the test are provided. Each of those results is called subtest of a test. Since you can execute any test in multiple variants, it is possible that some variants of a test will have multiple subtests. Subtest is subset of a variant and from implementation point of view, each variant has at least one subset.

Statistic

At the end of computation, each test will calculate some statistic based on measured data. Single (sub)test can provide multiple statistics. Statistics are subset of subtests. Based on these statistics we then decide whether the test is passing or failing.