Testing guidelines - ProofDrivenQuerying/pdq GitHub Wiki

• JUnit, the framework we use for unit tests, requires us to annotate types, fields and methods to indicate which tests are run. When doing so, the tests can be run automatically within Eclipse, but most importantly, they are run as part of Maven's build process, so if a test fails, compilation aborts.

• In each project, there is a main/src folder containing sources for the actual code, and a test/src containing unit tests.

• The package and classes structure in test/src should ideally match that of main, i.e., for each class some.package.R in main, there is a some.test.package.RTest in test.

In practice, it is not always easy nor practical to strictly adhere to that, but we should try to get something close. I think we can easily converge on what is "sufficient" case by case.

• Each RTest class hosts a bunch of tests for each public method in R. For instance, if there is a method R.doThis(arg1, arg2), then RTest should test that method under various circumstances. Typically:

  • checking whether doThis() works for some expected inputs,
  • checking whether doThis() works for some unusual inputs, when that makes sense, (e.g. lower-upper bounds number, negatives, etc.), and
  • checking whether doThis() throws the correct exception, when something goes wrong (e.g. inconsistent inputs, nulls, etc.)

• We use Mockito, a library for mock objects, to run fine-grain tests on complex objects, such as the planner or chasers, without having to initialize every underlying stuff (database, etc). The algebra package has very simple examples on how to use Mockito.

• In the process of adding unit tests, there are various little adjustments that need to be brought to the code to catch corner cases. So, after completing tests on a set of classes, and before committing, it is important to rerun the current regressions tests, to make sure no new problem was introduced.