PC2 JUnits - pc2ccs/pc2v9 GitHub Wiki

Overview

An important part of the development process for adding bug fixes and improvements to PC² is creation of JUnit tests. A JUnit is a Java class that supports executing a separate component, called the unit under test (UUT), and verifying that the UUT produces the correct (expected) results when given a specific set of input data. Currently PC² uses JUnit Version 4, although there are plans to upgrade to support JUnit 5.

It is expected that all Pull Requests submitted for incorporation into PC² include appropriate JUnit tests; failing to include appropriate JUnit tests could result in rejection of a PR regardless of the merits of the PR otherwise.

New PC² JUnit tests could be in the form of additions to existing PC² JUnit test classes (which might typically be the case when a PR modifies an existing PC² class for which there was already an existing JUnit test), and/or in the form of a completely new set of JUnit test classes.

JUnit Organization in PC²

JUnit Source Code

All PC² JUnits are stored under the folder test in the pc2v9 project. Beneath the test folder there exists a hierarchical Java folder (package) structure that mirrors the package structure of the src folder of the pc2v9 project. All JUnit tests for a given class under the src folder should reside at the corresponding location under the test folder.

For example, there exists a class under pc2v9/src whose fully-qualified class name is edu.csus.ecs.pc2.core.list.LoginList and whose source code lives in the file LoginList.java in the folder pc2v9/src/edu/csus/ecs/pc2/core/list. The corresponding JUnit test for the LoginList class is named LoginListTest (fully-qualified class name edu.csus.ecs.pc2.core.list.LoginListTest) and the source code for class LoginListTest lives in pc2v9/test/edu/csus/ecs/pc2/core/list/LoginListTest.java.

Note that the above implies that the source code file for all PC² JUnit tests must include an appropriate package statement, and that package statement should be identical to the package statement in the corresponding PC² class being tested (and note in particular that the package statement in the JUnit should not start with "test", just as the package statements in PC² source code files do not start with "src").

PC² JUnits are by convention named as the name of the class being tested (the UUT) followed by the word Test -- for example, the JUnit test class for the LoginList class is named LoginListTest. PC² JUnits should include detailed comments explaining what is being tested and what the expected results are.

JUnit Test Data and Output

PC² JUnits frequently need external input data to perform their tests. Test data files for PC² JUnits should be stored beneath the pc2v9/testdata folder, in a subfolder whose name corresponds to the JUnit test class.

For example, there exists a JUnit test class VivaAdapterTest which performs a variety of tests to insure that the VIVA Input Validator Adapter works correctly; in order to perform these tests the VivaAdapterTest class needs to read a variety of input data files and corresponding VIVA pattern files. The test data files for the VivaAdapterTest JUnit are stored in pc2v9/testdata/VivaAdapterTest. Any newly-developed JUnits which require input data should similarly have their input data files stored in a correspondingly-named folder beneath pc2v9/testdata.

PC² JUnits sometimes also need to produce output files -- for example, so that the JUnit can subsequently read back the produced output files to verify their correctness. All output produced by PC² JUnits should be stored under the folder pc2v9/testout, in a subfolder whose name corresponds to the JUnit test class. (For example, if a new JUnit named MyNewTest creates an output file, that output file should be written to folder pc2v9/testout/MyNewTest.)

See the section below on Additional PC² JUnit Development Support for details on conveniently accessing the PC² testdata and testout folders corresponding to a JUnit test class.

JUnit Development Support

Standard JUnit 4 Development

The conventional approach to defining a JUnit test class (at least, in JUnit 4) is to have the JUnit test class extend class junit.framework.TestCase. This allows JUnit tools to recognize the JUnit class as a unit test. For example, adding the JUnit plugin to Eclipse (as described for example in this tutorial) allows selecting the pc2v9/test folder then right-clicking and selecting Run As > Junit Test, which will execute all of the JUnit tests in the pc2v9/test folder. Similarly, selecting an individual JUnit class (for example, LoginListTest) and then selecting Run As > JUnit Test will run (just) that JUnit.

Individual test methods in a JUnit test class should have method names starting with the word test (this is how JUnit recognizes the methods it should invoke when the JUnit test class is executed). The remainder of the method name should provide insight to what the method is intended to test -- for example, testResultIsNotNull().

Implementing a JUnit test class by extending junit.framework.TestCase provides access to a variety of support facilities implemented by the JUnit library. For example, the test methods in a JUnit can be grouped together in test suites; the test class can invoke setup() and tearDown() methods to be executed prior to and following the execution of test methods; and the test class can invoke a wide variety of JUnit-provided assert() methods to verify that expected conditions hold (for example, assertEquals(), assertTrue(), assertNotNull(), assertSame(), etc.) The LoginListTest JUnit mentioned above is an example of a JUnit which extends junit.framework.TestCase and uses a variety of these techniques.

Additional PC² JUnit Development Support

PC² defines a special JUnit development support class named AbstractTestCase. This class directly extends junit.framework.TestCase, so a PC² JUnit test class can be created by extending AbstractTestCase. The advantage of extending AbstractTestCase is that it provides numerous additional PC²-specific support methods.

For example, AbstractTestCase provides method ensureOutputDirectory(), which checks for the existence of a folder in the testout folder corresponding to the name of the current JUnit, and creates the appropriately-named folder if it does not exist. This allows the JUnit developer to avoid having to explicitly create the appropriate output data folder name.

As another example, AbstractTestCase provides method getOutputDataDirectory(), which returns the path to the appropriately-named output folder for the current JUnit. getOutputDataDirectory() automatically includes the name of the current JUnit test class in the returned value.

AbstractTestCase also contains a variety of additional assertion methods and other utilities, such as:

  • assertDirectoryExists(String) -- verifies that a folder of the specified name exists.
  • assertFileExists(String) -- verifies that a file by the specified name exists.
  • assertFileContentsEquals(File,File) -- verifies that the contents of two files are identical.
  • countString(String, String) -- returns the number of occurrences of a given string in another string.
  • assertCount(String, int, String, String) -- verifies that a specific number of occurrences of a given string appears in another string.
  • many other useful utilities.

PC² contains a number of sample contests which can be used for testing, including testing in JUnits (these sample contests are stored in the samps/contests folder in the pc2v9 project). AbstractTestCase provides a variety of "contest-specific" utilities, some of which are generic and some of which utilize the preexisting sample contests. These contest-related methods include, for example:

  • createSampleDataFile(String) -- creates a test data file from a specified string.
  • createSerializedFiles(String []) -- creates a set of PC² SerializedFiles from a list of file names.
  • getAdministratorAccounts(IInternalContest) -- given a PC² IInternalContest (the "model" for a contest), returns an array of all the Administrator accounts defined in the contest (variations also exist for other types of accounts).
  • getSampleContestsDirectory() -- returns the path to the samps/contests folder (the place in the PC² distribution where sample contests are stored).
  • getSampleContestYaml(String) -- returns the name of the YAML Contest Configuration file for the specified sample contest.

These methods and many others are available in AbstractTestCase to help simplify the development of PC² JUnits; see the AbstractTestCase Javadoc for additional details and method names.