There are (at least) two types of tests to be included in the MULTIPLY platform: Unit tests and integration tests.

Unit Tests

Unit Tests shall serve to test the functionality of small code units (e.g, functions). In a unit test, an input is created and passed as a parameter to the code to be tested. Then, it is asserted that the result of the code is as expected. Ideally, the code will be tested for different inputs, including invalid data. As test framework we decided to use pytest (see https://docs.pytest.org/en/latest/ , where you will also find examples). Unit tests shall be written before the actual code is written. This is a good practice to ensure that the code is created with the objective to accomplish a specific task, to prevent bugs, and to identify problems as early as possible. All tests shall be contained in a directory "test" in the repository. Within the test directory, the directory structure of the python package to be tested shall be reproduced. There should be one test file per code file. The test file will be named like the file with the python code, except it carries the prefic "test_". In one such test file there can be multiple unit tests defined. A unit test shall be named like the function it is testing, except for the prefix "test_". The name of the unix test files and the unit tests can also be extended to explain what is tested: For example, if a function is called my_function, a test might be called test_my_function_invalid_input.

Integration tests

Opposed to unit tests, which test a small, encapsulated part of code, integration tests test larger portions. To be concise, they test the interplay of the various functions, often by running the whole system. Like unit tests, the input needs to be defined beforehand and it is later checked whether the output meets the expectations. As intergration tests usually do not refer to a single python module, they need to be named differently. A good way to do this is test_x_integration, where x stands for the part that is tested. In this way, whole components can be integration-tested. The dummy in the multiply-core can be seen as integration testing of the whole platform, as it uses inputs and outputs from all components and connects them in a meaningful way.

Travis CI

Travis CI is a system for continuous integration. We use it to automatically run all tests (unit and integration tests). To do this, a file named .travis.yml. needs to be placed in the repository (see here for how to set it up to work with conda: https://conda.io/docs/travis.html ). You can log in onto travis-ci.org with your github account. there you will have the choice to run travis ci on a project. After each push to github, travis will execute the steps stated in the yml-file and run all tests. A disclaimer can be added to the readme file of the repository to inform whether all tests have been run succesfully.