Testing is carried out using the pytest package and coverage with pytest-cov. To run a test you can use the command pytest -s --cov=reporter_app/ tests/ which is also the command run in the testing pipeline. Tests are not run on every commit but instead reserved for merges and commits to the main branch. A developer is expected to run pytest on their own machine before pushing a new commit.

File structure

Tests are split into functional (tests that fake the app and/or database) and unit tests (functions that do not need the app to be running to function). To further organise tests, tests are divided into their corresponding blueprint and what is being targeted. E.g. the users blueprint could have all of its functional tests in a folder containing a file for route tests, one for database tests and another for form tests.

Repository root
|
|-migrations
|     |
...   ...
|
|-reporter_app
|     |
...   ...
|
|-tests
|     |
|     |-functuonal
|     |     |
|     |     |-blueprint test folders
|     |     |    |
|     |     |    |-test files (.py)
...   ...   ...
|     |
|     |-unit
|     |     |
...   ...   ...
|     |
|     |-conftest.py
|     |-utils.py
...   ...

conftest.py

Conftest contains any objects we may need in multiple tests. The main object located here is a number of pytest fixtures that allow us to fake the Flask application and database. Each of these is rebuilt for every test which adds extra computation and thus runtime but means there is no way for one test to change another test (e.g. by leaving a user logged in). The Flask application works similar to a regular Flask application but the database is an sqlite database that runs in RAM. This has some benefits for testing but also comes with a few drawbacks which are detailed in the Limitations section.

util.py

Utils contains a number of functions that need to be run for multiple tests. This may for instance be logging in or creating a user.

Writing a new test

• The first parameters in the function need to be pytest fixtures (defined in conftest.py) followed by parameters in parametrize.

@pytest.mark.parametrize("param, expect", [
	([ADMIN_USER, 'admin', True],[200]),
	([STANDARD_USER, 'standard', False],[200]),
])
def demo_test(db, app_client, param, expect):
	"""
	GIVEN <what does the test take as an imput?>
	WHEN <what action is completed?>
	THEN <what should the system do?>

	parmas:
	param: object type and what it is
	expect: object type and what it is
	"""

        # Make a user and login
	create_user(db, param[0], param[1], verified=param[2])
        login(app_client, param[0]['email'], param[0]['password'])

	# get target user
	subject_user = User.query.filter_by(username=param[0]['param']).first()

	# toggle role and make sure the status code and role are expected
	response = app_client.get('/user/' + str(subject_user.id))
	assert response.status_code == expect[0]

Limitations

The test database is using SQlite3 and therefore may not behave the same as a MySQL database in all situations. Known ones are as follows:

• There is no validation for length of a string