Python Testing Tools - BKJackson/BKJackson_Wiki GitHub Wiki
Testing articles
Don't let other people break your toys - from Data Science is Software
How to use Test Driven Development in a Data Science Workflow - Timo Bohm, March 18, 2019
unittest.mock - mock object library
unittest.mock home
Why unit testing is great for LLM coding - Simon Willison's blog
Testing tools from cookiecutter and bus_number
Python has built in testing frameworks via:
- doctests:https://docs.python.org/3/library/doctest.html#module-doctest
- unittest: https://docs.python.org/3/library/unittest.html
Additionally, you'll want to make regular use of:
- pytest: https://docs.pytest.org/en/latest/
- pytest-cov: https://pypi.org/project/pytest-cov/
- hypothesis: https://hypothesis.readthedocs.io/en/latest
Cookiecutter (vanilla flavoured) comes witha setup for the tox testing framework built in.
Where to put tests
Where to put tests
If your package and test code are small and self contained, put the tests in with the package.
If the tests are large or complex, or require reading/writing files, or significant sample data, put your tests outside the package.
Pytest
pytest - Helps you write better programs. The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.
Pytest - Getting Started
Pytest Good Integration Practices - Includes handy instructions for pip install, setup.py, etc.
Solving Algorithmic Problems in Python with Pytest
pytest will run all files of the form test_*.py or *_test.py in the current directory and its subdirectories. More generally, it follows standard test discovery rules.
pytest unittest.TestCase Support
pytest supports running Python unittest-based tests out of the box. It’s meant for leveraging existing unittest-based test suites to use pytest as a test runner and also allow to incrementally adapt the test suite to take full advantage of pytest’s features.
To run an existing unittest-style test suite using pytest, type:
pytest tests
Code Coverage
What are those tests getting up to? Sometimes you think you wrote test cases that cover anything that might be interesting. But, sometimes you're wrong.
coverage.py is an amazing tool for seeing what code gets executed when you run your test suite. You can run these commands to generate a code coverage report:
coverage run --source src -m pytest
coverage html
coverage report
Profiling
Collecting results:
# CLI:
$ python -m cProfile -o step0.prof myscript.py
# Notebook:
%%prun -D step0.prof
Looking at results:
# CLI:
$ python -m pstats step0.prof
# Web browser:
snakeviz step0.prof
Also useful: line_profiler