Testing against Multiple Versions of Python with tox - goerz/cookiecutter-pypackage GitHub Wiki
The recommended way to support testing against multiple versions of Python is to use tox. It is configured by a file tox.ini in the root of the project folder. It is configured by a file tox.ini in the root of the project folder.
The following is an annotated walk-through for a typical tox.ini file:
[tox]
minversion = 3.7
envlist = py35-{test,runcmd}, py36-{test,runcmd}, py37-{test,runcmd}, run-{cmd,blackcheck,black,isort,isortcheck}, docs, coverage
The minversion is the minimum required version of tox, not the version of Python.
The envlist is a generative specification of all the testing environments. It expands to py35-test, py35-runcmd, py36-test, py36-runcmd, py37-test, py37-runcmd, un-cmd, run-blackcheck, run-black, run-isort, run-isortcheck, docs, and coverage. This is for a package that supports Python 3.5-3.7, where only the most current version Python 3.7 is used for running various linting tools or generating the documentation. In order for tox to run, all required Python versions must be available in the current shell environment. You will have to use pyenv to ensure this. An alternative would be to use the tox-conda plugin. You can do this by appending the line
requires = tox-conda
to the above [tox] section. You should not install tox-conda into your normal Python environment, as this will enable the use of conda for all invocations of tox in all projects (see tox-conda#14).
[testenv:.tox]
envdir = {toxworkdir}/.tox
The .tox testenv is used if the system tox does not meet the minversion defined earlier, or if you use tox-conda ("auto-provisioning"
[testenv]
basepython =
py37,run,docs,coverage: python3.7
py36: python3.6
py35: python3.5
envdir =
py37,run,docs,coverage: {toxworkdir}/py37
py36: {toxworkdir}/py36
py35: {toxworkdir}/py35
deps=
cython
numpy
scipy
usedevelop = true
extras=
dev
setenv =
MATPLOTLIBRC = tests
SPELLCHECK = en_US
description =
py{36,37}-test: Run tests in the corresponding environment
py{36,37}-runcmd: Run arbitrary command following "--" in the corresponding environment
commands_pre =
python -V
commands =
py{35,36,37}-runcmd: {posargs:python -c 'print("No command")'}
py{35,36,37}-test: py.test -vvv --doctest-modules --cov=krotov --nbval --sanitize-with docs/nbval_sanitize.cfg --durations=10 -x -s {posargs:src tests docs README.rst}
The [testenv] section of tox.ini defines the settings for all environments that do not override any settings. This uses some factor-conditional settings to compress the configuration , providing the commands for default *-runcmd and *-test environments. We also re-use the same environment folder depending only on the version of Python. That is, py37-test, py37-runcmd, run-cmd, run-blackcheck, run-black, run-isort, run-isortcheck, docs, and coverage, which all use Python 3.7, all use the same environment directory .tox/py37
The package to be tested gets installed in development mode (pip install -e ., due to usedevelop = true, which will also install all dev-dependencies defined in setup.py (due to extras=dev), The deps defined in tox.ini are dependencies that must be installed into the tox environment before the package to be tested itself gets installed (in this case, one of the dev-dependencies in setup.py builds extension modules, and thus requires cython and numpy/scipy development headers. If you've used tox-conda earlier, replace devs with e.g.
conda_deps =
qutip
conda_channels=
default
conda-forge
This will install qutip from the conda-forge channel.
The rest of the tox.ini file overrides commands for particular environments:
[testenv:docs]
description = Generate the docs
commands =
sphinx-build docs docs/_build {posargs:--color}
[testenv:coverage]
description = "Generate a coverage report"
depends = py37-test
commands =
coverage html
python -c 'import pathlib; print("coverage report available in \{\}".format(pathlib.Path(".") / "htmlcov" / "index.html"))'
[testenv:run-cmd]
description = Run arbitrary command following "--" in the latest venv, e.g. "tox -e run-cmd -- ipython"
commands = {posargs:python -c 'print("No command")'}
[testenv:run-black]
description = Run black formatter in the latest venv
commands = python -m black --skip-string-normalization --line-length 79 src tests
[testenv:run-blackcheck]
description = Run black formatter check in the latext venv
commands = python -m black --skip-string-normalization --line-length 79 --check src tests
[testenv:run-isort]
description = Run isort formatter in the latest venv
commands = python -m isort --recursive src tests
[testenv:run-isortcheck]
description = Run isort formatter check in the latest venv
commands = python -m isort --recursive --check-only src tests
[testenv:run-flake8]
description = Run flake8 formatter check in the latest venv
commands = python -m flake8 src tests
[testenv:run-pylint]
description = Run pylint formatter check in the latest venv
commands = python -m pylint -j 0 src