Every new feature in SymPy should have a test associated with it. In addition, whenever a bug is fixed, a test should be added that would fail before the bug was fixed but now passes. That way, the bug cannot be reintroduced without failing the test.

Tests go in the corresponding test file for a given file. For example, for sympy/solvers/ode/ode.py, the tests are in sympy/solvers/ode/tests/test_ode.py. Note that there isn't an exact correspondence between tests and files in all cases. For example, there might be a test file for testing a specific thing that doesn't correspond to a specific submodule. For example, sympy/core/tests/test_args.py tests the entire library and doesn't correspond to any one file.

Test files should have test functions with assertions. A test function should start with test_ and be named something useful corresponding to what is being tested. The test_ prefix is important. Any function that does not have it will not be run by the test runner. The assertions are lines like assert function(x) == result where function is what is being tested. For example, here is a test in sympy/functions/elementary/tests/test_exponentials.py:

def test_exp_infinity():
    assert exp(I*y) != nan
    assert refine(exp(I*oo)) is nan
    assert refine(exp(-I*oo)) is nan
    assert exp(y*I*oo) != nan
    assert exp(zoo) is nan
    x = Symbol('x', extended_real=True, finite=False)
    assert exp(x).is_complex is None

Note that the variable y is defined at the top of the file with no assumptions with

from sympy.abc import x, y, z

but the variable x needs special assumptions for the test, so it is defined only for the one specific test.

When in doubt, look at how existing tests are written.

Doctests

All docstrings should have examples. See https://docs.sympy.org/latest/documentation-style-guide.html#examples-section. These examples are tested (called "doctests"), but to be sure, you should not think of these as tests. Rather, they are examples that happen to be tested. This is important, because tests and doctests serve different purposes.

• Tests serve to make sure the behavior of a function is correct. Examples serve to help users understand what a function does.

• Tests should test lots of examples. Examples should only show as many examples as are useful to help users.

• Tests should test all corner cases of a function. Examples should only show corner cases if they are instructive. Often corner cases are confusing to show in examples.

• Tests count toward test coverage. Doctests do not.

• It doesn't matter what the code in a test looks like, as long as it tests the function. But a confusing example is harder to understand. Hence, you should avoid "working around" issues in doctests in ways that make the examples harder to read. The ... feature of doctest can help here. In some cases, it may be prudent to just skip a doctest rather than make it so convoluted that it can't be read (using # doctest: +SKIP).

Basically, don't think of doctests as tests. They are examples that just happen to be tested.

Tests gotchas

There are some gotchas to be careful of when writing tests. PR reviewers should always be on the lookout for these. It is also prudent to occasionally grep the repo to fix any instances of these that slipped in.

1. Tests without an assert. For example,

   # BAD
   sin(pi) == 0 
   

   instead of

   # GOOD
   assert sin(pi) == 0
   

   These won't actually do anything when they fail. Sometimes, this is done intentionally just to make sure something doesn't raise an exception. In that case, a comment should be added to note this.

2. Tests like assert expr == expr don't actually do anything. If you want to test that an expression does not evaluate, use the unchanged helper. For example, instead of

   # BAD
   assert Add(x, y) == Add(x, y)
   

   use

   # GOOD
   from sympy.core.expr import unchanged
   assert unchanged(Add, x, y)
   

3. The raises context manager should test exactly one exception. If there is more than one exception, it can only test the first one. It is better to use raises with the lambda form when possible to avoid this issue. For example, instead of

   # BAD
   with raises(ValueError):
       factorint(x)
       # This line is not actually tested
       factorint(1.0)
   

   use

   # GOOD
   with raises(ValueError):
       factorint(x)
   with raises(ValueError):
       factorint(1.0)
   

   or even better

   # BETTER
   raises(ValueError, lambda: factorint(x))
   raises(ValueError, lambda: factorint(1.0))
   

4. Similar to raises, the warns and warns_deprecated_sympy() context managers can only test one thing at a time. Instead of

   # BAD
   with warns_deprecated_sympy():
       deprecated_function1()
       deprecated_function2()
   

   use

   # GOOD
   with warns_deprecated_sympy():
       deprecated_function1()
   with warns_deprecated_sympy():
       deprecated_function2()