Unit testing with Polly with examples - App-vNext/Polly GitHub Wiki
âšī¸ This documentation describes the previous Polly v7 API. If you are using the new v8 API, please refer to pollydocs.org.
How to approach unit-testing code wrapped in Polly policies depends what you are aiming to test.
This page provides a longer version of our main unit-testing page, adding code examples.
TL:DR; Polly's NoOpPolicy
allows you to stub out Polly, to test your code as if Polly were not in the mix. Use DI to provide policies to consuming classes; tests can then stub out Polly by injecting NoOpPolicy
in place of real policies.
A common need is to test the logic of your system-under-test as if Polly were not part of the mix.
Perhaps you have code modules for which you already had unit tests, including success and failure cases. You then retro-fit Polly for resilience. How does having the Polly policy in play affect your existing unit tests? Do all the tests need adjusting? How do I test what my code does without Polly 'interfering'?
Equally, when you include Polly from the outset of a project, it can still make sense to test concerns separately - have one set of tests which assert on behaviour without Polly; and other tests which test the additional resilience, separately.
A simple strategy uses dependency injection, so that creating the policy is outside the system-under-test:
public class Repository<T> : IRepository<T> {
public Repository(IAsyncPolicy policy, /* other constructor params */)
{
this.resiliencePolicy = policy;
}
public async Task<T> GetById(string id)
{
return await resiliencePolicy.ExecuteAsync(() => /* code for getting from the underlying system */);
}
// ...
}
For tests where you want to remove Polly from the mix, you can then inject NoOpPolicy
rather than the policies you use in production.
var repoToTestWithNoPolly = new Repository<Foo>(Policy.NoOpAsync(), ...);
When code is executed through NoOpPolicy
, it does nothing but execute the code as if Polly was not involved. Polly is effectively stubbed out of the test.
The example is a repository class, but the pattern can be used anywhere you use Polly: when guarding calls to third-party APIs; to cloud components through cloud-provider SDKs; etc.
You can also construct tests using mocking tools like Moq or NSubstitute to inject eg a Mock<IAsyncPolicy>
:
var mockPolicy = new Mock<IAsyncPolicy>();
var repoToTestWithMockPolicy = new Repository<Foo>(mockPolicy.Object, ...);
This allows you to write tests confirming that the system-under-test does use the passed-in policy:
var resultNotOfInterest = repoToTestWithMockPolicy.GetById("someId");
mockPolicy.Verify(p => p.ExecuteAsync(It.IsAny<Func<Task<Foo>>>()));
The above example was intentionally the simplest possible to demonstrate providing policies by DI, and using NoOpPolicy
or mocks to stub out Polly. However, injecting IAsyncPolicy policy
directly is too simplistic for many scenarios.
If IAsyncPolicy policy
will be resolved from a DI container, this can imply you only have one registration of IAsyncPolicy
for the whole application. (This would be the case for example with .NET Core's in-built DI.) For real-world applications using Polly in multiple areas, this is unrealistic - you typically need different resilience strategies for multiple external systems.
PolicyRegistry
is designed to help you manage a collection of policies in your application, while still playing nicely with DI.
To consume policies by DI with PolicyRegistry, make a PolicyRegistry instance the item injected into the consuming classes. From that policy registry instance, retrieve the relevant policy/ies to use:
public class Repository<T> : IRepository<T> {
public Repository(IReadOnlyPolicyRegistry<string> policyRegistry, /* other constructor params */)
{
this.resiliencePolicy = policyRegistry.Get<IAsyncPolicy>("MyRepositoryPolicy"); // As an example - of course you can use consts rather than magic strings.
}
public async Task<T> GetById(string id)
{
return await resiliencePolicy.ExecuteAsync(() => /* code for getting from the underlying system */);
}
}
At the point of use, we only want to read policies from the registry, so we can inject IReadOnlyPolicyRegistry<string>
.
With this pattern, tests can stub Polly out of the picture by configuring the policy registry to return NoOpPolicy
policies:
var registryReturningNoOpPolicy = new PolicyRegistry {
{ "MyRepositoryPolicy", Policy.NoOpAsync(); }
};
var repoToTestWithNoPolly = new Repository<T>(registryReturningNoOpPolicy, ...);
As an alternative route to achieving the same end as the above snippet, you could use a mocking tool like Moq and inject a Mock<IReadOnlyPolicyRegistry<string>>
configured to return Policy.NoOpAsync()
for the policy to use.
Finally, with the PolicyRegistry case, you can also configure a registry to return mock policies:
var mockPolicy = new Mock<IAsyncPolicy>();
var registryReturningMockPolicy = new PolicyRegistry {
{ "MyRepositoryPolicy", mockPolicy.Object; }
};
var repoWithMockPolicy = new Repository<T>(registryReturningMockPolicy, ...);
Some applications have a requirement to update policies dynamically at run-time, for example to respond to changes in external configuration. One approach to this is to have changes in configuration push updates to a PolicyRegistry. If you use that approach, testing patterns for the code consuming policies would follow the strategies with PolicyRegistry above.
You may alternatively have code which generates policies dynamically each time they are needed, at run-time. If taking this approach, we recommend extracting the policy-generation logic as a policy factory:
interface IPolicyFactory {
IAsyncPolicy CreatePolicy(); // (example overload - you may have a wider range)
}
with consuming classes taking the IPolicyFactory
by DI:
public class Repository<T> : IRepository<T> {
public Repository(IPolicyFactory policyFactory, /* other constructor params */)
{
this.resiliencePolicy = policyFactory.CreatePolicy();
}
// ...
}
With this pattern, all the strategies previously described for PolicyRegistry
can be applied in tests:
- Tests can create a
NoOpPolicyFactory : IPolicyFactory
orMock<IPolicyFactory>
whoseCreatePolicy()
method always returns aNoOpPolicy
- allowing you to stub Polly out of tests. - Tests can create a policy factory configured to return instances of
Mock<IAsyncPolicy>
, allowing you to pass mock policies into the system-under-test.
TL;DR: Configure a mock of the underlying system to return faults the policies should handle. Can be useful as a specification for, and regression check on, the faults you intend to handle.
With policy creation separated from policy consumption as described above, you can:
- [2a] test policies with integration tests through the consuming code; or
- [2b] test policies with unit-tests, independent of the consuming code.
Let's expand our earlier respository example, imagining we have some underlying database, IUnderlyingDatabase db
:
public class Repository<T> : IRepository<T> {
private IAsyncPolicy resiliencePolicy;
private IUnderlyingDatabase db;
public Repository(IReadOnlyPolicyRegistry<string> policyRegistry, IUnderlyingDatabase db)
{
this.resiliencePolicy = policyRegistry.Get<IAsyncPolicy>("MyRepositoryPolicy");
this.db = db;
}
// (simple example - add CancellationToken support where available, in real code)
public async Task<T> GetById(string id);
{
return await resiliencePolicy.ExecuteAsync(() => db.GetByIdAsync(id));
}
}
To focus on structuring test code, we'll take a simple example, that the resilience policy just retries once for TimeoutException
:
public PolicyRegistry<string> ConfigureResiliencePolicies()
{
return new PolicyRegistry {
{ "MyRepositoryPolicy", Policy.Handle<TimeoutException>().Retry(1); },
// ... and other policies for elsewhere round the app
};
}
In our integration test, we then use a mock of the underlying system (IUnderlyingDb
) to throw particular faults or a certain fault sequence, and check that our policy handles them:
public async Task When_db_get_throws_single_TimeoutException_Then_policy_handles_exception()
{
// Arrange - registry
var registry = ConfigureResiliencePolicies(); // example kept simple; realistically, this method probably lives on a configuration or startup class in your app
// Arrange - mock database to throw one TimeoutException
Mock<IUnderlyingDb> dbMock = new Mock<IUnderlyingDb>();
int invocations = 0;
Foo someFoo = new Foo();
dbMock.Setup(db => db.GetById<Foo>(It.IsAny<string>()))
.Returns(invocations++ == 0 ? throw new TimeoutException() : someFoo);
// Arrange - repository
var repo = new Repository(registry, dbMock.Object, ...);
// Act
var returned = await repo.GetById("someId");
// Assert.
Assert.AreSame(someFoo, returned);
}
This style of test can be useful if you want tests that the policies you have configured do handle particular faults in a certain way. The tests can also have regression value: if somebody changes the resilience configuration, the test fails.
With policy creation separated from policy consumption (as described in section [1] above), you can alternatively unit-test how your policies respond to faults, independent of the consuming system.
public async Task MyRepositoryPolicy_handles_single_TimeoutException()
{
// Arrange - registry
var registry = ConfigureResiliencePolicies();
var policy = registry.Get<IAsyncPolicy>("MyRepositoryPolicy");
// Act
int invocations = 0;
var result = await policy
.ExecuteAsync<bool>(invocations++ == 0 ? throw new TimeoutException() : Task.FromResult(true));
// Assert.
Assert.IsTrue(result);
}
Note: With either [2a] or [2b], it is easy to stray towards recreating Polly's own test suite (see point [4] below]). For example, tests such as "If I configure 1 retry it does 1; if I configure 2 retries it does 2", or "if I attach an onRetryAsync
delegate to the policy, it does get invoked" would both duplicate tests Polly already covers. More useful are tests which verify the logic specific to your app ("if a retry is invoked, this gets logged to my logger").
[3] I want to unit-test how my code reacts to results or faults returned by the execution through Polly
TL;DR Mock your policies to return or throw particular outcomes, to test how your code responds.
You may want to test how your code reacts to results or faults returned by an execution through Polly. For instance, you may want to test how your code reacts if, despite resilience strategies, the execution eventually fails.
Polly policies all fulfil execution interfaces (ISyncPolicy
, ISyncPolicy<TResult>
, IAsyncPolicy
and IAsyncPolicy<TResult>
). These interfaces describe the .Execute/Async()
overloads available on policies.
You can configure these methods on a mock policy, to return or throw faults you want to simulate. Let's expand the Repository<T>
example:
public class Repository<T> : IRepository<T> {
/* elided - private variables to store params */
public Repository(IReadOnlyPolicyRegistry<string> policyRegistry,
IUnderlyingDatabase db,
ILogger<IRepository<T>> logger)
{
/* elided - assign params to private variables */
}
public async Task<T> GetById(string id);
{
try {
return await resiliencePolicy.ExecuteAsync(() => db.GetByIdAsync(id));
}
catch (Exception ex)
{
logger.logError(ex, /* etc */);
throw;
}
}
}
We can then write a test:
public async Task When_execution_eventually_fails_Then_logs()
{
// Arrange
var toThrow = new BrokenCircuitException();
var mockPolicy = new Mock<IAsyncPolicy>();
mockPolicy.Setup(p => p.ExecuteAsync(It.IsAny<Func<Task<Foo>>>())
.Throws(toThrow);
var registryReturningMockPolicy = new PolicyRegistry {
{ "MyRepositoryPolicy", mockPolicy.Object; }
};
var mockLogger = new Mock<ILogger<IRepository<Foo>>>();
var repo = new Repository<Foo>(registryReturningMockPolicy, new Mock<IUnderlyingDb>().Object, mockLogger);
// Act
await Assert.ThrowsAsync<BrokenCircuitException>(() => repo.GetById("someId"));
// Assert
mockLogger.Verify(l => l.LogError(toThrow, /* etc */));
}
If using the policy.ExecuteAndCapture/Async(...)
methods, the return types PolicyResult
and PolicyResult<TResult>
have public factory methods, allowing you to mock .ExecuteAndCapture(...)
overloads to return the PolicyResult
of your choice.
TL:DR; Bear in mind the Polly codebase already tests this for you extensively.
An understandable desire when introducing Polly to a project is to want to check the Polly policy does what it says on the tin. If I configure Policy.Handle<FooException>().Retry(3)
, it would be nice to check it really works, right?
Writing unit-tests to verify that Polly works can be a very valuable way to explore and understand what Polly does. But, to allow you to concentrate on delivering your business value rather than reinventing Polly's test wheel, keep in mind that the Polly codebase tests its own operation extensively. (As at Polly v6.0, the Polly codebase has around 1700 tests per target framework.)
You can also explore and run the Polly-samples to see policies working individually, and in combination.
If you write your own integration tests around policies in your project, note the possibility to manipulate Polly's abstracted SystemClock
. Where a test would usually incur a delay (for example, waiting the time for a circuit-breaker to transition from open to half-open state), manipulating the abstracted clock can avoid real-time delays. For insight into how to do this, pull down the codebase and check out how Polly's own unit tests manipulate the clock.
Thoughts/questions about unit-testing? Post an issue on the issues board.