Timeout - App-vNext/Polly GitHub Wiki
âšī¸ This documentation describes the previous Polly v7 API. If you are using the new v8 API, please refer to pollydocs.org.
To ensure the caller never has to wait beyond the configured timeout.
To enforce a timeout on actions having no in-built timeout.
Waiting forever (having no timeout) is a bad design strategy: it specifically leads to the blocking up of threads or connections (itself often a cause of further failure), during a faulting scenario.
Beyond a certain wait, success is unlikely.
TimeoutPolicy timeoutPolicy = Policy
.Timeout([int|TimeSpan|Func<TimeSpan> timeout]
[, TimeoutStrategy.Optimistic|Pessimistic]
[, Action<Context, TimeSpan, Task> onTimeout])
AsyncTimeoutPolicy timeoutPolicy = Policy
.TimeoutAsync([int|TimeSpan|Func<TimeSpan> timeout]
[, TimeoutStrategy.Optimistic|Pessimistic]
[, Func<Context, TimeSpan, Task, Task> onTimeoutAsync])
Parameters:
-
timeout: the time after which the execute delegate or func should be abandoned. Can be specified as an
int
(number of seconds),TimeSpan
, or func returning aTimeSpan
- timeoutStrategy (optional): whether to time out optimistically or pessimistically (see below)
-
onTimeout/Async (optional): an action to run when the policy times-out an executed delegate or func. The action is run before the
TimeoutRejectedException
(see below) is thrown.
Throws:
-
TimeoutRejectedException
, when an execution is abandoned due to timeout.
TimeoutPolicy
supports optimistic and pessimistic timeout.
TimeoutStrategy.Optimistic
assumes that delegates you execute support co-operative cancellation (ie honor CancellationToken
s), and that the delegates express that timeout by throwing OperationCanceledException
(as is standard for most .NET library code).
If the code executed through the policy is your own code, the recommended pattern is to call cancellationToken.ThrowIfCancellationRequested()
at suitable intervals in the cancellable work.
The policy combines a timing-out CancellationToken
into any passed-in CancellationToken
, and uses the fact that the executed delegate honors cancellation to achieve the timeout. You must use Execute/Async(...)
(or similar) overloads taking a CancellationToken
, and the executed delegate must honor the token passed in to the lambda expression:
IAsyncPolicy timeoutPolicy = Policy.TimeoutAsync(30, TimeoutStrategy.Optimistic);
HttpResponseMessage httpResponse = await timeoutPolicy
.ExecuteAsync(
async ct => await httpClient.GetAsync(requestEndpoint, ct), // Execute a delegate which responds to a CancellationToken input parameter.
CancellationToken.None // CancellationToken.None here indicates you have no independent cancellation control you wish to add to the cancellation provided by TimeoutPolicy.
);
You can also combine your own CancellationToken
(perhaps to carry independent cancellation signalled by the user). For example:
CancellationTokenSource userCancellationSource = new CancellationTokenSource();
// userCancellationSource perhaps hooked up to the user clicking a 'cancel' button, or other independent cancellation
IAsyncPolicy timeoutPolicy = Policy.TimeoutAsync(30, TimeoutStrategy.Optimistic);
HttpResponseMessage httpResponse = await timeoutPolicy
.ExecuteAsync(
async ct => await httpClient.GetAsync(requestEndpoint, ct),
userCancellationSource.Token
);
// GetAsync(...) will be cancelled when either the timeout occurs, or userCancellationSource is signalled.
We recommend using optimistic timeout wherever possible, as it consumes less resource. Optimistic timeout is the default.
TimeoutStrategy.Pessimistic
recognises that there are cases where you may need to execute delegates which have no in-built timeout, and do not honor cancellation.
TimeoutStrategy.Pessimistic
is designed to allow you nonetheless to enforce a timeout in these cases, guaranteeing still returning to the caller on timeout.
What is meant by timeout in this case is that the caller 'walks away': stops waiting for the underlying delegate to complete. An underlying delegate which does not honour cancellation is not magically cancelled - see What happens to the timed-out delegate? below.
For asynchronous executions, the extra resource cost is marginal: no extra threads or executing Task
s involved.
Note that pessimistic timeout for async executions will not timeout purely synchronous delegates which happen to be labelled async. It expects that the executed async
code conforms to the standard async
pattern, returning a Task
representing the continuing execution of that async work (for example when the executed delegate hits the first internal await
statement).
Unit tests attempting to demonstrate async timeout policy against a Thread.Sleep(...)
will fail due to the intentional design choice that async timeout policy is optimised for the majority well-behaved async case, not for the actually-synchronous edge case. To write unit-tests against async timeout policy, test with await Task.Delay(..., cancellationToken)
, not with Thread.Sleep(...)
. Detailed discussion and examples: #318; #340; #623.
For synchronous executions, the ability of the calling thread to 'walk away' from an otherwise un-timeout-able action comes at a cost: to allow the current thread to walk away, the policy executes the user delegate as a Task
on a ThreadPool
thread.
Because of this cost we do not recommend pessimistic synchronous TimeoutPolicy in scenarios where the number of concurrent requests handled is potentially high or unbounded. In such high/unbounded scenarios, that cost (effectively doubling the number of threads used) may be very expensive.
We recommend pessimistic synchronous TimeoutPolicy in conjunction with explicitly limiting the parallelism of calls on that codepath. Options to control parallelism include:
- using Polly
BulkheadPolicy
(which is a parallism-throttle) upstream of the Timeout policy - using a concurrency-limiting
TaskScheduler
upstream of the Timeut policy - using a circuit-breaker policy upstream of the TimeoutPolicy, with the circuit-breaker configured to break if too many downstream calls are timing out. The prevents an excessive number of calls being put through to the downstream system (and blocking threads) when it is timing out
- any other in-built parallelism controls of the calling environment.
A key question with any timeout policy is what to do with the abandoned (timed-out) task.
Polly will not risk the state of your application by unilaterally terminating threads. Instead, for pessimistic executions, TimeoutPolicy
captures and passes the abandoned execution to you as the Task
parameter of the onTimeout/onTimeoutAsync
delegate.
This prevents these tasks disappearing into the ether (with pessimistic executions, we are talking by definition about delegates over which we expect to have no control by cancellation token: they will continue executing until they either belatedly complete or fault).
The task
property of onTimeout/Async
allows you to clean up gracefully even after these otherwise ungovernable calls. When they eventually terminate, you can dispose resources, carry out other clean-up, and capture any exception the timed-out task may eventually raise (important, to prevent these manifesting as UnobservedTaskException
s):
Policy.Timeout(30, TimeoutStrategy.Pessimistic, (context, timespan, task) =>
{
task.ContinueWith(t => { // ContinueWith important!: the abandoned task may very well still be executing, when the caller times out on waiting for it!
if (t.IsFaulted)
{
logger.Error($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, eventually terminated with: {t.Exception}.");
}
else if (t.IsCanceled)
{
// (If the executed delegates do not honour cancellation, this IsCanceled branch may never be hit. It can be good practice however to include, in case a Policy configured with TimeoutStrategy.Pessimistic is used to execute a delegate honouring cancellation.)
logger.Error($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, task cancelled.");
}
else
{
// extra logic (if desired) for tasks which complete, despite the caller having 'walked away' earlier due to timeout.
}
// Additionally, clean up any resources ...
});
});
Note: In the async case of the above code, do not code await task.ContinueWith(...)
, as that will make the onTimeoutAsync:
delegate await the completion of the abandoned task
which the policy has just walked away from ... causing the policy to wait the full execution time of task
and defeating the timeout. The async case must just attach the continuation, TPL-style; not await
it.
For optimistic executions, it is assumed the CancellationToken
will cause the timed-out execution to clean up (if necessary) and then terminate by throwing for cancellation, per standard co-operative cancellation semantics.
This terminates the timed-out task and expresses that termination back to the caller. There is no separate, continuing, walked-away-from task as there was in the pessimistic case. The Task
parameter passed to onTimeout/onTimeoutAsync
is therefore intentionally always null
for optimistic timeout.
(Another way of understanding this is that in optimistic timeout, the time-governed work is executed on the caller's codepath and thus expresses the full detail of its termination directly back to the caller. If the cancellation of the timed-out work was additionally expressed to the onTimeout/onTimeoutAsync
delegate this would cause it to be expressed in two places, leading to ambiguity about where to process or handle its termination.)
For a good discussion on walking away from executions you cannot cancel (pessimistic timeout), see Stephen Toub on How do I cancel non-cancelable async operations?.
Every action which could block a thread, or block waiting for a resource or response, should have a timeout. [Michael Nygard: Release It!].
Do not use TimeoutStrategy.Pessimistic
with calls through HttpClient
! All HttpClient
calls exist in versions taking a CancellationToken
, so co-operative timeout with TimeoutStrategy.Optimistic
is possible.
- For a timeout-per-try, place a
TimeoutPolicy
inside aRetryPolicy
withPolicyWrap
- For a timeout applying to an operation overall, including any retries (eg: up to N tries, but if the whole operation takes longer than one minute, time out), place a
TimeoutPolicy
outside aRetryPolicy
withPolicyWrap
.
More in ordering policies with PolicyWrap can be found here.
For the specific case of applying an overall timeout to all tries where a retry policy is applied as a DelegatingHandler
within HttpClient
(perhaps configured through HttpClientFactory
), note that the HttpClient.Timeout
property can/will also provide this overall timeout: see our HttpClientFactory
doco for more detail.
The operation of TimeoutPolicy
is thread-safe: multiple calls may safely be placed concurrently through a policy instance.
TimeoutPolicy
instances may be re-used across multiple call sites.
When reusing policies, use an OperationKey
to distinguish different call-site usages within logging and metrics.