idempotency - BaryoDev/barakoCMS GitHub Wiki

Inbound idempotency

POST, PUT and PATCH on the authoring API (/api/contents and the rest of the bearer-token surface) honour an Idempotency-Key header. Send one and a retried request will not be applied twice, and the retry gets the first request's response back. This is a different thing from the Idempotency-Key header a Webhook workflow action sends on outbound deliveries (docs/webhooks.md); that one is generated by barakoCMS and identifies a retry of our own delivery. This page is about the key a client sends us.

No other verb reads the header. A GET or DELETE sent with Idempotency-Key set is not deduplicated; the header is ignored.

Sending a key

Set Idempotency-Key to a value your client generates once per logical write and reuses only when retrying that same write:

POST /api/contents
Idempotency-Key: 6c1f6b2a-6e0a-4b1e-9f0a-6b1a2f6c1f6b

A GUID or ULID generated client side per operation is a good key. A deterministic key, such as an order id, also works, as long as one key stands for one write within the key's lifetime (below). A constant is a bad key: the second unrelated write that uses it is refused as a different request.

The key is scoped to the caller: it is namespaced internally by tenant and by caller. The caller is the API key when the request authenticated with one, otherwise the user, otherwise an anonymous bucket shared by every unauthenticated caller. So two users, a user and an API key acting for that user, or the same user in two tenants, can use the identical raw key without colliding, and a stored response is never replayed to anyone but the caller that made the first request. There is no length limit or format check: any non-empty header value is accepted and treated as a key. A missing header, or one that is empty or whitespace, is not an error either; the request just runs without idempotency protection, every time.

What happens on a retry

What a request with a key already in use gets depends on the first request:

First request	The retry gets
Succeeded (2xx), same method, path, query string and body	The first response replayed
Succeeded, but a different method, path, query string or body	`422`
Succeeded, response over the size limit	`409`
Succeeded, on a route that never replays (below)	`409`
Succeeded, from an unauthenticated caller, same request	`409`
Still running	`409`
Failed (any status outside 2xx)	Runs as a new request

A replay carries the first response's status code, Content-Type, Location (if it had one) and body, plus a header saying it is a replay:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Idempotent-Replayed: true

{"id":"9a1f...","version":1}

The handler does not run again, so nothing is written twice and no workflow fires twice. The query string and body are compared by an HMAC-SHA256 taken over the exact bytes sent, so a retry has to send the same bytes, not just equivalent JSON. On a route that never replays only the method and path are compared.

A retry that matches the key but not the request is refused with:

HTTP/1.1 422 Unprocessable Entity

This Idempotency-Key was already used for a different request.

Only responses up to Idempotency:MaxStoredResponseBytes (default 65536, at most 1048576) are kept. A bigger one still counts as done, so the write is not repeated, but a retry is answered:

HTTP/1.1 409 Conflict

Request with this Idempotency-Key already processed. Its response was too large to keep, so it cannot be replayed.

Set it to 0 to keep no response bodies at all, which makes every completed retry a 409.

Routes that never replay

A route whose response carries a credential, or whose request carries a password, never stores its response or a hash of its request, so a retry is answered with the 409 below and nothing is replayed. Those secrets are kept only as hashes everywhere else, and a stored copy would be a second one. The routes are:

POST /api/auth/login, POST /api/auth/refresh, POST /api/auth/otp/verify, POST /api/auth/mfa/verify and POST /api/me/switch (access and refresh tokens)
POST /api/auth/mfa/setup (the TOTP secret) and POST /api/auth/mfa/enable (recovery codes)
POST /api/api-keys (the raw key)
POST /api/preview (a preview token)
POST /api/site/share-links and POST /api/contents/{id}/share-links (the link key)
POST /api/auth/register, POST /api/me/password and POST /api/users/{userId}/password (a password in the request)
POST /api/public/site/share-links/open, which answers with what a link shows now, so a link revoked since is not opened again from a stored copy

HTTP/1.1 409 Conflict

Request with this Idempotency-Key already processed.

The key still stops the request running twice: a retried POST /api/api-keys does not mint a second key. A module endpoint opts in the same way, with [NoIdempotentReplay] on its class. IdempotencyReplayRulesTests fails when an endpoint's response type carries a token, key, secret, password or recovery codes, or its request type a password, without it. A route with no typed response has to be marked by hand.

Unauthenticated callers

Every unauthenticated caller shares one key bucket, so a stored response there could reach a stranger who sent the same key. They get deduplication only: the first request runs, a retry of the same request is the 409 above, and no response is stored. The request is still compared, so a different request under the same key is the 422, and somebody who learns a key and uses it first cannot make the real request read as already done. A key that failed is released for them the same as for anyone else.

Keys from before 4.7.0

A key that completed before 4.7.0 has no stored response, and a retry of it is answered 409 with Request with this Idempotency-Key already processed. until it expires.

How long a key is remembered

A key is honoured for Idempotency:KeyHours hours (default 24, between 1 and 720), counted from when the request claimed it. After that the key is free: the same key runs as a new request, whether or not its record has been deleted yet. A sweep runs every hour, starting two minutes after startup, and deletes expired records in batches of 500 for up to 30 seconds, so the table holds about one window of keyed writes. When it runs out of time with expired records left, it logs a warning with the count and runs again a minute later instead of an hour. A value outside 1 to 720 stops the API at startup with a message naming the setting.

A key claimed by a request that then failed (validation error, thrown exception, or any response outside 2xx) is released as part of handling that failure, so the very next request with the same key runs as a fresh attempt.

The one exception is a request that claims a key and then the process dies before it can succeed or fail, an in-progress claim with nothing to release it. That claim is treated as orphaned, and may be reclaimed, once it is more than 10 minutes old. Until then a retry gets:

HTTP/1.1 409 Conflict

A request with this Idempotency-Key is still in progress.

Two requests racing on the same key

Send the same key on two requests at once and exactly one of them proceeds. The first to insert the key wins; the database's unique constraint on the key rejects the second insert, and that request gets the "still in progress" 409 above. Neither request is queued behind the other or made to wait for the first to finish. Retry after the first one answers and the retry gets its response.

Two retries that both find an expired or orphaned record race the same way. Replacing the old record deletes it only while it is still expired or orphaned, in the same transaction as the new claim, so the second retry finds the first one's fresh claim and gets the 409. The sweep deletes a record only while it is still past the window, so it never removes a claim made after it listed the key.

Each claim carries its own id, and a request completes or releases the record only while it still holds that id. A request that ran past the 10 minute orphan window and lost its claim to a retry leaves the retry's claim alone when it finishes.

What is stored

Each record holds the scoped key, a claim id, when it was claimed, the method and the path. Except on a route that never replays it also holds an HMAC-SHA256 of the query string and body. On a 2xx from an authenticated caller on a route that replays it holds the status code, Content-Type, Location and the response body. The HMAC key and the body's encryption key are both derived for this use alone from Secrets:Key (falling back to JWT:Key), so neither is the key any other secret uses, and the body is sealed with the record's scoped key as associated data: a value copied in from another record or another secret does not open, and the retry gets the 409 above. After the key material is rotated, a retry within the window no longer matches its stored hash, so it gets the 422 for a different request rather than a replay, and nothing runs twice. The response is one the caller was already sent, and only that caller can have it replayed. Request and response bodies are never logged. Records live in mt_doc_idempotency_records and are removed by the sweep above.

Where this is enforced

IdempotencyFilter (barakoCMS/Infrastructure/Filters/IdempotencyFilter.cs) claims the key, or answers from the stored record, before the handler runs. IdempotencyFinalizer (barakoCMS/Infrastructure/Filters/IdempotencyFinalizer.cs) completes it with the response or releases it afterwards. IdempotencyRequestBuffering makes a keyed request's body readable a second time for the hash, and IdempotencyRetentionService runs the sweep. The filter and finalizer are registered globally, so the behaviour above applies to every POST, PUT and PATCH on the authoring API, not only content creation.

Generated from docs/idempotency.md by scripts/wiki-sync.sh. Edit the doc in the repository, not this page.