Connecting an Application to DEMI - digitalspace/eagle-demi GitHub Wiki
How another EAO application gets a credential for DEMI, and which one to ask for. The reasoning behind this is ADR-007-Service-to-Service-Credentials.
No credential values appear on this wiki. They live in app settings and OpenShift Secrets. This page is procedure only.
| You are | Use |
|---|---|
| An application that can hold a Keycloak client (Track, Submitβ¦) | A Keycloak service account. This is the standard |
| A machine that genuinely cannot hold a Keycloak client (the extraction host) | A registry API key |
| ENGAGE, pushing engagements | An APIM subscription; see Pushing engagements from ENGAGE |
| A person | Your own IDIR login. Nothing on this page applies |
If you are reaching for a registry key because obtaining a CSS client is slower, ask for the client anyway. The key path exists for callers Keycloak cannot serve, not for callers in a hurry.
An uncredentialed request to GET /api/projects is capped at 100 rows a page, and asking for more
is refused with a 400 rather than quietly truncated. Any verified credential raises the cap to 1000,
where a larger pageSize is clamped instead. Page past either cap with the x-continuation-token
header.
GET /api/documents pages with page and count=true instead (x-total-count carries the total).
Continuation tokens exist only on GET /api/documents/scan, which needs a credential. See
Search-Query-Construction "Native GET /documents".
Ask for the least that works. Most integrations read.
| Role | Grants |
|---|---|
demi-service-read |
Reads everything the ACL allows. Cannot write anything |
demi-service-write |
Reads and writes data. Cannot reach /admin/*
|
staff / demi-admin / sysadmin
|
Read and write, including delete, and /admin/*
|
compliance |
Reads records carrying compliance in their read[]
|
project:<id> |
Restricts the caller to that project. Combines with the above |
A read-only integration asking for sysadmin will be sent back. demi-service-read exists so that
"we only need to read" is expressible rather than aspirational.
-
Request a client through BC Gov CSS for realm
eao-epic, in each environment you need. Service accounts enabled. Note the client id β it becomes yourazp, which is what DEMI logs and what the allowlist matches. -
Ask the DEMI team to assign roles to the service account, from the table above.
-
Get a token with
client_credentials:POST https://<loginproxy-host>/auth/realms/eao-epic/protocol/openid-connect/token grant_type=client_credentials&client_id=<your-client>&client_secret=<secret> -
Cache it. Tokens are short-lived and a token request per API call is wasteful on both sides.
met-apicaches for 55s against a 60s lifetime β copy that shape. -
Call DEMI with
Authorization: Bearer <token>. -
Store the client secret in your platform's secret store β an OpenShift Secret on Silver, an app setting on Azure. Never in a repo, never in a Helm values file, never in a pipeline variable.
DEMI_ALLOWED_CLIENTS is the allowlist of permitted azp values. It is empty, which admits every
verified client; a client outside a non-empty list keeps its identity but has its privileges
stripped. See the ADR for why empty is the shipped default.
Only for callers that cannot hold a Keycloak client.
-
Ask the DEMI team. Someone with a privileged credential issues one:
POST /api/admin/api-keys { "name": "extraction-host", "roles": ["demi-service-read"] }Granting a write-capable role additionally requires
"allowWrite": truein the body, so a writer is never issued by accident. -
The plaintext key is in that response and nowhere else, ever. It is not stored and cannot be recovered. Losing it means being reissued, not looked up.
-
Send it as
X-Api-Key: demi_<env>_<keyId>_<secret>. -
Keys expire after 90 days by default, matching the landing zone's maximum secret lifetime. Diarise it. An expired key fails closed with a 401 that reads exactly like a wrong key.
Issue the new key, deploy it, confirm traffic has moved (lastUsedAt on the old key stops
advancing), then revoke the old one:
DELETE /api/admin/api-keys/<keyId>
There is no update endpoint on purpose β this order is what makes rotation possible with no window where you have no working credential.
Revocation is immediate on the instance that processed it and takes up to 60 seconds elsewhere, because verified lookups are cached. If a key is believed compromised, revoke it and treat that minute as real.
ENGAGE reaches DEMI through API Management, not Keycloak. azure/modules/apim.bicep creates one
subscription per machine consumer, eagle-api and engage, so each key rotates or revokes alone.
APIM asserts the caller as the registry principal apim:engage. The route is behind
authMiddleware, requireWrite and requireEngageWriter, which admits only:
- a registry identity whose id is in
DEMI_ENGAGE_PRINCIPALS(defaultapim:engage; set but empty refuses everyone); - that holds
demi-service-write; - with no project scope.
The registry row apim:engage therefore holds demi-service-write and carries no project scope. eagle-api's
principal gets 403 on this route, and apim:engage gets 403 on /api/eagle/*.
PUT /api/engage/engagements/{engagementId}:
{
"pushedAt": "2026-10-08T17:00:00Z",
"engagement": {
"id": 42, "projectId": "588511d0aaecd9001b825604", "trackingId": null,
"name": "...", "description": "...", "status": 2,
"start": "2026-10-10 07:00:00", "end": "2026-11-10 07:00:00",
"metURL": "...", "metURLAdmin": "...", "bannerUrl": "...",
"isPublished": true, "isDeleted": false
}
}engagement.id must equal engagementId in the path. projectId is the Eagle id of the parent
project. status is ENGAGE's status_id. start and end are ISO 8601 or
YYYY-MM-DD HH:MM:SS; a time with no zone is read as UTC, and any other text is refused. metURL
is required unless the engagement is unpublished or deleted.
trackingId is the Eagle comment period the engagement already has, if any;
the first push adopts that row. pushedAt is an ISO 8601 time, not the millisecond number the Eagle
push uses. A push older than the stored engagePushedAt is refused; an equal one writes again.
Field storage and ownership: ENGAGE-Rows-Reference.
| Status | code |
Meaning |
|---|---|---|
| 200 | An existing row was updated. | |
| 201 | A row was created. | |
| 400 |
ENGAGEMENT_ID_MISMATCH, PUSHED_AT_INVALID, TRACKING_ID_INVALID, PUSHED_DATES_INVALID, METURL_REQUIRED
|
The body is wrong. Retrying does not help. |
| 403 | The caller is not in DEMI_ENGAGE_PRINCIPALS. |
|
| 404 |
PARENT_NOT_FOUND, PARENT_REF_INVALID
|
The parent is in neither projects nor notifications. Retry only for PARENT_NOT_FOUND. |
| 409 | STALE_PUSH |
A newer push already landed. The body carries id and engagePushedAt. The only 409: treat it as already current. |
| 422 | TRACKING_ID_CLAIMED |
Another engagement owns that Eagle period. |
| 422 | PARENT_KIND_UNSUPPORTED |
The parent is a project notification. Send that engagement to Eagle directly. |
| 500 | DUPLICATE_ID |
The engagement is stored in more than one row. Nothing was written. |
| 500 | The row is stored and queued; the comments under it were not all re-derived. Or the write failed. | |
| 503 | The row changed while the push retried. Push it again. |
A 200 or 201 body is { id, engagementId, eagleId, engagePushedAt, queued }. eagleId is null until
sync-out creates the Eagle period. queued names the consumers the row was queued for; it is empty
when none is enabled or the enqueue failed (Sync-Architecture).
To rotate the key, read the subscription's keys with az apim subscription list-keys, give ENGAGE the
second key, then regenerate the first. The key lives in ENGAGE's secret store, never in a repo.
GET /config/public needs no credential β it is the one read on this list that is meant to be
called anonymously, by eagle-public at boot. It serves DEMI's live mirror of eagle-api's
/api/config, filtered to an allowlist of keys that are already public, and answers 503 rather
than a default payload when that mirror is missing. See ADR-012-Public-Config-Surface and
Seeding for how the mirror stays current.
Do not accept a probe that cannot fail. GET /api/projects returns data anonymously, so a 200
there proves nothing about your credential β it proves DEMI is up.
Use a discriminator instead: call an endpoint whose response differs by privilege, and compare against the same call with no credential. If the two agree, your credential is not being applied.
- A privileged read that anonymous cannot make:
GET /api/search/summary - A write, if you were granted one: it should succeed for a writer and return
403
Forbidden. This credential is read-only.fordemi-service-read
That 403 is the expected, correct answer for a read-only consumer. It is not a misconfiguration.
eagle-api pushes each project and document it writes into DEMI, fire and forget: a DEMI outage
never fails an Eagle write. All /eagle/* routes below take those pushes, behind authMiddleware,
requireWrite, and requireEagleMirror. requireEagleMirror is pinned to eagle-api: the caller
must be a registry identity (an APIM subscription or a presented registry key, never a Keycloak
user token) whose principal id is listed in the app setting DEMI_EAGLE_MIRROR_PRINCIPALS
(default apim:eagle-api, which is eagle-api arriving through API Management), hold
demi-service-write, and carry no project scope. Setting DEMI_EAGLE_MIRROR_PRINCIPALS to empty
refuses every caller rather than admitting every writer. Everyone else β staff users, the
break-glass ADMIN_API_KEY, the extractor key, any other demi-service-write key β gets 403.
eagle-api authenticates with a demi-service-write key, held in the OpenShift secret
demi-push-secret in 6cdc9e-prod.
PUT /api/eagle/projects/{eagleId} |
Mirror one project |
PUT /api/eagle/documents/{eagleId} |
Mirror one document |
PUT /api/eagle/commentperiods/{eagleId} |
Mirror one comment period |
PUT /api/eagle/comments/{eagleId} |
Mirror one comment |
PUT /api/eagle/organizations/{eagleId} |
Mirror one organization |
PUT /api/eagle/notifications/{eagleId} |
Mirror one project notification |
PUT /api/eagle/updates/{eagleId} |
Mirror one recent-activity item |
eagleId is the Eagle _id of the record. The body is:
{
"doc": { "_id": "588511d0aaecd9001b825604", "name": "...", "read": ["public"] },
"labels": { "type": "Application", "milestone": "Application Review" },
"pushedAt": 1757894400000
}doc is the raw Eagle record, exactly as eagle-api stores it β DEMI holds the merge and
transform rules, so a push and a re-seed produce the same row. doc._id must equal eagleId in
the path; a mismatch is a 400. labels applies to documents only: it carries the resolved List
names for the record's type, milestone, projectPhase and documentAuthorType ObjectId
references, and DEMI stores each label beside its id.
pushedAt is when eagle-api sent the push, in milliseconds since the epoch. DEMI stores it as
eaglePushedAt on the row and uses it to order pushes for the same record across eagle-api pods.
A push whose pushedAt is strictly older than the stored eaglePushedAt is refused: nothing is
written, nothing cascades, and the answer is 200 {"ok": true, "ignored": "stale", "eaglePushedAt": ...}.
An equal stamp writes β two pushes sharing a millisecond came from one pod, which ordered them
itself. pushedAt is optional; a push with none writes too and leaves the stored stamp as it was,
so an older client can still update a row a newer client had stamped. A staff caller writing
through the ordinary record routes can never set or clear eaglePushedAt (or the
cascadePendingAt marker below) β both are carried off the stored row on every write that is not
an eagle-api push.
A push that lands writes the row under the same etag guard as every other whole-item write in
DEMI: up to three tries, each rebuilt against whatever Cosmos actually has after the last one lost
its race. A push that loses all three gets 503 {"error": "The record is being written by another request. Push it again."} β push the same body again. Every /api/eagle/* route answers with
this shape on a lost race, and every route answers a landed push with
200 {"id": "...", "action": "upsert"} ("delete" for a delete push on documents and comment
periods). Pushing the same eagleId again replaces the record, so a retry after any answer other
than a lost etag race is safe.
A project push that lands but whose visibility cascade onto its documents fails answers 500, and
DEMI marks the project cascadePendingAt so the cascade is not lost. The next push for that
project that lands runs the cascade again and clears the marker.
A comment period, comment, organization or notification push whose required parent is not yet in
DEMI (a comment period's project, a comment's period) is refused with a 404: push the parent
first. The 404 body is { error, code }: code is PARENT_NOT_FOUND when no admissible parent
row exists (retry once the parent lands) and PARENT_REF_INVALID when the parent reference is empty
or not a 24-character hex id (retry cannot help). Details: Eagle-Mirror-Reference#refusal-codes.
List has no push and no route β Eagle has no write controller for it, so DEMI's only writer
of List rows is the backfill β see Public-Read-Backfill.
What DEMI decides, not the push:
-
Identity. A project matched to Track keeps the Track project id; an unmatched one is keyed
eagle-<eagleId>. A document keeps its Eagle_id. -
Visibility.
read[]comes from the pushed record, and a document's is intersected with its parent project's β a document can never out-rank its project.isPublishedmirrorsread[]. A project whose publication state changes cascades onto its documents. - Extraction state. Carried off the row already in DEMI, never from the push.
-
Enrichment. Source blocks DEMI adds itself, such as
sources.wildfire, survive a push.
A document push whose parent project is not in DEMI returns 404 with the same code field: push the project first.