ADR 014 Sync Out - digitalspace/eagle-demi GitHub Wiki

ADR-014: Sync-out to Eagle

Status

Accepted 2026-10-08. The product owner made the decision. Sync-out and the ENGAGE ingest are in the eagle-demi repository; this page does not record which environment runs them.

Context

ENGAGE (met-api) creates an engagement for a project and pushes it to eagle-api as a comment period. The push goes to Eagle only. DEMI sees the period when eagle-api mirrors it (Sync-Architecture). A failed ENGAGE push leaves no trace on either side: the engagement exists in ENGAGE, the period does not exist in Eagle, and nothing reports the gap.

eagle-public's Angular line still reads comment periods from eagle-api, so Eagle has to hold them until that line retires. DEMI serves the React line and already holds a copy of every period.

Decision

Engagements flow ENGAGE → DEMI → Eagle. DEMI is the source of truth for them.

  • Ingest. ENGAGE calls PUT /api/engage/engagements/{engagementId}. The route is specific to ENGAGE. There is no generic ingest route. See Connecting-an-Application-to-DEMI.
  • Outbox. Each stored row is queued on an Azure Storage queue. The worker re-reads the row when it runs, so a late or repeated message sends the current state.
  • Consumer registry. src/sync-out/index.js lists consumers, each with name, enabled(), wants(row) and send(row). Eagle is the first. Removing it means deleting src/sync-out/eagle.js, one line in the registry, the syncOut.eagle settings and their bicep app settings, and the Eagle half of reconcile-engage.js.
  • Eagle writes go through eagle-api's HTTP API. DEMI signs in as the Keycloak client demi-sync-out. Nothing writes Eagle Mongo directly.
  • Ownership by sourceSystem. A row with sourceSystem: 'engage' belongs to ENGAGE. The Eagle mirror writes only the Eagle-owned fields onto it (ENGAGE-Rows-Reference).
  • Echo. eagle-api pushes every period it writes back to DEMI. The mirror keeps that push to the Eagle-owned fields and queues nothing, so the loop ends after one hop.
  • Reconcile. reconcile-engage.js compares ENGAGE rows with Eagle's copy and with ENGAGE's public list. --repair queues drifted rows again, and DEMI's copy wins.

Consequences

  • A failed Eagle write is retried, then the worker moves it to a poison queue and logs the line the alert reads. A failed ENGAGE push gets an HTTP error that ENGAGE can log.
  • A second system that needs these rows is one more consumer in the registry.
  • staff is a bypass role in eagle-api (JWT_BYPASS_ROLES in api/helpers/auth.js). The demi-sync-out secret is therefore broad write access to Eagle and lives only in Key Vault.
  • A staff edit in eagle-admin to an ENGAGE-owned period is overwritten by the next send or --repair.
  • When the Angular line retires, the Eagle consumer is deleted and nothing else changes.
  • The cutover and rollback order is in Sync-Architecture.

Alternatives rejected

  • A bridge from DEMI to Eagle outside DEMI. It needs its own copy of the row rules and its own credentials to DEMI, and failures sit in a third place.
  • Cosmos change feed as the trigger. The feed also fires on ACL cascades and on the echo from Eagle, and it needs a lease container. Storage queues already have a poison queue and an alert.
  • A generic /ingest route. Each source has its own ownership rules and field names. A shared route needs a per-source switch inside it.
  • Writing Eagle Mongo from DEMI. eagle-api stays the one writer of its database; its controllers set read, isPublished and the milestone.

See also