What is deployed and how it behaves. Where another page disagrees with the live system, the live system wins. Deploying and verifying it is Operating-DEMI-in-Azure; probing it is Operating-DEMI-Data-Plane.

The API is Flex Consumption, so az functionapp is the right command family in both environments — no app anywhere in this estate takes az webapp. Flex has no Kudu/SCM site and no SSH: nothing can reach it that way, and dev access to Cosmos/Search goes through the devbox VM instead.

There is no VNet in the resource group, and there must not be — the landing zone owns one. Subnet ids are parameters pointing into the platform spoke.

The real subnets are the platform vWAN spokes, one per environment:

test: /subscriptions/…/resourceGroups/c4b0a8-test-networking/providers/Microsoft.Network/
        virtualNetworks/c4b0a8-test-vwan-spoke/subnets/c4b0a8-test-cond-ext-pe-subnet
prod: /subscriptions/…/resourceGroups/c4b0a8-prod-networking/providers/Microsoft.Network/
        virtualNetworks/c4b0a8-prod-vwan-spoke/subnets/c4b0a8-prod-cond-ext-pe-subnet

Pass one as privateEndpointSubnetId; the Flex app takes its own delegated subnet, snet-demi-func-fc1-<env>, as apiFlexSubnetId. The landing zone also carries a sibling subnet, snet-app-service, in each spoke — nothing in this repo references it. Do not deploy main.bicep expecting it to build networking, and do not expect it to be how live app settings change.

Private DNS is attached by Azure Policy, not by this repo. Each endpoint carries a zone group named deployedByPolicy whose zone lives in a different subscription. Rolling our own would fail on virtualNetworkLinks — the VNet is in the networking resource group, which this identity cannot even list — and would create a second zone competing with the platform's. Create the endpoint only, and let policy wire DNS. See BC-Gov-Azure-Landing-Zone.

Hub BGP routes swallow the OpenShift API's public range. The spoke VNet learns routes from the landing-zone hub over BGP, and the hub advertises all of BC Gov's public range, 142.34.0.0/16. Any subnet with no route table sends that whole range to the hub, so traffic to api.silver.devops.gov.bc.ca (142.34.194.119) gets dropped instead of going out to the internet. Policy forbids creating route tables, so the fix is the platform-provided one, openshift-public-endpoint (in the networking resource group): it disables BGP route propagation and adds two /32 routes to Internet for the OpenShift API. Attach it to a subnet that needs to reach OpenShift's API:

az network vnet subnet update -g c4b0a8-<env>-networking --vnet-name c4b0a8-<env>-vwan-spoke \
  -n <subnet> --route-table openshift-public-endpoint

snet-servers (the devbox) already carries it. snet-demi-func-fc1-test got it 2026-09-11 for demi-secret-sync-test; prod has no route table and policy blocks creating one, so there is no prod sync app — prod OpenShift secrets stay in OpenShift. demi-api-fc-<env> and analytics-api-fc-<env> share the subnet; disabling BGP propagation there has had no observed effect on them.

No ENABLE_ORYX_BUILD or WEBSITE_RUN_FROM_PACKAGE: Flex takes its runtime from functionAppConfig.runtime and rejects those, unlike the App Service settings a legacy app needs.

demi-api-fc-prod carries the same names with prod values. Three differ by design: CORS_ORIGIN and ENRICHMENT_SOURCES are empty, and SUMMARY_ENABLED is false.

appSettings is a whole-collection PUT, so a template run that omits a setting deletes it from the running app. Every credential setting is now a Key Vault reference the Bicep composes from a secret name (@Microsoft.KeyVault(SecretUri=...)), never a value, so a template run cannot blank one by omission. scripts/deploy-infra.sh checks, before deploying, that demi-kv-<env> holds every name the app will resolve — see Secret-Sync.

demi-apim-<env> sits in front of the Flex app demi-api-fc-<env>, Consumption tier, Bicep module azure/modules/apim.bicep, gated by the deployApim param (true in both main.test.bicepparam and main.prod.bicepparam). It buys metering and a rate limit, not a security boundary: Consumption has no VNet, so demi-api-fc-<env>.azurewebsites.net stays publicly reachable and demi-api still authenticates every caller itself.

Two APIs share that one backend, in both environments. demi-api at path /api is anonymous, for the browser. demi-machine at path /machine requires Ocp-Apim-Subscription-Key and sits on product machine, rate-limited 300 calls/60 s per subscription. Subscriptions eagle-api, extractor, admin exist on that product in both demi-apim-test and demi-apim-prod; their keys come from az apim subscription list-keys, never from this repo.

The gateway's global policy deletes any client-supplied X-Gateway-Secret and X-APIM-Subscription header, then sets its own before forwarding: the secret from the Key Vault named value gateway-secret (apim-gateway-secret in demi-kv-<env>), and the resolved subscription name. demi-api trusts the subscription header only when the secret matches app setting APIM_GATEWAY_SECRET (a Key Vault reference); the caller's identity then resolves through an apikeys row apim:<subscription-name> that carries no key material, minted with POST /api/admin/api-keys, id: "apim:<name>".

Only test has a DEMI frontend, and it reaches the gateway same-origin: its Front Door endpoint adds a /api, /api/* route to it (eagle-edge, gated by demiApimDeployed), so /api/config reports API_LOCATION empty. eagle-public's /demi-search route on eao-nginx proxies to the gateway in both environments (demi origin, https://demi-apim-test.azure-api.net in values-test.yaml, https://demi-apim-prod.azure-api.net in values-prod.yaml). The extraction host does not go through the gateway either: it posts chunks straight to the Function App host with its own key, because APIM Consumption caps a request at 30 s and NDJSON ingest can run longer. eagle-api's push into DEMI (api/helpers/demiPush.js) does go through the gateway. It calls the demi-machine API with the eagle-api subscription key in Ocp-Apim-Subscription-Key (env DEMI_APIM_KEY), so the machine product's 300 calls/60 s limit applies to every push. DEMI_API_BASE is https://demi-apim-prod.azure-api.net/machine in prod and https://demi-apim-test.azure-api.net/machine in test (eagle-api helm/eagle-api/values-*.yaml).

Front Door (eagle-edge, profile eagle-edge-<env>) terminates every connection one hop outside APIM, so without a check the address APIM asserts is Front Door's own egress address, shared by every visitor. Rule sets rulesbackend (the /demi-search/*, /demi-projects/* passthrough) and rulesapi (the DEMI frontend and admin /api/* routes) each carry a rule named stampedgesecret that overwrites the request header X-Edge-Secret with a shared value before forwarding, dropping whatever a client sent. demi-api trusts the header only when it matches app setting EDGE_SECRET (constant-time compare, same helper as the gateway secret above); a match lets it read X-Azure-SocketIP — the peer Front Door itself accepted the connection from — as the caller's address. X-Azure-ClientIP is never read: Front Door derives it from the caller's own X-Forwarded-For, so any browser can pick it.

The value is one shared secret per environment. On the DEMI side it is Key Vault secret edge-secret, set by hand from the environment's devbox and read by demi-api-fc-<env> as a reference — see Secret-Sync. Test names edge-secret in optionalSecretNames; prod does not yet, so prod Front Door callers still share the anonymous rate-limit key rather than resolving X-Azure-SocketIP. eagle-edge has no Key Vault — its scripts/deploy-infra.sh reads EDGE_SECRET straight from the environment (falling back to the OpenShift secret demi-app-secrets in 6cdc9e-test only; prod requires it exported) and passes the value directly into the Front Door rule. Its GitHub Actions deploy reads the same key from GitHub environment secrets EDGE_SECRET (test, prod). Rotate both sides together, since a mismatch just falls back to the APIM path below. An empty value is the off switch: demi-api ignores the header, and on eagle-edge an empty value leaves an already-deployed stampedgesecret rule in place with its old value, because the deploy is Incremental — removing the header takes an explicit az afd rule delete.

TRUSTED_PROXY_IPS is still empty in prod, so this path changes nothing there yet: see Bulk-Download Access rules for the full derivation.

src/utils/caller-ip.js (bulk-download quota key, request log, audit trail) walks X-Forwarded-For from the rightmost hop, never takes the first entry. A caller can prepend any value it wants to that header, so the first hop is spoofable and never trustworthy as an identity key; only the hop each proxy in the chain itself appended is trustworthy, and each proxy appends to the end. See Bulk-Download Access rules for the full derivation, including the TRUSTED_PROXY_IPS case where the OpenShift rproxy in front of eagle-public is itself skipped.

Neither is a bug and neither has a fix on the current SKUs.

• The $web endpoint stays publicly reachable, so Front Door can be bypassed. A request straight to demiwebtestvymaysch2agdq.z9.web.core.windows.net gets the same bundle with none of the security headers. Three separate dead ends: AFD Standard cannot reach an origin over Private Link (Premium only, about ten times the price); storage networkAcls accepts resource-instance rules but not for Microsoft.Cdn/profiles; and storage cannot inspect the X-Azure-FDID header, which is how an App Service origin would prove the request came through the profile. Accepted because the contents are a public React bundle — what a bypasser loses is their own browser's headers. Public access on these accounts is not a Deny-PublicPaaSEndpoints collision either: that assignment sets StoragePublicIpDenyEffect to Disabled.
• Nothing at the edge can authenticate. $web is anonymous by definition, and an AFD rule set rule can only rewrite and set headers — there is no action that challenges a request for credentials. DEMI's Keycloak login is unaffected, it is inside the app, but the shell is open to anyone with the hostname. The upgrade that does not change SKU is a WAF custom rule attached to the endpoint: Standard supports custom rules (match plus Block, for example on an IP CIDR), and it is only the managed rule sets that are Premium-only. Anything that actually prompts for a credential is an Entra app registration in front, not a Front Door feature.

Index, indexer and data-source definitions are owned by Search-and-Retrieval, including the api-version split. The operational traps:

• The indexer identity needs the control-plane Cosmos DB Account Reader Role on the Cosmos account. Data-plane access alone fails the indexer PUT with Unable to retrieve account endpoint. In prod that identity is eagle-search-identity-prod.
• The shared private link to Cosmos is created in Pending and must be approved on the Cosmos side (az cosmosdb private-endpoint-connection approve) or nothing indexes.
• apply-search-definitions.js treats an index name absent from the live service as a greenfield create, not an update. Check the live index list before running it against an environment for the first time.

• POST /admin/sync/nrpti 504s at 240 s on the App Service request timeout and keeps running server-side regardless. Use ?async=true and watch /db/stats records until stable.
• List reads are capped, so never answer a data-loss question from a list response; read the ids individually. Search-and-Retrieval owns the page-size rules.
• _id is mixed. EPIC imports carry real ObjectIds, DEMI-created rows use strings.
• The object store needs port 443, SSL, and a pinned region, or presign hangs about 135 s while the SDK does a bucket-region lookup. eagle-api reads its credentials from the OpenShift secret nr-object-store-credential in each namespace (6cdc9e-test, 6cdc9e-prod); DEMI reads Key Vault secrets minio-access-key / minio-secret-key in demi-kv-test / demi-kv-prod instead.
• /name is indexed on the NoSQL API precisely so ORDER BY works, so sorting belongs in the query rather than in memory.
• The OpenShift eagle-demi-api pod crash-loops on its own unpaginated /documents. Seed documents from eagle-api directly.
• The bulk-download queue trigger needs messageEncoding: "none" in host.json. The producer sends a bare job UUID as the message body; the default base64 encoding would hand the trigger a decoded-then-re-decoded string that is not the id. See Bulk-Download.
• s3Key has maxVis: 0 in the document catalog. Any read that is not systemAccess() gets it redacted empty, including an admin-level API caller. Sampling s3Key through the API proves nothing about what is actually stored — check Cosmos directly for that.
• A chunk-ingest retry loop is a Cosmos cost incident, not just a failed call. From 2026-09-16 to 2026-09-21 a backfill client retried the same 38,090-chunk document every ~12 minutes, holding demi-cosmos-test at 4-8M RU an hour, about 70 CAD a day. POST /documents/:id/chunks now caps a document at MAX_CHUNKS_PER_DOCUMENT chunks (413 over it) and refuses a document that has failed three times inside 24 hours (409, until the cause is fixed and the call retried with ?force=true) — see Extraction-Pipeline. The alert demi-cosmos-ru-<env> now watches Cosmos RU directly, and demi-chunk-ingest-failures-<env> watches the failure lines, both described in Azure-Environments.

Signing in to app.powerbi.com returns:

Your organisation doesn't currently allow its users to purchase Microsoft Fabric free. Contact your IT admin for more information.

Self-service purchase is disabled tenant-wide, so both Power BI Pro and Fabric free need a request through CITZ (My Service Centre, or the JSM portal, quoting c4b0a8).

This is a different gate from the landing-zone policy. Deny-Fabric-Capacity blocks the ARM capacity resource; the purchase restriction blocks licensing on the tenant side. Power BI Pro creates no Microsoft.Fabric/capacities resource, so the policy never applied to it.

Checking the licence list directly is also blocked: conditional access rejects the Azure CLI from an unmanaged device, so az rest against Microsoft Graph fails whatever scope is requested.

The consequence for a portal-based dashboard: every viewer needs portal access plus Log Analytics Reader on the workspace, which does not scale to EAO staff at large. Audit, usage and cost avoid that gate: demi-identity-<env> holds Log Analytics Reader on demi-audit-<env>/demi-logs-<env> and Cost Management Reader on the resource group, so demi-api answers /admin/audit, /admin/analytics and /admin/cost on its own credential — the admin panel's Audit log, Site analytics and Azure spend screens, no portal access needed. Azure-Environments, ADR-008-Audit-and-Usage-Analytics-Store.

• Operating-DEMI-in-Azure — deploying, restarting and verifying
• Operating-DEMI-Data-Plane — devbox access, probes, audit and cost reads
• Architecture — what the application does with all this
• Search-and-Retrieval — AI Search, including its private-endpoint traps
• Azure-Environments — subscriptions, estates, Bicep, cost
• BC-Gov-Azure-Landing-Zone — the platform rules behind most of the above