Handover 169 JWT - noetl/ai-meta GitHub Wiki

Handover โ€” Finish #169 (Auth0 JWT signature verification)

Focused handover for taking Auth0 login JWT signature verification from its current shadow state to enforce, safely, and closing noetl/ai-meta#169.

Repo copies (source of truth): docs/HANDOVER-169-JWT.md (full reference) + docs/CODEX-169-HANDOFF-PROMPT.md (copy-paste Codex kickoff).

The one danger: prod runs a single server replica โ†’ no partial-replica canary. shadow verifies-and-meters but always allows. The moment you set enforce, a wrong issuer/audience/JWKS config rejects 100% of logins. Every enforce step is gated on a real success observation and paired with an instant one-variable rollback (NOETL_AUTH_VERIFY_SIGNATURE=shadow).

Where things stand (2026-07-04)

Leg PR State
Server sync-path RS256/JWKS verify server#276 Merged โ†’ v3.52.0 โ†’ deployed to prod, running in shadow
Drive-path (Python-stdlib) mirror e2e#84 OPEN / unmerged (same flag, default off; only runs when NOETL_AUTH_SYNC off)

Prod: server-rust:v3.52.0, single replica, NOETL_AUTH_VERIFY_SIGNATURE=shadow, NOETL_AUTH0_DOMAIN=mestumre-development.us.auth0.com (bare host), NOETL_AUTH0_AUDIENCE unset. JWKS reachable from the pod (2 RS256 keys). outcome="success" not yet captured โ€” the verify metric is lazily registered and appears only after the first real claims-valid login hits shadow.

The finish sequence

  1. Capture shadow success โ€” trigger one real Muno login, read noetl_auth_jwt_verify_total{mode="shadow"}, require success only. Any bad_* / no_domain / jwks_unavailable = config mismatch to fix first.
  2. Set audience โ€” confirm the real aud (= SPA client_id, never printed) and set NOETL_AUTH0_AUDIENCE so enforce validates aud.
  3. Merge e2e#84 โ€” both paths verify (flag-gated; prod path is the sync one).
  4. Flip to enforce โ€” pre-checks pass โ†’ NOETL_AUTH_VERIFY_SIGNATURE=enforce โ†’ immediately verify a real login + latency โ†’ instant rollback to shadow at the first bad_*.
  5. Close #169 once both paths verify, enforce is live + stable, and the metric shows success.

Flag tri-state (NOETL_AUTH_VERIFY_SIGNATURE)

  • unset / off โ€” claims-only decode (pre-#169, byte-identical).
  • shadow โ€” verify + meter + log, still allow. โ† prod is here.
  • enforce โ€” verify and reject on failure. โ† target.

The bare-host gotcha

verify_signature builds the issuer via format!("https://{domain}/"), so NOETL_AUTH0_DOMAIN must be the bare host โ€” a URL value would produce https://https://โ€ฆ// and break every verify (a full login outage under enforce).

Exact prod Auth0 config to match

  • Issuer: https://mestumre-development.us.auth0.com/
  • JWKS: https://mestumre-development.us.auth0.com/.well-known/jwks.json
  • Audience: the SPA client_id in GSM auth0_client.data.client_id (project noetl-demo-19700101) โ€” value never printed.

Related