Release Process - digitalspace/eagle-demi GitHub Wiki

Release Process

How a version number gets cut, deployed and rolled back. The workflows that do this are catalogued in CI-Workflows; deploying and verifying the Azure side is Operating-DEMI-in-Azure.

The tag is cut at test time, on every push to main

draft-release.yaml ("Tag and draft release") runs on every push to main (also workflow_dispatch, guarded to main at the job level so a manual run cannot tag an unmerged branch). It computes the next version, pushes an annotated git tag for it, and refreshes a single draft GitHub release to match. The two staging deploy workflows in CI-Deploy-Workflows deploy that same push to demi-api-fc-test and $web on their own path filters — together the three are the "test" half of the release model. azure-deploy-prod.yaml is the other half: it deploys one of these tags and, only on a fully green deploy, flips that tag's draft to published.

The tag exists before anything is deployed to production, on purpose. A draft release alone mints no git ref — GitHub creates the tag only when a human clicks Publish — so a build sitting on staging would otherwise have no name, and "deploy the tag you verified on staging" could not be obeyed until after a human had already published it. That forced publishing to happen before the prod deploy, so a published release meant "somebody intended to ship this" rather than "this is in production." Minting the tag at test time makes every staging build addressable the moment it lands, and leaves "published" free to mean exactly one thing: it is live.

What that costs: a version is spent on every push to main, including a docs-only change or a push whose staging deploy later fails. A git tag is a 40-byte ref, so the cost is one unused tag name, not a resource. In exchange, every staging build is referable after the fact by the tag it minted.

Computing the next version

scripts/next-version.js, run by draft-release.yaml, reads Conventional Commit messages between the last release tag and the pushed commit and picks the next vX.Y.Z:

  • A subject with a ! before the colon (feat(x)!: ...), or a BREAKING-CHANGE/BREAKING CHANGE footer anchored at the start of a line, bumps major.
  • A feat: subject with no breaking marker bumps minor.
  • Everything else — fix:, chore:, docs:, an unformatted commit, a merge commit — bumps patch.

Inputs come from the GitHub API (gh api repos/.../tags, .../compare/<lastTag>...<sha>), not from local git history: git describe walks only HEAD's ancestry, so a tag sitting on a commit that is not an ancestor of HEAD would silently resolve to an older base and produce a version lower than one already released.

The base is the highest existing vX.Y.Z tag, not the latest published release. Every push mints a tag, so a base computed from published releases only would recompute the same number on the next push and collide with the tag just created.

The compare endpoint caps its response at 250 commits and reports the true total separately (total_commits) without saying anywhere in its body that it truncated. next-version.js checks the returned commit count against that total and fails loudly if they disagree, rather than silently scoring a version bump from an incomplete commit list.

There is no "no previous release" fallback — the seed tag v0.1.0 is created once by hand, and the script's only recovery path if that seed is missing or renamed is to fail with the exact gh release create command to reseed it.

The draft release

Exactly one draft release exists at a time: the current candidate. Each run of draft-release.yaml deletes the previous draft — matched to drafts authored by github-actions[bot] whose tag matches the vX.Y.Z shape only, so a hand-authored draft, or one under a different tag name, survives — and creates a new one for the version it just tagged, with --generate-notes.

The notes cover a different range than the version bump: the bump is computed from the last tag (a version has to climb once per push), but --generate-notes infers its own start boundary, the last published release — the last version that actually reached production. A candidate that never ships still spent a version, and the release notes on the one that does ship need to include that unshipped work too. A hand-edit to a draft's notes is discarded on the next push, since the draft is deleted and recreated rather than updated in place.

Deploying to production

azure-deploy-prod.yaml ("Deploy DEMI to Azure production") is workflow_dispatch only, taking a version input shaped vX.Y.Z. There is no push: trigger and there must never be one — merging to main must never, by itself, ship to prod.

Prerequisites, none of which the workflow creates (from the workflow's own header, all created by hand once):

  • Resource group rg-demi-prod exists in Azure.
  • If the extractor Function App is wanted in prod, it is deployed once from azure/modules/extractor.bicep by hand — this workflow only ships code to it if it already exists; it never runs an ARM deployment.
  • UAMI demi-cicd-prod holds Website Contributor on demi-api-fc-prod (and, once the extractor exists, the same role on it) and nothing at resource-group scope, nothing on demi-search-prod, no Storage roles.
  • A federated credential named gh-env-prod on demi-cicd-prod, subject repo:digitalspace/eagle-demi:environment:prod, issuer https://token.actions.githubusercontent.com, audience api://AzureADTokenExchange. Declaring environment: prod in the workflow is what produces that subject claim; renaming the environment without a matching credential fails Azure Login with AADSTS700213.
  • That identity is kept separate from any runtime identity, because eagle-demi is a public repository: a federated credential on the runtime UAMI would let any workflow on main mint a token carrying Cosmos and AI Search data access.
  • On the prod GitHub environment: secrets AZURE_CLIENT_ID / AZURE_TENANT_ID, variables AZURE_SUBSCRIPTION_ID / AZURE_RESOURCE_GROUP (rg-demi-prod), and a deployment tag policy restricting dispatches to the pattern v* — see CI-Deploy-Workflows for the live settings.

Checklist before dispatching:

  • Run scripts/demi-devbox.sh drift --env prod; apply if it reports drift (Runbook-Search-Outage). The workflow's own verify-search-schema job checks the same thing against the tag being deployed and stops the release on a 503, but that gate only compares the tag to whatever is already live — running it by hand first is what catches drift before a human is waiting on a dispatch.
  • Two alerts, demi-search-failures-prod and demi-search-5xx-ratio-prod, page on the two log shapes a search outage takes; a third, demi-search-availability-prod, checks the public endpoint every five minutes. Confirm the action group they mail is being read before shipping — the alert firing does not help if nobody sees it.

Five jobs run in this order:

  1. verify-search-schema — probes the live index with the incoming tag's index definitions (/health/search-schema) and stops the release on a 503; see CI-Deploy-Workflows and Runbook-Search-Outage.
  2. deploy-extractor (waits on verify-search-schema) — checks out refs/tags/<version>, deploys the extractor if eagle-extractor-prod exists, otherwise skips.
  3. deploy-api (waits on deploy-extractor) — checks out the same tag, config-zips demi-api-fc-prod, then runs the post-deploy smoke test against it.
  4. rollback (waits on deploy-api, runs if deploy-api failed after the zip reached the app, including a failed smoke test) — redeploys the newest published release, re-smokes it, and leaves the run red.
  5. publish-release (waits on deploy-api, skipped if rollback ran) — flips the tag's release to published and --latest.

The deploy jobs are serial, not concurrent, even though they sit behind the same prod approval and could otherwise start together the instant it is granted. Ordering means the API — the thing that would be serving live traffic — deploys only once the extractor feeding it is up, and a failure anywhere in the chain stops the release rather than leaving half of it shipped.

Publishing is the last step, gated on deploy-api's smoke test passing and skipped entirely if rollback ran. Nothing publishes a release for code that is not actually serving in production.

Rollback

Two paths exist. The automatic one: deploy-api's post-deploy smoke test fails, and the rollback job redeploys the newest published release, re-runs the smoke test against it, and leaves the workflow run red with the just-deployed tag unpublished — no human action needed. Runbook-Search-Outage covers the manual trigger for the same recovery, for the case where the break is found later than the deploy that caused it.

The manual one: dispatch azure-deploy-prod.yaml again with an older tag. The publish-release job moves GitHub's "latest release" pointer to whichever tag was just deployed, including backwards, because the pointer answers "what is in production", not "what is the highest version number". If that older tag's own draft has already been deleted by a later draft-release.yaml run (the normal case — a run keeps only one draft, for the current candidate), publish-release creates a fresh published release for it with --generate-notes instead, so a rollback still leaves a release record behind rather than a gap in the history of what has been in production.

See also

⚠️ **GitHub.com Fallback** ⚠️