Continuous deployment - ram290476/investor-dashboard GitHub Wiki

Continuous deployment

How the Deploy production GitHub Actions workflow publishes infrastructure, the jobs image, and the static dashboard to AWS.

Workflow file .github/workflows/deploy.yml
Workflow name Deploy production
Environment GitHub production (required reviewers + main only)
AWS role invdash-terraform-deploy via OIDC (secrets.AWS_DEPLOY_ROLE_ARN)
Region vars.AWS_REGION (production: us-west-1)
Timeout 55 minutes (OIDC session is 1 hour)
Concurrency Group investor-dashboard-production, does not cancel in-progress deploys

Companion pages: Continuous integration, Deployment issues, Build issues, Data collectors API keys.

End-to-end picture

push / merge to main
        │
        ▼
   CI workflow (validate)
        │ success + push event on main
        ▼
 Deploy production (workflow_run)
        │ GitHub environment approval (human)
        ▼
 OIDC → assume invdash-terraform-deploy
        │
        ├─ (bootstrap only) terraform apply with empty jobs_image_uri
        ├─ build/push arm64 image → ECR  tag = <sha>-<attempt>
        ├─ terraform apply with that image URI
        ├─ write apps/web/config.json from terraform output
        ├─ s3 sync static site + CloudFront invalidation
        └─ curl site URL + config.json smoke checks

API keys are never part of Deploy. They live in SSM and are set with rotate-key.sh (Data collectors API keys).

When Deploy runs

Automatic (workflow_run)

Deploy listens for CI completing, then the job if: requires all of:

  1. Event is workflow_run with conclusion success
  2. The CI run that finished was triggered by a push (not a PR-only CI)
  3. That push was on branch main
  4. The head repository is this repository (not a fork)

So: merge (or push) to main → CI green → Deploy is queued → human approves the production environment → job runs.

Manual (workflow_dispatch)

Allowed only when github.ref == refs/heads/main (dispatch from the default branch). Same environment gate and same steps. Checkout uses github.sha instead of workflow_run.head_sha.

What never starts Deploy

  • CI success on a pull request alone
  • CI failure or cancelled runs
  • Dispatch from a non-main branch
  • Fork workflow_run heads

Permissions model

Top-level permissions: {} (deny by default). The deploy job then grants only:

Permission Why
contents: read Checkout the commit being deployed
id-token: write Mint the GitHub OIDC token for AWS

No packages write, no pull-requests write, no long-lived AWS keys in the repo.

Production environment configuration

Set these on the GitHub production environment (not repository-level), documented in docs/operations.md:

Name Kind Purpose
AWS_REGION Variable Must match Terraform region (us-west-1)
AWS_DEPLOY_ROLE_ARN Secret arn:aws:iam::308639168050:role/invdash-terraform-deploy
TF_BACKEND_CONFIG Secret Full backend "s3" { ... } block (state bucket, key, region, encrypt, lockfile)
TERRAFORM_TFVARS Secret Full terraform.tfvars contents plus sec_user_agent; omit jobs_image_uri (workflow sets it)

Protect the environment with required reviewers and deployment branches main only. That human approval is the gate before terraform apply -auto-approve.

The first step Check deploy configuration fails fast with a clear error if any of the four are missing.

OIDC sign-in to AWS

aws-actions/configure-aws-credentials
  role-to-assume: secrets.AWS_DEPLOY_ROLE_ARN
  role-session-name: gha-deploy-<run_id>-<attempt>
  role-duration-seconds: 3600
  aws-region: vars.AWS_REGION

The IAM trust policy on invdash-terraform-deploy requires exact claim matches (aud, sub in both immutable-ID and legacy forms, repository_id, repository_owner_id, ref, job_workflow_ref). Details and troubleshooting: Deployment issues and docs/operations.md.

The role is roughly PowerUserAccess plus bounded project IAM, with an explicit deny of self-modification (CI cannot PutRolePolicy on itself / OIDC / boundary). Changes to that module need an administrator apply on a laptop.

Step-by-step job flow

1. Checkout

Checks out workflow_run.head_sha (automatic) or github.sha (manual), credentials not persisted.

2. Check deploy configuration

Verifies the four environment values exist (see table above).

3. Setup Terraform / Docker

Terraform 1.16.5 with terraform_wrapper: false so $(terraform output -raw …) is clean. QEMU + Buildx for linux/arm64.

4. Configure AWS credentials (OIDC)

Assumes the deploy role for up to one hour.

5. Prepare Terraform configuration

Writes backend.tf and terraform.tfvars from secrets (umask 077). Because the repo is public, every e-mail address found in tfvars is registered with ::add-mask:: so plan logs do not leak alert addresses.

6. Initialize and provision base infrastructure (bootstrap gate)

terraform init -input=false
# Snapshot state into an array — do NOT pipe state list into grep -q under pipefail

Logic:

State condition Action
module.data_lake.aws_ecr_repository.jobs exists Skip bootstrap; jobs already have an ECR home
Job Lambdas exist but ECR does not Refuse empty-URI apply (error)
Neither (true first bootstrap) terraform apply -auto-approve -var='jobs_image_uri=' once to create lake/ECR/base stack

Why this matters: module.jobs enables nothing when jobs_image_uri is empty. A later apply with an empty URI destroys every job Lambda, schedule, and related IAM. That bite Deploy run 37496345092; the array-based check and the refuse-if-empty apply later are the fix (Deployment issues, Build issues).

Exports ECR_REPOSITORY from terraform output -raw ecr_repository_url.

7. ECR login + build/push image

Tag format:

${ECR_REPOSITORY}:${DEPLOY_SHA}-${GITHUB_RUN_ATTEMPT}

DEPLOY_SHA is the commit being deployed; GITHUB_RUN_ATTEMPT makes re-runs unique. Build:

docker buildx build \
  --file infra/docker/Dockerfile \
  --platform linux/arm64 \
  --provenance=false \
  --push \
  --tag "$image_uri" \
  .

Sets JOBS_IMAGE_URI for the next step.

8. Deploy job functions and generate site configuration

# Refuse apply if JOBS_IMAGE_URI is empty (would destroy jobs)
terraform apply -input=false -auto-approve -var="jobs_image_uri=${JOBS_IMAGE_URI}"
terraform output -json site_runtime_config > ../../apps/web/config.json

Also exports SITE_BUCKET and CLOUDFRONT_DISTRIBUTION_ID from terraform output -json site.

This apply updates Lambdas to the new image, schedules/triggers as coded, Cognito/API/site modules, etc., according to state drift.

9. Publish static dashboard

aws s3 sync apps/web s3://${SITE_BUCKET} --delete --exclude config.example.json
aws cloudfront create-invalidation --distribution-id … --paths '/*'

--delete removes objects from the bucket that are no longer in apps/web. config.json written in step 8 is included in the sync (public Cognito/API identifiers only — never API keys).

10. Verify deployed site

curl --fail --retry … "${site_url}/"
curl … "${site_url}/config.json" | jq -e '.apiBaseUrl and .clientId and .callbackUrl'

Confirms the CloudFront origin serves the index and a well-formed runtime config. Retries absorb brief invalidation/propagation lag.

What Deploy does not cover

Gap Operator action
Provider API keys still REPLACE_ME rotate-key.sh (Data collectors API keys)
First Yahoo history / empty charts Explicit invdash-backfill invoke (Deployment issues)
Changes to deploy role trust / PassRole / boundary Admin SSO apply of module.github_deploy
Cognito user invite / password Console or CLI outside this workflow

Hard rules for operators

  1. Never approve a Deploy (or local apply) that passes an empty jobs_image_uri once jobs exist.
  2. Keep TERRAFORM_TFVARS free of secrets; API keys stay in SSM.
  3. Treat production environment approval as the last human gate before -auto-approve.
  4. If assume-role fails, fix trust with an admin apply — do not paste long-lived keys into Actions.

Related wiki

⚠️ **GitHub.com Fallback** ⚠️