Continuous integration

How the CI GitHub Actions workflow validates every change before production Deploy can run.

Workflow file	.github/workflows/ci.yml
Workflow name	CI
Runs on	Every pull request, and every push to main
Job	Single job validate on ubuntu-latest
Permissions	contents: read only (no AWS, no OIDC write)
Concurrency	Group ci-${{ github.ref }}; cancels in-progress runs on PRs only (main pushes are not cancelled mid-flight)

Companion pages: Continuous deployment, Build issues, Deployment issues.

Why CI exists

CI is the quality and security gate for the monorepo: Python data jobs, browser JS, Terraform, the arm64 Lambda image, and GitHub Actions / Dockerfile checks. A failed CI run never deploys. Production Deploy only starts after a successful CI on a push to main (or a manual Deploy from main — see Continuous deployment).

Triggers in detail

on:
  pull_request:
  push:
    branches: [main]

• Pull request — validates the merge result of the PR branch. Concurrent PR runs for the same ref cancel older ones so you only wait on the latest commit.
• Push to main — re-validates what landed. This is the signal Deploy watches via workflow_run (Deploy does not run on PR CI success alone).

There is no workflow_dispatch on CI; you push or open a PR to run it.

Tooling the job installs

Third-party actions are pinned to full commit SHAs (version tags only in comments), which reduces supply-chain drift.

Step	Tool	Version / notes
Checkout	actions/checkout	Credentials not persisted (persist-credentials: false)
Python	actions/setup-python	3.12, pip cache from services/data-jobs/requirements-jobs.txt
Node	actions/setup-node	22 (syntax-check only for the static site)
Terraform	hashicorp/setup-terraform	CLI 1.16.5
Docker	QEMU + Buildx	arm64 (matches Lambda Graviton / linux/arm64 image)

Python packages installed for the job: deps from requirements-jobs.txt + requirements-test.txt, plus ruff and pip-audit.

What validate runs (in order)

1. Lint Python

working-directory: services/data-jobs
ruff check src tests scripts

Style and common-bug checks for collectors, jobs app code, tests, and helper scripts.

2. Test Python

pytest -q

Unit / fixture tests under services/data-jobs (quiet mode). Failures block the green check.

3. Audit Python dependencies

pip-audit -r services/data-jobs/requirements-jobs.txt
pip-audit -r services/data-jobs/requirements-test.txt

Known vulnerability scan on runtime and test requirement locks. A finding fails the job until deps are upgraded or explicitly addressed.

4. Check browser JavaScript syntax

node --check apps/web/app.js

No frontend bundler: the site is static HTML/CSS/JS. This only parses apps/web/app.js for syntax errors (no browser runtime tests in CI today).

5. Format and validate Terraform

working-directory: infra/terraform
terraform fmt -check -recursive
terraform init -backend=false
terraform validate

• fmt -check — fails if .tf files are not formatted (no auto-rewrite in CI).
• init -backend=false — downloads providers / modules without talking to the S3 state backend (no AWS creds on CI).
• validate — configuration graph is valid. This is not a terraform plan against live state.

6. Build the Lambda image (arm64)

docker buildx build --file infra/docker/Dockerfile --platform linux/arm64 --load -t investor-dashboard-jobs:ci .

Proves the jobs image builds for the architecture Deploy will push to ECR. CI loads the image locally (--load); it does not push to ECR. Publish happens only in Deploy.

7. Checkov security scan

bridgecrewio/checkov-action
  directory: .
  framework: terraform,dockerfile,github_actions
  quiet: true

Static analysis for risky Terraform, Dockerfile, and Actions patterns. Quiet mode keeps logs shorter; findings still fail the job.

What CI deliberately does not do

Not in CI	Where it happens instead
Assume AWS roles / OIDC	Continuous deployment only
terraform plan / apply against real state	Deploy (and rare admin laptop applies)
Push images to ECR	Deploy
Sync the static site or invalidate CloudFront	Deploy
Store or read provider API keys	SSM via rotate-key.sh (Data collectors API keys)

Interpreting results

Outcome	Meaning
All green on a PR	Safe to review/merge from a tooling perspective; still read the diff
Green on main push	Unlocks automatic Deploy if the production environment is approved
Red	Fix before merge; Deploy will not auto-start from a failed main CI

Common failure themes and fixes: Build issues.

Local parity (optional)

Rough local equivalents before you push:

cd services/data-jobs
ruff check src tests scripts
pytest -q
cd ../../infra/terraform
terraform fmt -check -recursive
terraform init -backend=false && terraform validate
# optional: docker buildx build --file infra/docker/Dockerfile --platform linux/arm64 -t investor-dashboard-jobs:local .

You still need the GitHub Actions Checkov + pip-audit steps (or run those tools locally) for full parity.