CI CD & infrastructure - mzogheib/quoll GitHub Wiki

Summary

  • GitHub Actions for CI/CD
  • Terraform for deploying AWS resources
    • Terraform Cloud for the backend
    • Source is in another private repo

Authenticating with AWS via Open ID Connect (OIDC)

  • Avoid long lived AWS keys stored in Terraform Cloud or GitHub
  • Instead, authenticate via an OIDC provider
  • Basic steps:
    1. Create the OIDC provider in IAM
    2. Create the role in IAM
    3. Add permission policies to the role, e.g. permission to create ECR repositories
    4. Add a trust policy to the role that allows the provider to assume the role
    5. Add the role's ARN to your provider, e.g. as an env variable or workflow input
  • Given that this is a one time setup it can be done via the AWS and Terraform UIs
  • Guides

Example trust policies

  • Replace all the <...> with your values
  • The condition values can be wildcards * to allow all, e.g. run_phase:*

Terraform

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT-ID>:oidc-provider/app.terraform.io"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "app.terraform.io:aud": "aws.workload.identity"
        },
        "StringLike": {
          "app.terraform.io:sub": "organization:<ORGANIZATION>:project:<PROJECT>:workspace:<WORKSPACE>:run_phase:<RUN-PHASE>"
        }
      }
    }
  ]
}

GitHub

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT-ID>:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:<ORGANIZATION>/<REPO>:<BRANCH>"
        }
      }
    }
  ]
}

Pipeline steps

⚠️ WIP

  1. Push to master
  2. Workflow runs to raise a release PR
  3. Workflow runs to run all tests
  4. If tests pass, can merge the release PR
  5. Workflow runs to create new application and packages and publish applicable packages
  6. If new application tags created, workflow runs to build and push Docker images
  7. If new images created, workflow runs to deploy to cloud

api

Tag trigger: @quoll/api@*

  1. Build Docker image
  2. Push to Docker repository

Notes for build and push Docker image

Run the latest image

  1. Log in to EC2 instance via EC2 Instance Connect
  2. Create/update the .env, docker-compose.yaml files
  3. Get the ECR credentials (it has an IAM role that has the permission to do this)
    aws ecr get-login-password --region <REGION> | sudo docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com
  4. Stop the existing image if running
    sudo docker-compose down
  5. Run the latest image
    sudo docker-compose up -d

Notes

  • Can't use the default GITHUB_TOKEN if the intention is to trigger one workflow as a side effect of another, e.g. workflow A creates a git tag then workflow B is triggered on the creation of that tag.
    • See here. It's intentional.
    • Workaround is to use a personal access token.
⚠️ **GitHub.com Fallback** ⚠️