Deploying VRO - department-of-veterans-affairs/abd-vro GitHub Wiki

VRO is deployed to the LHDI platform. To deploy to LHDI, Docker images for deployment must exist in the GHCR (GitHub Container Registry).

As mentioned in Uses of container images, a GH Action pushes packages to GHCR so that the packages can be retrieved by EKS in the LHDI platform, initiated by calls to helm and kubectl.

NOTE: The processes described here should only be done by VRO Engineers

Unless otherwise specified, all deployments should be made from changes on the develop branch

Deploy to envs that don't require SecRel

  1. Ensure images have been published for the commit you want to deploy by checking the automated run of the SecRel workflow on the internal repo.
  2. Update deployments for the commit and images you want.

Deploying to envs requiring SecRel

NOTE: These steps can only be performed on the current head of the develop branch

  1. Ensure images have been published for the commit you want to deploy by checking the automated run of the SecRel workflow on the internal repo.
  2. Verify and Fix SecRel if necessary and verify that image signatures have been performed.
  3. Update deployments for the commit and images you want.

Deploy using a custom branch

We allow deployments based on a custom branch to the dev and qa environments only in most cases.

  1. Post in #benefits-vro-engineering channel with an @vro-eng that you will be doing a deployment and list the images you will be deploying to make sure there are no conflicts with others. Ideally, wait at least 30 minutes before continuing with a deployment to give others time to object.
  2. In the internal repo, run the secrel workflow using the workflow from your custom branch to publish any new container images.
  3. Update deployments for the commit and images you want.

Update Deployments

  • For the VRO Platform Team, click the "Run workflow" here for one of the "Update Deployment" GitHub Action workflows:

    Please use the following "Update Deployment" workflows based on the env you are deploying:

    • Update Deployment is used to deploy various Helm charts, including api-gateway, platform microservices, and domain-*.
    • Update Deployment - App is only needed to update the vro-app Helm chart, which exposes Java-based REST API and updates the DB schema (via the db-init container).
    • Update Deployment - Platform - Used to deploy changes to Postgres, RabbitMQ, Redis, CLI (rarely needs to be used)

For the "Update Deployment" GitHub Action workflow inputs, run bash scripts/image_vars.src imageVersions to show the latest image tag values for each container image. Details at Container Image Versions.

Verify and Fix SecRel

  • Any SecRel alerts that have either an expired acknowledgement (from prior releases) or is new to the changes in the release will require remediation. If SecRel alerts can be addressed without compromising the release date, they should be addressed and this process will have to be repeated. Otherwise, the two engineers will need to collaborate to weigh the severity of alerts and the harms of delaying the release date to determine next steps. If acknowledging alerts is chosen, engineers should be able to clearly articulate their argument for acknowledgement to the SecRel assessor if needed and file an issue summarizing this argument.
  • Once all SecRel alerts have been acknowledged or addressed, rerun the SecRel GitHub workflow.

Process diagram

Legend:

  • rounded boxes: activity performed by a person
  • rectangular boxes: automation or object
  • solid line: causal connection or trigger
  • dotted line: ordering association