Deployment - AndersenLab/CAENDR GitHub Wiki

Overview

This page outlines how to deploy a new version of a module to the public site.

There are two steps required to deploy a new version:

  1. Compile the (changed) code to a Docker image, and push it to GCP.
  2. Deploy the pushed image to the relevant Cloud Run Service.



Pushing a Module as a Docker Image

This section describes how to compile your changed code and upload it to GCP, where it can then be deployed. All(?) of the CaeNDR modules are compiled as Docker containers, which are pushed to the Google Cloud Registry.

Pre-requisites:

  • Ensure that you are logged in to the GCLOUD GCP project in the CLI, or using a devops service account.

Pushing a New Version

  1. Open a terminal at the root of the project.

  2. Set the required environment variables (read more):

    export ENV={ ENV_TO_DEPLOY }
    export GOOGLE_APPLICATION_CREDENTIALS={ PATH_TO_GCP_CREDENTIALS }
  3. (Re)authenticate with gcloud, if necessary (read more):

    gcloud auth login
    • Note: Make sure you authorize with the correct GCP project.
  4. Increment the versions for each module that is being updated as part of the deployment:

    • Update the version property for the module in the /env/{env}/global.env
    • Update version in the file src/modules/{module_name}/module.env
  5. Move to each module folder and configure the modules for deployment:

    cd src/modules/{module_name}
    make configure
    • The module root folder should now contain a .env file
    • The module root folder SHOULD NOT contain a venv folder
  6. Push the module to GCR (src/modules/{module_name}):

    make publish
  7. When the command completes, check the GCR and confirm your image with the proper version tag is appearing.




Deploying a Pushed Image

This section describes how to deploy a new version of the code, once it has been pushed to GCP.

Currently, the pushed site image(s) are deployed via Google Cloud Run. They used to be deployed from the command line to App Engine, using Terraform, but this has been deprecated. A section on the App Engine process is retained here for the sake of completeness, but it should NOT be used.


Deploying Manually with Google Cloud Run

  1. Push any new images that need to be deployed, as described above.

  2. In a browser, navigate to Google Cloud Run in the GCP console.

  3. Select the Service that you want to deploy.

  4. Click "Edit & deploy new revision".

  5. Under "Edit Container", select the image that you deployed in step (1). Personally, I check:

    • ...the image tag against the one I know I just deployed.
    • ...the first few characters of the hash value against the terminal output from the deployment.
    • Note: Double-check that you picked the right Service in step (3)! (I've been burned by this once.)
  6. Enter all relevant environment variables for the container under "Variables & Secrets".

    • Note: If you've already deployed a version of this container, you can probably leave the existing values.
    • Note: HOWEVER, IF you added or changed one or more environment variables in the .env files as part of your change, you MUST update them here! (I've been burned by this, too.)
    • Note: Our app / library requests secret values in the code, using a service account, so you don't need to explicitly expose any secret values here.
  7. If desired, click to serve this revision immediately.

  8. Click "Deploy".

  9. Once the new site version is active, check it in your browser!


Deploying with Terraform & Google App Engine - Old Method (DEPRECATED)

This is the old way we used to deploy the CaeNDR site; it is only included here for completeness. Note that this method deploys all(?) modules together, which (a) is rather inefficient and (b) can create race conditions.

  1. Push any new images that need to be deployed, as described above.

  2. Set up a Terminal window with the required environment variables in the root directory, as described above.

  3. Use make to deploy the new app version:

    make cloud-resource-deploy

Targeted deployment for individual modules / components was under construction until isolated TF states could be establish for each module, but since we've moved away from this deployment model, I believe this is no longer relevant.




Troubleshooting

  • Even if ENV and GOOGLE_APPLICATION_CREDENTIALS are set correctly you will need to be logged into gcloud and configure docker to enable publishing containers to GCR since the service account does not have permissions to publish.

  • Sometimes after deployment of the full application the ext_assets folder will not copy to the GCP static bucket, but terraform state will reflect the correct bucket resources. You'll notice the CeNDR logo and worms video will not show up on the home page. Simply redeploy the full application and the assets should be correctly copied to the GCP static bucket, fixing the issue.

  • Deployment will not work if a virtual environment exists in img_thumb_gen, giving an error like the following:

    ╷
    │ Error: Error while updating cloudfunction configuration: Error waiting for Updating CloudFunctions Function: Error code 14, message: The service has encountered an error during container import. Please try again later
    │ 
    │   with module.img_thumb_gen.google_cloudfunctions_function.generate_thumbnails,
    │   on modules/img_thumb_gen/cloud-function.tf line 1, in resource "google_cloudfunctions_function" "generate_thumbnails":
    │    1: resource "google_cloudfunctions_function" "generate_thumbnails" {
    │ 
    ╵
    make: *** [cloud-resource-deploy] Error 1
    

    Remove the venv directory and try redeploying.

  • Due to a race condition, sometimes Terraform will attempt to access the new site image before it has been built and published to GCP. Manually publishing the image by running make publish in src/modules/site (or src/modules/site-v2), then deploying, should fix this issue.

⚠️ **GitHub.com Fallback** ⚠️