Releases 101 - opengovsg/GoGovSG GitHub Wiki

This release document is adapted from Checkfirst.

What is a release?

A release is a process where the product team delivers some changes in code in order to give their users more features, or fix some bugs that the team has previously discovered.

What happens during a release?

While the actual change/code deployment is important, we believe that it is important to set up a process around releases because human errors that result in production outages could happen.

Here are the 3 major steps that our team does during a release:

Deploying code changes to staging

  • For testing
  • For PM and Designer validation

Updating release version and change log

  • For anyone wanting to know exactly what changed between this release and the previous one

Running through a release checklist for production

  • Database migrations (if any)
  • Update environment variables (if any)
  • Deploy code to production
  • Manual tests run by engineering team on the production environment post-release, mostly for sanity checks

Release procedure

GoGov adopts the Git flow branching strategy. If you're unfamiliar with this branching strategy, refer to the Branching Strategy wiki to familiarise yourself with the branching strategy before the release.

1 day before release [Prep for review on staging]

  • Get PRs reviewed and merge them into develop
  • Pull the latest develop branch on your local machine by running the following command on your terminal git checkout develop && git pull origin develop.
  • Verify that all the commits going into release are in the develop branch by running git log and reviewing the commit history.
  • Push existing develop branch code to staging by running the following command on your terminal: git push origin <branch_name>:edge [-f]. The reason why we do this is because our GitHub Actions deploys changes to the AWS staging environment from our edge branch.
  • Request for PM and designer input. If there are small change requests from PM and designers, make the changes in the develop branch, and prepare them for release. If there are significant changes such that you might delay the release, make a judgement call - perhaps we could delay the release of this specific feature/commit.

11am of release day or earlier

  • Ensure that fixes and reviews are merged into the develop branch by 11am at the latest.

3pm of release day [Prep to release]

  • First, figure out if your release is going to be a patch, minor, or major one. Refer to https://semver.org/ for details.
  • Locally, create a branch off the latest version of develop, named release-x.xx, by running git checkout -b release-x.xx. If necessary, check which version number we are on by referring to the package.json file.
  • Create a new package version and update the change log by running the following command: npm version [patch | minor | major]. If you run git log, you should see a new commit with commit message x.xx, containing updates to the CHANGELOG.md, package.json, and package-lock.json files.
  • Push the new release branch to GitHub, by running git push origin release-x.xx
  • Push tags to GitHub using git push --tags
  • Make three pull requests. One from release-x.xx -> release, one from release-x.xx -> develop and lastly from from release-x.xx -> master. The reason why we merge to release is because our GitHub Actions deploys changes to the AWS production environment from our release branch.
  • Copy the Release Checklist tests into the release-x.xx -> release PR, and add any additional release notes on migrations, new environment variables, new infrastructure, special tests and so on. Sample notes: https://github.com/opengovsg/GoGovSG/pull/1813
  • Run through the checklist of tests on the go gov staging environment. We can skip running tests on edu and health staging.

3.30pm of release day [Release]

  • Have your release notes ready. The release notes should be auto-generated and added to CHANGELOG.md.
  • Hop on a video call with at least 1 other engineer. This only applies if there are sufficient engineers on the product team, of course.
  • Run through the additional release notes on the PR. Do you have any migrations to perform on the production database? Do you have to add new environment variables to the production environment? Do you need to add new pieces of infrastructure/third-party integrations? Do you have specialised manual tests to run on staging environment for a sanity check?
  • When ready, merge the release-x.xx branch into release by merge commit. Wait for GitHub Actions to build and deploy.
  • Once the application has been updated, run through the manual tests in the release notes on the production environment.
  • If everything is fine, merge the release-x.xx branch back into develop by merge commit.
  • If things are not fine, perform a manual rollback by going into the AWS console and deploying a previous version of the application on EB. We have three environments on EB to roll back on.

Post-release

  • After verifying that the production environment is stable for a few days, merge the release-x.xx branch into master by merge commit.