👉 Decision document about how we manage dependencies

Python packages

Each Python app should have:

• requirements.in – top-level application dependencies
• requirements_for_test_common.in – common dependencies for local development and running tests (automatically copied from notifications-utils)
• requirements_for_test.in – additional dependencies for local development and running tests

These files are used to generate:

• requirements.txt - pinned versions of all application dependencies (installed when building apps to run on ECS)
• requirements_for_test.txt - pinned versions of all application, development and test dependencies (installed locally and on CI)

If you change any of the *.in files, run make freeze-requirements to regenerate their *.txt counterparts.

Note: for Dockerised apps, you'll need to run ./scripts/run_with_docker.sh make freeze-requirements instead.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

NPM packages

We use specific version numbers, not semantic version ranges so, when installing new versions of packages, use the --save-exact flag.

Security updates

We have a weekly job that audits Production packages for security vulnerabilities:

• Check the output in Concourse to see which packages have vulnerabilities.
• Make a PR to update affected packages to their patched versions.

If the packages are listed in package.json under dependencies, update the versions there and then run npm install to update the lockfile. For sub-dependencies, there's unfortunately no easy way to tell NPM to "update just this package and anything that conflicts with it":

• Try updating just the affected packages in package-lock.json and then run npm ci to see if there are any conflicts.
• If you see conflicts with other packages, it's usually safe to just run a dumb npm install, which will update everything.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

Docker images

Security updates

We have a weekly job that scans Production Docker images for vulnerable packages, in this repo

These images are stored in the notify-deploy account.

The resources were created to allow scan of images on the notify-deploy account.

An event bridge is scheduled to run once per week triggering the lambda function.

The lambda function scans the ECR repository to list the repositories and it also finds the very latest image version pushed by date.

After that if starts the scan on the images and if any CRITICAL vulnerability is found, it aggregates the results into a findings.txt file that will be sent to a SNS group at the end in the form of an email with the subject: ECR SCAN-Critical Vulnerability Report