Encrypting Secrets - ocd-scm/ocd-meta GitHub Wiki

Introduction

OCD uses helmfile which supports mozilla/sops via helm-secrets. This means you have two alternative options of how to decrypt secrets:

Use helm-secrets to encrypt your yaml and add a secrets section to helmfile such as

    # will attempt to decrypt it using helm-secrets plugin
    secrets:
      - vault_secret.yaml

Use git-secret to encrypt your yaml and then decrypt them using ${OCD_SCRIPTS_PATH}/ocd-decrypt-secrets.sh that you must call manually from ocd-pre-apply-hook with

#!/bin/bash
# run the defaults that will try to gpg decrypt any *.secret files it finds
${OCD_SCRIPTS_PATH}/ocd-decrypt-secrets.sh

Note that with git-secret you don't add a secrets section to your helmfile.yaml instead you add a regular values file such as

    values:
      - "values.yaml"

That file must be in the .gitignore list and an encrypted version such as values.yaml.secret is held within the git repo.

If you are decrypting your secrets in ocd-pre-apply-hook you should consider calling ${OCD_SCRIPTS_PATH}/ocd-delete-secrets.sh from ocd-post-apply-hook to delete them after each use. Note that the ocd-environment-webhook runs a global pre and post hook and a pre and post hook per helmfile it installs. So to speed things up we decrypted in the global pre hook then delete the secrets in global post hook. See the ocd-environment-webhook README.md

git-secret only supports GnuPG encryption but helm-secrets supports GnuPG and additional key vaults. If you would like to use GnuPG with either git-secret or helm-secrets you will need to import a GnuPG private key. This document covers how to import a passphrase protected GnuPG key into your image below.

Using a GnuPG private key

In order to have OCD decrypt secrets with GPG with either git-secret or helmsecrets we first need OCD to have its own GPG public and private key. The private key needs to be secured with a strong passphrase. The private key must be explicitly stored in git at gpg/*.prv.key which is safe as long as we use a strong passphrase. The passphrase we store directly within a Kubernetes secret.

First, let's generate a strong passphrase to protect the private key. As we are being git driven we do this inside our environment repo:

# ensure that we don't accidentally publish the passphrase to the key
echo passphrase >> .gitignore
git add .gitignore
git commit -m "ignore passphrase"
# generate a random passphrase
base64 < /dev/urandom | head -c 20 > passphrase
# print it out
echo $(<passphrase)

IMPORTANT: You might want to backup this passphrase. If you are using git-secret how to backup passphrase is covered below.

Now generate a key pair for OCD deployment tools using that passphrase using the cli wizard:

gpg --full-generate-key

Now you need can export both the public and private key using the email you provided to the wizard. In the rest of this document, we will assume that is [email protected] but you should use something recognisable to your team as being the OCD system so that they understand why it can decrypt your secrets:

mkdir gpg
[email protected]
EXPORT_FINGER=$(gpg --list-secret-key --with-colons $EXPORT_EMAIL | awk -F':' '$1=="fpr"{print $10}' | head -1)
gpg --export-secret-key -a $EXPORT_EMAIL > gpg/$EXPORT_FINGER.prv.key && git add gpg/$EXPORT_FINGER.prv.key
gpg --export -a $EXPORT_EMAIL > gpg/$EXPORT_FINGER.pub.key && git add gpg/$EXPORT_FINGER.pub.key
git commit

You should now test whether the OCD container image can use the passphase to import the private key:

# run docker
docker run -it \
  -e passphrase=$(<passphrase) \
  -e EXPORT_FINGER=${EXPORT_FINGER} \
  -v $(pwd):$(pwd) \
  -w $(pwd) \
  simonmassey/ocd-environment-webhook:latest \
  /bin/bash
# import the secret key from $(pwd) in host
echo "$passphrase" | gpg --pinentry loopback --import --passphrase-fd 0 gpg/$EXPORT_FINGER.prv.key
# check that the key was imported
gpg --list-secret-keys

The import command should report "secret keys imported: 1" and the list secret keys command should show that gpg keyring holds the imported private key. If not check for cut-and-paste errors that the passphrase of the key doesn't match the text file on disk.

Load The GPG Private Key Passphrase

Finally we need to push the passphrase into a k8s secret so that the OCD image can use it:

# see https://superuser.com/a/1379872/285325
(passphrase=$(<passphrase); oc create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: openshift-passphrase
stringData:
  passphrase: ${passphrase}
EOF
)

Using `git secret`

Once you have a GnuPG private key ready you can choose to use git-secret to protect your secrets. WARNING that when you are using git secret everyone needs to be using versions of GPG that can share the same key ring stored in git. This typically means using the same major.minor number version of GPG. If it’s not possible for everyone to use the same GPG version they can run a docker container with a matching GPG version. OCD uses GPG 2.2 so you should manage secrets with that version and can use the ncpierson/git-secret container to hide or reveal the secrets on your laptop.

Let's say your sensitive file is "database-password.yaml" that you want to protect with git secret:

# one time init - here you should tell yourself the secrets so use your GPG email address
git secret init
git secret tell [email protected]
# now also tell OCD the secret using the email address of its key
git secret tell [email protected]
# only ignored files can be added to git secret to prevent accidental publication
echo database-password.yaml >> .gitignore
git add .gitignore
# add the database password to git secret manifest
git secret add database-password.yaml
# actually encrypt the file. this implicitly stores the public key under folder .gitsecret
git secret hide
# git add the encrypted file and the git secret store
git secret add database-password.yaml.secret .gitsecret/*
# and we are done
git commit

It is a good idea to repeat the "git secret tell" to share secret access with other team members who's GPG public key you have imported with gpg --import public.key. They can then reveal/hide the credentials to be able to update the secret information.

You might also want to encrypt the passphrase and store it with git secret so that you have a backup: