Container Creation - woveon/wovtools GitHub Wiki

< Home

Containers are built from a recipe in wovtools/containers, for a specific microservice. These recipes are used by wov-push-container to build and push a container to a container repository for deployment.

Terms

• Recipe - a bash script that is run to build a Docker container
• Container Code - a short descriptive code for a container. When paired with the project name, forms the microservice name. ex. container code 'data' with project 'alywan' produces a microservice 'alywandata'.
• Stages
  • prebuild - called before all files copied into container (do yarn/npm installs here)
  • build - all code is in the container so run these commands to get it ready to run
  • run - instructions on how to run container
  • NOTE: WORKDIR is /usr/src/app for all stages

1. make sure wovtools/containers exists
2. define a contianer code for your microservice. make it short as it will be appended to your project name to name the microservice. ex. 'foo'
3. write the recipe in file wovtools/containers/CONTAINERCODE
4. run wov-push-container -l to make sure WovTools can see your recipe
5. run wov-push-container -P to build the container and skip the push
6. in your Docker Repository (ex. AWS ECR) create a repository for these containers as '{PROJECT}/{PROJECT}{CONTAINERCODE}'.
   • To see what your Repo is set to currently, type wov-env --var WOV_ARCHIVEREPOSITORY.
   • ex. for project 'foo' and container 'bar' and WOV_ARCHIVEREPOSITORY set to AWSACCOUNTNUMBER.dkr.ecr.us-east-1.amazonaws.com, you need a repository in AWS ECR for 'foo/foobar'.
7. run wov-push-container to build and push the container to your Docker Repository

Tips:

• rsync only the necessary code to run, from SRCDIR (i.e. project dir), to DESTDIR (wovtools/cache/containers).
• place commands to build your microservice in: ${DESTDIR}/SECRET/build.docker
  • this will already have access to private code repos (via ssh keys)
  • assumes you followed: https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ssh-unixes.html
• place rules to run your microservice in : ${DESTDIR}/SECRET/run.docker

#!/usr/bin/env bash

if [ $DOECHO -ge 2 ]; then echo "  ... add src"; fi
# --------------------------------------------------------------------- 
rsync -ai \
    --exclude etc --exclude doc --exclude Makefile --exclude README.md \
    --exclude ".*.swp" --exclude ".*.mk" --exclude ".DS_Store" \
    --delete --delete-excluded \
    ${SRCDIR}/MYMICROSERVICE/src/* ${DESTDIR}/src/


if [ $DOECHO -ge 2 ]; then echo "  ... add node_modules"; fi
# --------------------------------------------------------------------- 
rsync -ai \
    --exclude etc --exclude doc --exclude Makefile --exclude README.md \
    --exclude ".*.swp" --exclude ".*.mk" --exclude ".DS_Store" \
    --delete --delete-excluded \
    ${SRCDIR}/MYMICROSERVICE/node_modules/*  ${DESTDIR}/node_modules

if [ $DOECHO -ge 2 ]; then echo "  ... add individual files"; fi
# --------------------------------------------------------------------- 
echo "  ... add index.js, package.json, package-lock.json"
cp ${SRCDIR}/MYMICROSERVICE/index.js ${DESTDIR}/.
cp ${SRCDIR}/MYMICROSERVICE/package.json ${DESTDIR}/.

# If you need to install private repos, you can add your ssh key to the container here
# if [ $DOECHO -ge 2 ]; then echo "    ... add ssh key (NOTE: IT IS REMOVED FROM THE CONTAINER FOR YOU!!!)"; fi
# --------------------------------------------------------------------- 
# cp ${HOME}/.ssh/id_rsa ${DESTDIR}/id_rsa

# NOTE: WORKDIR is /usr/src/app for all stages

if [ $DOECHO -ge 2 ]; then echo "  ... Docker commands for build"; fi
# --------------------------------------------------------------------- 
cat <<EOF > ${DESTDIR}/SECRET/build.docker
RUN npm install
EOF

if [ $DOECHO -ge 2 ]; then echo "  ... Docker commands for run"; fi
# --------------------------------------------------------------------- 
cat <<EOF > ${DESTDIR}/SECRET/run.docker

# Define default port
ENV APP_PORT_INT 80

# Run command
CMD  [ "npm", "run", "start" ]

EOF

You are going to have private code repositories and when you build your containers, they won't be accessible via git commands unless they have your ssh keys in the container. But, you don't want your personal keys to stay in the container. Wovtools handles this for you.

Option 1: Copy in your ssh key via cp ${HOME}/.ssh/id_rsa ${DESTDIR}/id_rsa

SERIOUSLY? Seriously, it's ok. Yes, Docker images cache data as layers after each command to optimize container building, so normally even deleting files in a container can still leave them in a container. But, from your Container Recipe, WovTools generates a Dockerfile that builds a container for pre-build and build stages, then starts over with a new container and copies in the 'DESTDIR'. So, your key NEVER EXISTS in the resulting container, only in the pre-build and build stages container, which is deleted.

Option 2 : Create a key per repo

TL;DR: Create a key per repo and use it on your machine via .ssh/config settings. wov-build-container will get those settings from 'WOV_SECRETFILE' and push to the container in the build phase. There should be no action on your part afterwards.

• Create a key for your repo. (and set AWS permissions)
  • AWS CodeCommit: https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ssh-unixes.html
  • Github : https://developer.github.com/v3/guides/managing-deploy-keys/#deploy-keys
• Use ssh to access your repos (i've found https unreliable for things like NPM).
• setup your config file to use the key. AWS requires you to use it for CodeCommit, while GitHub will allow you to use regular auth.

Host git-codecommit.us-east-1.amazonaws.com
  User AAAAAAAAAAAAAAAAAAAA
  IdentityFile ~/.ssh/wovtools/aws_rsa

• Add this information to secrets (json formatted file in wovtools/secrets location) under 'repositories', 'repo' with 'user' and 'privkeyloc'. Here is an example:

"repositories": {
    "git-codecommit.us-east-1.amazonaws.com": {
      "user": "AAAAAAAAAAAAAAAAAAAA",
      "privkeyloc": "~/.ssh/wovtools/aws_rsa"
    },

• for each recipe in 'wovtools/containers'
  • pack the container with the recipe and generate Docker commands in 'wovtools/cache'
  • wov-push-container-check to see if we need to build (diff to last build) and/or push
  • build if needed or forced
    • and archive this build
  • tag and push to remote repo if needed
    • update any AWS expired tokens if needed