Dockerization Guide

This guide is meant to provide a resource for Dockerizing / containerizing Elixir and Phoenix apps.

Example Tech-Stack

Elixir, Phoenix, Ecto, Postgresql, and Redis for app development.

Docker (including Docker Desktop and CLI), Distillery, and Digital Ocean with Ubuntu 18.04 for deployment.

Docker images include Alpine-Linux, Alpine-Elixir, Nginx, Redis, and Postgres-Alpine.

Installation

Install Docker Desktop and CLI.

Setup and initialize your release with the Distillery Elixir dependency. A release does not need to be created yet but can be done to confirm functionality.

Distillery documentation also has guides for deploying to Digital Ocean with Docker if stuck. The files and help in this guide can be referenced against it.

Releases and Distillery Configuration

The configuration should be pretty straight forward and can be done as shown in the Distillery documentation. The initialization process should begin from the app's root directory for an umbrella app. Here are some things to be sure were covered for "distillation" of a release.

Umbrella Apps

For an umbrella app, the following line in the umbrella app config includes the config of each dependency app: import_config "../apps/*/config/config.exs".

Umbrella apps should also ensure that the app's root directory mix.exs file has a version number and app name just like those found in the dependency app mix.exs files. See mix.exs in the app root for an example.

See Distillery documentation for umbrella apps if each dependency is meant to be maintained and released independently.

Docker Setup and Configuration

Create the necessary files for Docker configuration.

touch Dockerfile
touch Makefile
touch .dockerignore
touch docker-compose.yml
mkdir -p config && touch config/docker.env

Some initial content for these files can be obtained from Distillery documentation.

Note that the Makefile should be indented with tabs not spaces. In Atom press command + shift + p and search for "Whitespace: Convert Spaces to Tabs". Running this command will quickly ensure the whitespace is properly formatted.

Docker Usage

Familiarize yourself with the concepts of containers, images, etc with the Docker documentation. Their purpose and use will be more apparent as you continue to dockerize your app.

As you build, debug, and edit your images, you may start to accumulate unused images and volumes. These should be pruned from time to time to avoid significant wasted drive space by running docker image prune.

Environment Variables

Edit rel/config.exs to include Mix Config Providers.

Once config_providers and overlays are set in rel/config.exs (as done early in Distillery Configuration) environment variables can be set via config/docker.env if desired by adding the following line to Dockerfile:

RUN \
  ...
  # mix release --verbose && \ # old release command
  env `cat config/docker.env | grep -v '^\s*#'` mix release --verbose && \
  ...

Of course the names of the environment variables found in your apps' config/prod.exs file(s) need to match. Note also that quotes in this file are escaped when inserted into the environment and should likely be left out.

Without this line environment variables found in the Docker Env are only used for run-time configuration. Files such as those for production configuration are compiled on release and do not reflect changes to this file. Any desired change in these variables requires a new release to be "distilled".

Run make build when ready to ensure that all these files work together to make a Dockerized image of your app release.

Database Setup and Migrations

Distillery documentation environment variables for the database can be edited slightly to make the database setup easier. With the right environment variable names the Docker Postgresql image will generate a database in it's volume when first created.

Renaming the environment variables in config/docker.env will likely make the setup much easier.

POSTGRES_HOST=db              # should be the name of the postgres image
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
PGPASSWORD=postgres           # same as the above password
POSTGRES_DB=docker_guide_db   # name of the first database

PGPASSWORD seems to be redundant, but this variable is used by the psql cli application to authenticate commands without a prompt. Toward the bottom of the Dockerfile add the following if it has not yet been included.

RUN apk update && \
    apk add --no-cache \
      bash \
      openssl-dev \
      postgresql-client # add this line to enable psql within the container

Set up migrations to occur when your containers are started.

Apps with Multiple Databases

This Docker container named "admin" set in docker-compose.yml starts the process of adding an additional database and migrations for each via this line: command: ["/opt/app/bin/loyalty", "setup"].

In order to ensure this setup file is found in the container, copy this script to the same location in your project and edit it to your needs.

This script first ensures that the Postgresql database is open to connections, retrying a set number of times before timing out. If the database connects, it then creates the additional database and runs migrations for the dependency apps.

Using Docker Compose

As with the Docker CLI, you will also frequently use the Docker Compose CLI.

At this point the main goal should be to have make build successfully creating images of your app. The setup and configuration for Docker compose should also be to a point where you can start creating containers with your images.

If make build successfully creates an image and docker-compose up creates a container for that image and it's dependencies, then it's time to deploy.

Server Preparation

At this point the release should successfully compile and docker-compose up allows you to see your app on the local machine (unless already configured otherwise).

Create a server for use with the application and do any initial setup for the server.

Create a folder for the application on the server and copy the files used for docker-compose.

# from server
mkdir -p /etc/dockerization_guide/config

# from local machine
scp ./docker-compose.yml [email protected]:/etc/dockerization_guide
scp ./config/docker.env [email protected]:/etc/dockerization_guide/config/docker.env

Dockerhub

In order to move our dockerized release to the server we can use Dockerhub and the Docker client application. In this example we have called our app "dockerization_guide" and are releasing version 0.1.0. After a successful make build we can run the following:

docker push dockerization_guide:0.1.0

# Or with an organization

docker push organization/dockerization_guide:0.1.0

Once this completes we simply run the following from the server

docker pull dockerization_guide:0.1.0

# Or with an organization

docker pull organization/dockerization_guide:0.1.0

If you are tagging images with latest and deploying that tag instead (not recommended), deploying an update this way may not work, as the image will not be refreshed if it has already been pulled. You can force an upgrade like so:

• Pull the latest image docker pull username/something/app1.0.0
• On the Server Change directory to where the docker-compose.yml exists /etc/app/
• Stop the current running Containers docker-compose down
• And start back up the Containers docker-compose up

SSL: Certbot and Nginx

Next we want to set up automatically renewing SSL certificates and proxying. Follow along and reference against the example Nginx and Docker Compose config files to see the adjustments made. Note also that if used in conjunction with Cloudflare's DNS the DNS record's "Status" field was set to "DNS only" since we are handling HTTPS redirects.

Pay close attention to the ports specified in docker-compose.yml as they are somewhat altered from the original Distillery guide. Note that Nginx was set to be dependent upon the web app.

/etc/letsencrypt/options-ssl-nginx.conf was not included by default and was found from the Let's Encrypt Github (included in data/nginx/options-ssl-nginx.conf, copy to server in appropriate location)

Deployment

Once the project has been pulled onto the server and Nginx has been set up, docker can be set to automatically start.

The following enables docker to start when the server restarts.

sudo systemctl enable docker

Once everything is set up you can run the following from the folder where the docker-compose.yml file is:

docker-compose up -d

The -d flag daemonizes the process and it will also be restarted if the docker service has been enabled to restart via systemctl. Remove this flag to see the server output directly for debugging.

Interfacing as a Developer

With the current setup, any access to the app or its databases need to occur through Docker. The volumes where the data is stored are in a somewhat hidden location from the user (as a developer).

Moreover, you needn't have installed a programming environment on the server for Docker to work, so no applications such as iex or mix are available on the server. Here are some ways to interface with your Docker app.

Docker Exec

From within the server, docker exec can be used to interface with the container via command line.

# first find CONTAINER - the id of the container - for docker exec    docker container ps container
docker exec -it CONTAINER COMMAND

### Useful commands
# Access local postgres database with a password
docker exec -it CONTAINER psql "user=postgres dbname=database host=db password=postgres"

# Access the Elixir app via console
docker exec -it CONTAINER sh
cd bin/
./myapp remote_console

Docker Logs

From within the server, docker logs can be used to see the container log output for debugging and monitoring.

NOTE: docker logs without specifying any "tail length" will print all available log information and is probably excessive.

# first find CONTAINER - the id of the container - for docker logs    docker container ps container
docker logs CONTAINER

### Useful commands
# See the last 30 lines of the logs
docker logs --tail 30 CONTAINER

# See the logs as they are updated live (trimmed to 30 lines)
docker logs --follow --tail 30 CONTAINER

pgAdmin 4

The setup for pgAdmin 4 is very quick. From the pgAdmin client select "Object" > "Create" > "Server...". Under the "Connections" tab set the "Hostname/address", "Port", "Username", and "Password" with the same settings defined on the server (via docker.env).

"Maintenance Database" can be left as the default "postgres" unless changed previously. Under the "Advanced" tab, set "Host address" as the IP address of the server (without url scheme, port, etc.). After saving the database should be accessible.

Areas for Improvement

Docker swarm may eventually be a desired next step for the deployment setup. Currently if the server is restarted, Docker Compose starts the containers when docker is restarted thanks to systemctl.

Currently Nginx error logging is not working as intended. There should be a file specified in the config file matching that found in docker-compose.yml where errors can be logged, but it was not working properly when attempted.

Maintaining version numbers for future releases, particularly for umbrella apps, may be something of a hassle to maintain in multiple locations. Maybe specifying docker_guide:latest in the right locations makes this easier. Version number is automaticaly configured from the mix.exs file.

MyPhoenixApp.Endpoint config (in conjunction with respective fields in docker-compose.yml) may need to be altered for a more appropriate and secure web / ssl setup. Currently the app is still using localhost of it's container for production host.

Phoenix's default cache_manifest.json file created by mix digest and defined in the production config was having trouble being found during docker build (make build). This may be useful or even necessary for some apps to function properly.