Deploy Phoenix Project - SolarisJapan/lunaris-wiki GitHub Wiki

Deploying a New Phoenix/Elixir Project to Digital Ocean/Vultr

These steps assume you have a working phoenix/elixir project and have reached the point where you would like to deploy this project to a server.

🔥 In all code examples, you must to replace NewApplicationName and new_application_name with the name of your app!

Create Target Server on Digital Ocean

Start by creating a new Droplet on Digital Ocean, some key points:

  • Image: select distribution Ubuntu 20.04 x64
  • 2GB droplet minimum so things run smoothly
  • Datacenter Region, closest to Japan, Singapore?
  • SSH keys, be sure to add you own along with others that may need to access this Server
  • Hostname, choose something unique (we have a Bee theme going). This is difficult to change later, so try to pick a name you will be happy with in the future

Set up the new Server

Log into the new Server as root. The IP address can be found on the profile of the new droplet on Digital Ocean. If your SSH key was correctly added, you can SSH from you local machine into the server, i.e.:

ssh root@(ip address)

Update locale

We'll set some system locale variables to prevent some warnings as we progress:

sudo update-locale LC_ALL=en_US.UTF-8
sudo update-locale LANGUAGE=en_US.UTF-8

Create new deploy user

The next step is to create a non-root user on the server which will own our application and handle deployments. We'll configure the .ssh directory for the user, configure the user to have passwordless sudo access, and disabled password login to harden the server a bit.

Let's create the user deploy.

adduser deploy

Select a password for this user and fill in information as needed.

Next, run this series of commands to configure the user's .ssh directory:

sudo mkdir -p /home/deploy/.ssh
sudo touch /home/deploy/.ssh/authorized_keys
sudo chmod 700 /home/deploy/.ssh
sudo chmod 644 /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy
nano /etc/ssh/sshd_config

Verify that password authentication is set to "no":

PasswordAuthentication no

Add deploy to sudoers

Finally we'll add the deploy user to sudo. run the command:

visudo

This will open the sudoers file where we can add the deploy users privileges below root like this:

# User privilege specification
root     ALL=(ALL:ALL) ALL
deploy   ALL=(ALL) NOPASSWD: ALL

Add your ssh public key to the deploy user's authorized_keys

Next, get your public ssh key from your local machine. One way to get this is run the command on your local machine:

cat ~/.ssh/id_rsa.pub

Copy the key and now we need to paste it into the 'deploy' user's authorized keys file. On the target server, paste the key into authorized_keys:

nano /home/deploy/.ssh/authorized_keys

If this is done correctly, you should now be able to log into the server from your local machine as the deploy user without a password.

Install Erlang and Elixir on the target server

Log in as the deploy user and use the following commands to add erlang apt repository on your system.

Note: You can switch to the deploy user with su deploy than back to root with exit or you can ssh in and keep another session open, ssh deploy@(ip address)

Elixir can be installed with the version managing tool asdf or standalone.

Version Manager (asdf)

This section details installing Elixir with the version manager asdf. This is recommended if you have multiple projects on the same server or if the development project uses asdf. Please skip this section if you prefer to use the standalone elixir.

First, install asdf and add the necessary entries to your .bash.rc and .profile files as the guide suggests. Next, install the Elixir plugin, Erlang plugin and Nodejs plugin.

After installing asdf and its plugins; create your version file, enter your own Erlang and Elixir versions via editor or command line, install necessary dependencies, and install the versions specified therein.

touch ~/.tool-versions
echo "erlang 23.1.1" >> ~/.tool-versions
echo "elixir 1.11.1-otp-23" >> ~/.tool-versions
echo "nodejs 14.14.0" >> ~/.tool-versions
sudo apt-get -y install build-essential autoconf m4 libncurses5-dev libwxgtk3.0-gtk3-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libssh-dev unixodbc-dev xsltproc fop libxml2-utils libncurses-dev openjdk-11-jdk
asdf install

Standalone Elixir

Consider using asdf to manage your Elixir and Erlang versions. If you have decided to use asdf and read through that section, you may skip this section.

You can simply download erlang repository package from its official website and install on your system. This will install all of its dependencies as well.

wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
sudo dpkg -i erlang-solutions_1.0_all.deb
sudo apt-get update
sudo apt-get install esl-erlang

Next, lets install elixir

sudo apt-get install elixir

Optionally you can confirm erlang is installed by running it with the command:

erl

and confirm elixir is installed with the command

elixir --version

Install hex

As the 'deploy' user, use Mix to install Hex.

mix local.hex

When prompted to confirm the installation, enter Y.

OutputAre you sure you want to install "https://repo.hex.pm/installs/1.5.0/hex-0.17.1.ez"? [Yn] Y
* creating .mix/archives/hex-0.17.1

Install Nodejs

As the 'deploy' user, install Nodejs on the target server with these commands (the first one is to make sure we have curl installed):

sudo apt install curl
curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
sudo apt-get install -y nodejs

Install Postgresql

Next, login with the root account on our target server and run these commands to get Postgresql installed:

sudo apt update
sudo apt install -y postgresql postgresql-contrib

Create new database user

During the postgres installation, a postgres root user postgres was created, but we don't want to connect to the server with this user. We need to create a separate postgres user for our app.

On the target server, switch to the postgres user, now let's create a phx database user and set the new user's password:

su postgres
createuser phx --pwprompt

This will prompt you for a password and ask you to confirm it by entering it again.

Make note of these credentials we will need them later when we are configuring our Phoenix application

Create our application production database

We'll need to create our production database manually as edeliver only manages migrations.

Select a name for the production database, usually it should match the name in development with a _prod suffix, for this wiki we will use the name new_application_database_prod replace it with your database name in the following commands

As the postgres user on the target server, run this:

createdb new_application_database_prod

Next, log in to psql

psql

Ensure that you are logged into the postgres cli tool and run:

GRANT ALL PRIVILEGES ON DATABASE new_application_database_prod TO phx;

Make note of the database credentials: username ("phx"), password, and database name

Dedicated Database Server / DB Server on the app server.

For the best optimization results, and the maximum efficiency for your app, you should run your database server on a dedicated database server instance. The most limiting factor to Database performance is usually the number of CPU cores, so if you are looking to tweak a database server, a CPU optimized droplet might be the best option.

However, especially while your app is running in an alpha version etc. running the database on the same server should be absolutely ok. You can move the app to a new server later on, and use the original app server as your new dedicated DB server for this app.

Postgres configuration (dedicated DB server)

When using a dedicated postgres server it is recommended to use https://pgtune.leopard.in.ua/#/ to fine tune the postgres configuration.

Restart postgresql after editing the configuration:

sudo service postgresql restart

In order to allow incoming connections from the outside, we need to tell postgres to listen everywhere, and then set the rules to who is allowed to connect.

Find the listen_addresses setting in /etc/postgresql/[YOUR VERSION]/main/postgresql.conf and change it to

listen_addresses = '*'

There are different ways to set the permissions for incoming permissions, but only allowing specific IPs with password authentication would be maximum security.

in /etc/postgresql/[YOUR VERSION]/main/pg_hba.conf add a line that should look something like this:

host    all         all         [YOUR CLIENT IP]/32    md5

You can read more about this configuration here

🔥 Try to restart your client and database server if you feel like the configurations should work, but you keep getting a connection refused error! You might have accidentally blocked your client server because of failed connections. 🔥

Additional Server Configuration

Add github to list of knownhosts

attempt to connect to github via ssh

$ ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts

and confirm to add it to list of knownhosts

Project Dependences

Your project may have dependences that need to be installed, for example the Lunaful project needed redis to be installed and wkhtmltopdf is needed to generate pdfs. Make sure to install the dependencies needed by your project.

Production Configuration

Create prod.secret.exs

Phoenix created a config/prod.secret.exs file with your project. This is production connectivity information and thus ignored by git for security. We need to create this file on the target server. We're also going to store our applications in ~/apps/ format which is the equivalent of /home/deploy/apps/.

Switch over to our deploy user. On the target machine:

su deploy

Make a new directory for our secrets (replace new_application_name with the actual application name):

mkdir -p apps/new_application_name/secret

Then create a new prod.secret.exs file in that directory:

nano ~/apps/new_application_name/secret/prod.secret.exs

Add this content to it:

use Mix.Config

config :new_application_name, NewApplicationNameWeb.Endpoint,
  secret_key_base: "notreal4eG460CU9/2x68+hGBjJ5DIJ8YSxNeb6/S/uY8bm0x5VlpN4VsYtSwR3sqmdXt"

config :new_application_name, NewApplicationName.Repo,
  username: "phx",
  password: "the_password_for_the_phx_database_user",
  database: "new_application_database_prod",
  pool_size: 15

several things to replace in this file, database credentials, Application name, and secret key.. make special note of the CamelCased name on line 3, NewApplicationNameWeb.Endpoint.. put your CamelCased Application name in here, but be sure to retain Web on the end of the name.

You are also going to need a secret for line 4, you can generate a new secret key by using mix phx.gen.secret on your local machine.

Install Distillery and edeliver

Distillery compiles our Phoenix application into releases, and edeliver uses ssh and scp to build and deploy the releases to our production server. These are needed in our local project that we plan to deploy.

On your local machine, open mix.exs and add 2 deps:

     {:edeliver, ">= 1.6.0"},
     {:distillery, "~> 2.0", warn_missing: false},

Also add :edeliver to extra_applications in the application block, i.e.:

 def application do
   [
     mod: {NewApplicationName.Application, []},
     extra_applications: [:logger, :runtime_tools, :edeliver]
   ]
 end

Use mix to install deps with the command:

mix deps.get

Initialize Distillery

Distillery requires a build configuration file that is not generated by default.

Let's generate it with:

mix distillery.init

This generates configuration files for Distillery in the rel directory. We don't need to make any changes to the default config.

Configure edeliver

In the project on your local machine, create a .deliver directory in your project folder. Next add the config file, i.e.:

mkdir .deliver
cd .deliver
nano config

Here are the contents of the config file:

#!/bin/bash

APP="new_application_name"

BUILD_HOST="new_application_name.slrs.io"
BUILD_USER="deploy"
BUILD_AT="/tmp/edeliver/${APP}/builds"

START_DEPLOY=true
CLEAN_DEPLOY=true

# prevent re-installing node modules; this defaults to "."
GIT_CLEAN_PATHS="_build rel priv/static"

PRODUCTION_HOSTS="new_application_name.slrs.io"
PRODUCTION_USER="deploy"
DELIVER_TO="/home/deploy/apps"
AUTO_VERSION=revision

# For Phoenix projects, symlink prod.secret.exs to our tmp source
pre_erlang_get_and_update_deps() {
  local _prod_secret_path="/home/deploy/apps/${APP}/secret/prod.secret.exs"
  if [ "$TARGET_MIX_ENV" = "prod" ]; then
    status "Linking '$_prod_secret_path'"
    __sync_remote "
      [ -f ~/.profile ] && source ~/.profile
      mkdir -p '$BUILD_AT'
      ln -sfn '$_prod_secret_path' '${BUILD_AT}/config/prod.secret.exs'
    "
  fi
}

pre_erlang_clean_compile() {
  status "Running npm install"
    __sync_remote "
      [ -f ~/.profile ] && source ~/.profile
      set -e
      cd '${BUILD_AT}'/assets
      npm install
    "

  status "Compiling assets"
    __sync_remote "
      [ -f ~/.profile ] && source ~/.profile
      set -e
      cd '${BUILD_AT}'/assets
      node_modules/.bin/webpack --mode production --silent
    "

  status "Running phoenix.digest" # log output prepended with "----->"
  __sync_remote " # runs the commands on the build host
    [ -f ~/.profile ] && source ~/.profile # load profile (optional)
    set -e # fail if any command fails (recommended)
    cd '$BUILD_AT' # enter the build directory on the build host (required)
    # prepare something
    mkdir -p priv/static # required by the phoenix.digest task
    # run your custom task
    APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest $SILENCE
    APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest.clean $SILENCE
  "
}

post_extract_release_archive() {
  status "Removing start_erl.data if it exists"
  __remote "
    [ -f ~/.profile ] && source ~/.profile
    cd $DELIVER_TO/$APP/var $SILENCE
    rm start_erl.data
  "
}

Note on host name, we tend to deploy projects on a subdomain of our slrs.io domain, but this may differ depending on your particular project. Confirm the target url, configure it on CloudFlare ( https://www.cloudflare.com/ ), and put the correct url in this config file, or use the Server IP instead.

These are mostly environment variables use by Edeliver in the shell scripts. Go through them one by one and try to understand what they're doing.

You can read more about the configuration variables in their wiki ( https://github.com/edeliver/edeliver/wiki/Configuration-(.deliver-config) )

Enabling your App to Restart

If the server reboots for any reason, you likely want to app to automatically come back online. We can create a systemd service to do this.

sudo touch /etc/systemd/system/my_app.service

Edeliver's wiki provides a template service file for this (use the systemd format). Insert the contents in the file we just created. Update the [Unit] and [Service] blocks for your app.

[Unit]
Description=my_app
Environment=HOME=/home/deploy/apps/my_app
ExecStart=/home/deploy/apps/my_app/bin/my_app start
ExecStop=/home/deploy/apps/my_app/bin/my_app stop

Run the following command to ensure this service starts on reboot:

sudo systemctl enable my_app.service

Project prod configuration

We'll need to make some changes to the default production configuration in our project.

Open config/prod.exs and update it to this:

use Mix.Config

# For production, don't forget to configure the url host
# to something meaningful, Phoenix uses this information
# when generating URLs.
#
# Note we also include the path to a cache manifest
# containing the digested version of static files. This
# manifest is generated by the `mix phx.digest` task,
# which you should run after static files are built and
# before starting your production server.
config :new_application_name, NewApplicationNameWeb.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [host: "new_application_name.slrs.io", port: 80],
  cache_static_manifest: "priv/static/cache_manifest.json",
  server: true,
  code_reloader: false

# Do not print debug messages in production
config :logger, level: :info

config :phoenix, :serve_endpoints, true
import_config "prod.secret.exs"

Again, make special note of the CamelCased name NewApplicationNameWeb.Endpoint.. put your CamelCased Application name in here, but be sure to retain Web on the end of the name.

These are settings recommended by Distillery. You might notice that we're setting the system port using the PORT environment variable. This needs to be available during the build process and we can add it to the ~/.profile file on our production server.

Add env Variables to Target Server

Login as deploy to the target server.

nano ~/.profile

And add these lines:

export MIX_ENV=prod
export PORT=4000

This will ensure that our system runs on port 4000, and that the environment is set to prod.

We're using port 4000 because we'll be routing traffic to it using Nginx (discussed later)

Deployment

Now we are ready to build and deploy our first release.

On your local machine, we need to build the production release. In your project directory, run the command:

mix edeliver build release production

This builds the release on the target server, and stores the archive in your local .edeliver/releases directory. If everything went well with the build, lets deploy:

mix edeliver deploy release to production

This uploads the release archive to the specified directory and extracts it, and starts the production server.

Next, lets migrate our database schema with the command:

mix edeliver migrate production

Here are some other sample commands:

  • mix edeliver ping production # shows which nodes are up and running
  • mix edeliver version production # shows the release version running on the nodes
  • mix edeliver show migrations on production # shows pending database migrations
  • mix edeliver migrate production # run database migrations
  • mix edeliver restart production # or start or stop

You can read more about Edeliver in their documentation.

Test it in the browser

You should now be able to access your project at your IP or domain and port 4000

i.e.: http://new_application_name.slrs.io:4000

Install Nginx

Nginx is a powerful HTTP server and reverse proxy which is widely adopted and straightforward to install and configure.

You could run phoenix without Nginx as shown in this wiki, but you should probably not do that unless you have a good reason.

Log in as root to the target server and install Nginx with these commands:

sudo apt update
sudo apt install -y nginx

Remove default configurations:

rm /etc/nginx/sites-enabled/default /etc/nginx/sites-available/default

Create a new new_application_name.slrs.io file (use your own domain)

sudo nano /etc/nginx/sites-available/new_application_name.slrs.io

Here is the content of that file:

upstream phoenix {
    server 127.0.0.1:4000;
}

server {
  listen 80 default_server;
  listen [::]:80 default_server;
  server_name _;
  location / {
    allow all;

    # Proxy Headers
    proxy_pass http://phoenix;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-Cluster-Client-Ip $remote_addr;

    # WebSockets
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";

  }
}

Test Nginx config:

Run this command to check the config file.

sudo nginx -t

If everything looks good, let's activate the site by adding a link to this config in sites-enabled.

ln -s /etc/nginx/sites-available/new_application_name.slrs.io /etc/nginx/sites-enabled/new_application_name.slrs.io

Next lets restart Nginx:

sudo service nginx restart

You can now access your new server new_application_name.slrs.io and nginx will route requests to your Phoenix application running on port 4000.

You will also want to ensure that Nginx restarts on server reboot with the following command:

sudo systemctl enable nginx.service

SSL Certification with Certbot

After setting up your Nginx service, you can use Certbot to create SSL certificates and automatically renew them. Resource: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04

sudo apt install certbot python3-certbot-nginx -y

Ensure the server name is set in your Nginx site configuration file /etc/nginx/sites-available/new_application_name.slrs.io:

server {
  server_name new_application_name.slrs.io;
  ...
}

Create a security certificate, you may need to allow HTTP traffic through the UFW firewall if you have not already. You can also choose to have Certbot configure Nginx to redirect HTTP traffic through HTTPS.

sudo certbot --nginx -d new_application_name.slrs.io

Certbot will auotmatically renew SSL certificates before they expire. You can manually attempt a dry run of the renewal process to ensure there will be no issues.

sudo systemctl status certbot.timer
sudo certbot renew --dry-run

Hardening the Server

Secure Shared Memory

Configuring secured shared memory in Ubuntu Server 18.04 One of the first things you should do is secure the shared memory used on the system. If you’re unaware, shared memory can be used in an attack against a running service. Because of this, you’ll want to secure that portion of system memory. You can do this by modifying the /etc/fstab file.

First, logged in as root, open the file for editing by issuing the command:

nano /etc/fstab

Next, add the following line to the bottom of that file:

tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0

Save and close the file. In order for the changes to take effect, you must reboot the server with the command:

sudo reboot

Install fail2ban

The fail2ban system is an intrusion prevention system that monitors log files and searches for particular patterns that correspond to a failed login attempt. If a certain number of failed logins are detected from a specific IP address (within a specified amount of time), fail2ban will block access from that IP address.

To install fail2ban, open a terminal window and issue the command:

sudo apt-get install fail2ban

Within the directory /etc/fail2ban, you'll find the main configuration file, jail.conf. Also in that directory is the subdirectory, jail.d. The jail.conf file is the main configuration file, and jail.d contains the secondary configuration files. Do not edit the jail.conf file. Instead, we’ll create a new configuration that will monitor SSH logins with the command:

sudo nano /etc/fail2ban/jail.local

In this new file add the following contents:

[sshd]
enabled = true
port = 22
filter = sshd
logpath = /var/log/auth.log
maxretry = 5

Save and close that file. Restart fail2ban with the command:

sudo systemctl restart fail2ban

Activating the firewall

In order to activate the UFW firewall, issue the command:

sudo ufw enable

At this point, the firewall is active and will also start on a system reboot. However, there's one tiny issue: The firewall is now running and blocking all incoming traffic.

Enable SSH and http connections with these commands:

sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https

Completion

At this point the server should be set up and your app should be deployed to it... enjoy

⚠️ **GitHub.com Fallback** ⚠️