Deploy Phoenix Project - SolarisJapan/lunaris-wiki GitHub Wiki
These steps assume you have a working phoenix/elixir project and have reached the point where you would like to deploy this project to a server.
🔥 In all code examples, you must to replace NewApplicationName
and new_application_name
with the name of your app!
Start by creating a new Droplet on Digital Ocean, some key points:
- Image: select distribution
Ubuntu 20.04 x64
- 2GB droplet minimum so things run smoothly
- Datacenter Region, closest to Japan, Singapore?
- SSH keys, be sure to add you own along with others that may need to access this Server
- Hostname, choose something unique (we have a Bee theme going). This is difficult to change later, so try to pick a name you will be happy with in the future
Log into the new Server as root
. The IP address can be found on the profile of the new droplet on Digital Ocean. If your SSH key was correctly added, you can SSH from you local machine into the server, i.e.:
ssh root@(ip address)
We'll set some system locale variables to prevent some warnings as we progress:
sudo update-locale LC_ALL=en_US.UTF-8
sudo update-locale LANGUAGE=en_US.UTF-8
The next step is to create a non-root user on the server which will own our application and handle deployments. We'll configure the .ssh directory for the user, configure the user to have passwordless sudo access, and disabled password login to harden the server a bit.
Let's create the user deploy
.
adduser deploy
Select a password for this user and fill in information as needed.
Next, run this series of commands to configure the user's .ssh directory:
sudo mkdir -p /home/deploy/.ssh
sudo touch /home/deploy/.ssh/authorized_keys
sudo chmod 700 /home/deploy/.ssh
sudo chmod 644 /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy
nano /etc/ssh/sshd_config
Verify that password authentication is set to "no":
PasswordAuthentication no
Finally we'll add the deploy
user to sudo. run the command:
visudo
This will open the sudoers file where we can add the deploy users privileges below root like this:
# User privilege specification
root ALL=(ALL:ALL) ALL
deploy ALL=(ALL) NOPASSWD: ALL
Next, get your public ssh key from your local machine. One way to get this is run the command on your local machine:
cat ~/.ssh/id_rsa.pub
Copy the key and now we need to paste it into the 'deploy' user's authorized keys file. On the target server, paste the key into authorized_keys:
nano /home/deploy/.ssh/authorized_keys
If this is done correctly, you should now be able to log into the server from your local machine as the deploy
user without a password.
Log in as the deploy
user and use the following commands to add erlang apt repository on your system.
Note: You can switch to the
deploy
user with su deploy than back to root with exit or you can ssh in and keep another session open, ssh deploy@(ip address)
Elixir can be installed with the version managing tool asdf or standalone.
This section details installing Elixir with the version manager asdf. This is recommended if you have multiple projects on the same server or if the development project uses asdf. Please skip this section if you prefer to use the standalone elixir.
First, install asdf and add the necessary entries to your .bash.rc
and .profile
files as the guide suggests. Next, install the Elixir plugin, Erlang plugin and Nodejs plugin.
After installing asdf and its plugins; create your version file, enter your own Erlang and Elixir versions via editor or command line, install necessary dependencies, and install the versions specified therein.
touch ~/.tool-versions
echo "erlang 23.1.1" >> ~/.tool-versions
echo "elixir 1.11.1-otp-23" >> ~/.tool-versions
echo "nodejs 14.14.0" >> ~/.tool-versions
sudo apt-get -y install build-essential autoconf m4 libncurses5-dev libwxgtk3.0-gtk3-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libssh-dev unixodbc-dev xsltproc fop libxml2-utils libncurses-dev openjdk-11-jdk
asdf install
Consider using asdf to manage your Elixir and Erlang versions. If you have decided to use asdf and read through that section, you may skip this section.
You can simply download erlang repository package from its official website and install on your system. This will install all of its dependencies as well.
wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
sudo dpkg -i erlang-solutions_1.0_all.deb
sudo apt-get update
sudo apt-get install esl-erlang
Next, lets install elixir
sudo apt-get install elixir
Optionally you can confirm erlang is installed by running it with the command:
erl
and confirm elixir is installed with the command
elixir --version
As the 'deploy' user, use Mix to install Hex.
mix local.hex
When prompted to confirm the installation, enter Y.
OutputAre you sure you want to install "https://repo.hex.pm/installs/1.5.0/hex-0.17.1.ez"? [Yn] Y
* creating .mix/archives/hex-0.17.1
As the 'deploy' user, install Nodejs on the target server with these commands (the first one is to make sure we have curl installed):
sudo apt install curl
curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
sudo apt-get install -y nodejs
Next, login with the root
account on our target server and run these commands to get Postgresql installed:
sudo apt update
sudo apt install -y postgresql postgresql-contrib
During the postgres installation, a postgres root user postgres
was created, but we don't want to connect to the server with this user. We need to create a separate postgres user for our app.
On the target server, switch to the postgres
user, now let's create a phx
database user and set the new user's password:
su postgres
createuser phx --pwprompt
This will prompt you for a password and ask you to confirm it by entering it again.
Make note of these credentials we will need them later when we are configuring our Phoenix application
We'll need to create our production database manually as edeliver only manages migrations.
Select a name for the production database, usually it should match the name in development with a
_prod
suffix, for this wiki we will use the namenew_application_database_prod
replace it with your database name in the following commands
As the postgres
user on the target server, run this:
createdb new_application_database_prod
Next, log in to psql
psql
Ensure that you are logged into the postgres cli tool and run:
GRANT ALL PRIVILEGES ON DATABASE new_application_database_prod TO phx;
Make note of the database credentials: username ("phx"), password, and database name
For the best optimization results, and the maximum efficiency for your app, you should run your database server on a dedicated database server instance. The most limiting factor to Database performance is usually the number of CPU cores, so if you are looking to tweak a database server, a CPU optimized droplet might be the best option.
However, especially while your app is running in an alpha version etc. running the database on the same server should be absolutely ok. You can move the app to a new server later on, and use the original app server as your new dedicated DB server for this app.
When using a dedicated postgres server it is recommended to use https://pgtune.leopard.in.ua/#/ to fine tune the postgres configuration.
Restart postgresql after editing the configuration:
sudo service postgresql restart
In order to allow incoming connections from the outside, we need to tell postgres to listen everywhere, and then set the rules to who is allowed to connect.
Find the listen_addresses
setting in /etc/postgresql/[YOUR VERSION]/main/postgresql.conf
and change it to
listen_addresses = '*'
There are different ways to set the permissions for incoming permissions, but only allowing specific IPs with password authentication would be maximum security.
in /etc/postgresql/[YOUR VERSION]/main/pg_hba.conf
add a line that should look something like this:
host all all [YOUR CLIENT IP]/32 md5
You can read more about this configuration here
🔥 Try to restart your client and database server if you feel like the configurations should work, but you keep getting a connection refused
error! You might have accidentally blocked your client server because of failed connections. 🔥
attempt to connect to github via ssh
$ ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
and confirm to add it to list of knownhosts
Your project may have dependences that need to be installed, for example the Lunaful project needed redis to be installed and wkhtmltopdf is needed to generate pdfs. Make sure to install the dependencies needed by your project.
Phoenix created a config/prod.secret.exs file with your project. This is production connectivity information and thus ignored by git for security. We need to create this file on the target server. We're also going to store our applications in ~/apps/ format which is the equivalent of /home/deploy/apps/.
Switch over to our deploy
user. On the target machine:
su deploy
Make a new directory for our secrets (replace new_application_name with the actual application name):
mkdir -p apps/new_application_name/secret
Then create a new prod.secret.exs file in that directory:
nano ~/apps/new_application_name/secret/prod.secret.exs
Add this content to it:
use Mix.Config
config :new_application_name, NewApplicationNameWeb.Endpoint,
secret_key_base: "notreal4eG460CU9/2x68+hGBjJ5DIJ8YSxNeb6/S/uY8bm0x5VlpN4VsYtSwR3sqmdXt"
config :new_application_name, NewApplicationName.Repo,
username: "phx",
password: "the_password_for_the_phx_database_user",
database: "new_application_database_prod",
pool_size: 15
several things to replace in this file, database credentials, Application name, and secret key.. make special note of the CamelCased name on line 3, NewApplicationNameWeb.Endpoint.. put your CamelCased Application name in here, but be sure to retain Web on the end of the name.
You are also going to need a secret for line 4, you can generate a new secret key by using mix phx.gen.secret on your local machine.
Distillery compiles our Phoenix application into releases, and edeliver uses ssh and scp to build and deploy the releases to our production server. These are needed in our local project that we plan to deploy.
On your local machine, open mix.exs
and add 2 deps:
{:edeliver, ">= 1.6.0"},
{:distillery, "~> 2.0", warn_missing: false},
Also add :edeliver to extra_applications in the application block, i.e.:
def application do
[
mod: {NewApplicationName.Application, []},
extra_applications: [:logger, :runtime_tools, :edeliver]
]
end
Use mix to install deps with the command:
mix deps.get
Distillery requires a build configuration file that is not generated by default.
Let's generate it with:
mix distillery.init
This generates configuration files for Distillery in the rel directory. We don't need to make any changes to the default config.
In the project on your local machine, create a .deliver directory in your project folder. Next add the config file, i.e.:
mkdir .deliver
cd .deliver
nano config
Here are the contents of the config file:
#!/bin/bash
APP="new_application_name"
BUILD_HOST="new_application_name.slrs.io"
BUILD_USER="deploy"
BUILD_AT="/tmp/edeliver/${APP}/builds"
START_DEPLOY=true
CLEAN_DEPLOY=true
# prevent re-installing node modules; this defaults to "."
GIT_CLEAN_PATHS="_build rel priv/static"
PRODUCTION_HOSTS="new_application_name.slrs.io"
PRODUCTION_USER="deploy"
DELIVER_TO="/home/deploy/apps"
AUTO_VERSION=revision
# For Phoenix projects, symlink prod.secret.exs to our tmp source
pre_erlang_get_and_update_deps() {
local _prod_secret_path="/home/deploy/apps/${APP}/secret/prod.secret.exs"
if [ "$TARGET_MIX_ENV" = "prod" ]; then
status "Linking '$_prod_secret_path'"
__sync_remote "
[ -f ~/.profile ] && source ~/.profile
mkdir -p '$BUILD_AT'
ln -sfn '$_prod_secret_path' '${BUILD_AT}/config/prod.secret.exs'
"
fi
}
pre_erlang_clean_compile() {
status "Running npm install"
__sync_remote "
[ -f ~/.profile ] && source ~/.profile
set -e
cd '${BUILD_AT}'/assets
npm install
"
status "Compiling assets"
__sync_remote "
[ -f ~/.profile ] && source ~/.profile
set -e
cd '${BUILD_AT}'/assets
node_modules/.bin/webpack --mode production --silent
"
status "Running phoenix.digest" # log output prepended with "----->"
__sync_remote " # runs the commands on the build host
[ -f ~/.profile ] && source ~/.profile # load profile (optional)
set -e # fail if any command fails (recommended)
cd '$BUILD_AT' # enter the build directory on the build host (required)
# prepare something
mkdir -p priv/static # required by the phoenix.digest task
# run your custom task
APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest $SILENCE
APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest.clean $SILENCE
"
}
post_extract_release_archive() {
status "Removing start_erl.data if it exists"
__remote "
[ -f ~/.profile ] && source ~/.profile
cd $DELIVER_TO/$APP/var $SILENCE
rm start_erl.data
"
}
Note on host name, we tend to deploy projects on a subdomain of our slrs.io domain, but this may differ depending on your particular project. Confirm the target url, configure it on CloudFlare ( https://www.cloudflare.com/ ), and put the correct url in this config file, or use the Server IP instead.
These are mostly environment variables use by Edeliver in the shell scripts. Go through them one by one and try to understand what they're doing.
You can read more about the configuration variables in their wiki ( https://github.com/edeliver/edeliver/wiki/Configuration-(.deliver-config) )
If the server reboots for any reason, you likely want to app to automatically come back online. We can create a systemd service to do this.
sudo touch /etc/systemd/system/my_app.service
Edeliver's wiki provides a template service file for this (use the systemd format). Insert the contents in the file we just created. Update the [Unit]
and [Service]
blocks for your app.
[Unit]
Description=my_app
Environment=HOME=/home/deploy/apps/my_app
ExecStart=/home/deploy/apps/my_app/bin/my_app start
ExecStop=/home/deploy/apps/my_app/bin/my_app stop
Run the following command to ensure this service starts on reboot:
sudo systemctl enable my_app.service
We'll need to make some changes to the default production configuration in our project.
Open config/prod.exs and update it to this:
use Mix.Config
# For production, don't forget to configure the url host
# to something meaningful, Phoenix uses this information
# when generating URLs.
#
# Note we also include the path to a cache manifest
# containing the digested version of static files. This
# manifest is generated by the `mix phx.digest` task,
# which you should run after static files are built and
# before starting your production server.
config :new_application_name, NewApplicationNameWeb.Endpoint,
http: [port: {:system, "PORT"}],
url: [host: "new_application_name.slrs.io", port: 80],
cache_static_manifest: "priv/static/cache_manifest.json",
server: true,
code_reloader: false
# Do not print debug messages in production
config :logger, level: :info
config :phoenix, :serve_endpoints, true
import_config "prod.secret.exs"
Again, make special note of the CamelCased name NewApplicationNameWeb.Endpoint.. put your CamelCased Application name in here, but be sure to retain Web on the end of the name.
These are settings recommended by Distillery. You might notice that we're setting the system port using the PORT
environment variable. This needs to be available during the build process and we can add it to the ~/.profile
file on our production server.
Login as deploy
to the target server.
nano ~/.profile
And add these lines:
export MIX_ENV=prod
export PORT=4000
This will ensure that our system runs on port 4000, and that the environment is set to prod.
We're using port 4000 because we'll be routing traffic to it using Nginx (discussed later)
Now we are ready to build and deploy our first release.
On your local machine, we need to build the production release. In your project directory, run the command:
mix edeliver build release production
This builds the release on the target server, and stores the archive in your local .edeliver/releases directory. If everything went well with the build, lets deploy:
mix edeliver deploy release to production
This uploads the release archive to the specified directory and extracts it, and starts the production server.
Next, lets migrate our database schema with the command:
mix edeliver migrate production
Here are some other sample commands:
- mix edeliver ping production # shows which nodes are up and running
- mix edeliver version production # shows the release version running on the nodes
- mix edeliver show migrations on production # shows pending database migrations
- mix edeliver migrate production # run database migrations
- mix edeliver restart production # or start or stop
You can read more about Edeliver in their documentation.
You should now be able to access your project at your IP or domain and port 4000
i.e.: http://new_application_name.slrs.io:4000
Nginx is a powerful HTTP server and reverse proxy which is widely adopted and straightforward to install and configure.
You could run phoenix without Nginx as shown in this wiki, but you should probably not do that unless you have a good reason.
Log in as root
to the target server and install Nginx with these commands:
sudo apt update
sudo apt install -y nginx
Remove default configurations:
rm /etc/nginx/sites-enabled/default /etc/nginx/sites-available/default
Create a new new_application_name.slrs.io
file (use your own domain)
sudo nano /etc/nginx/sites-available/new_application_name.slrs.io
Here is the content of that file:
upstream phoenix {
server 127.0.0.1:4000;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
location / {
allow all;
# Proxy Headers
proxy_pass http://phoenix;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Cluster-Client-Ip $remote_addr;
# WebSockets
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
Run this command to check the config file.
sudo nginx -t
If everything looks good, let's activate the site by adding a link to this config in sites-enabled.
ln -s /etc/nginx/sites-available/new_application_name.slrs.io /etc/nginx/sites-enabled/new_application_name.slrs.io
Next lets restart Nginx:
sudo service nginx restart
You can now access your new server new_application_name.slrs.io and nginx will route requests to your Phoenix application running on port 4000.
You will also want to ensure that Nginx restarts on server reboot with the following command:
sudo systemctl enable nginx.service
After setting up your Nginx service, you can use Certbot to create SSL certificates and automatically renew them. Resource: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04
sudo apt install certbot python3-certbot-nginx -y
Ensure the server name is set in your Nginx site configuration file /etc/nginx/sites-available/new_application_name.slrs.io
:
server {
server_name new_application_name.slrs.io;
...
}
Create a security certificate, you may need to allow HTTP traffic through the UFW firewall if you have not already. You can also choose to have Certbot configure Nginx to redirect HTTP traffic through HTTPS.
sudo certbot --nginx -d new_application_name.slrs.io
Certbot will auotmatically renew SSL certificates before they expire. You can manually attempt a dry run of the renewal process to ensure there will be no issues.
sudo systemctl status certbot.timer
sudo certbot renew --dry-run
Configuring secured shared memory in Ubuntu Server 18.04 One of the first things you should do is secure the shared memory used on the system. If you’re unaware, shared memory can be used in an attack against a running service. Because of this, you’ll want to secure that portion of system memory. You can do this by modifying the /etc/fstab file.
First, logged in as root
, open the file for editing by issuing the command:
nano /etc/fstab
Next, add the following line to the bottom of that file:
tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0
Save and close the file. In order for the changes to take effect, you must reboot the server with the command:
sudo reboot
The fail2ban system is an intrusion prevention system that monitors log files and searches for particular patterns that correspond to a failed login attempt. If a certain number of failed logins are detected from a specific IP address (within a specified amount of time), fail2ban will block access from that IP address.
To install fail2ban, open a terminal window and issue the command:
sudo apt-get install fail2ban
Within the directory /etc/fail2ban, you'll find the main configuration file, jail.conf
. Also in that directory is the subdirectory, jail.d
. The jail.conf
file is the main configuration file, and jail.d
contains the secondary configuration files. Do not edit the jail.conf
file. Instead, we’ll create a new configuration that will monitor SSH logins with the command:
sudo nano /etc/fail2ban/jail.local
In this new file add the following contents:
[sshd]
enabled = true
port = 22
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
Save and close that file. Restart fail2ban with the command:
sudo systemctl restart fail2ban
In order to activate the UFW firewall, issue the command:
sudo ufw enable
At this point, the firewall is active and will also start on a system reboot. However, there's one tiny issue: The firewall is now running and blocking all incoming traffic.
Enable SSH and http connections with these commands:
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
At this point the server should be set up and your app should be deployed to it... enjoy