For a recent project at work I have used the Phoenix Framework, which is a quasi-MVC Web framework built on top of Elixir, a functional language running on top of the Erlang Virtual Machine. I made it a Single-Page Application, with React as UI library, Redux for managing application state, and TypeScript for type checking.

For persistence, I would normally go with PostgreSQL, but the client only agreed on Phoenix under the premise that we use MySQL on Amazon Redshift. The whole application was to be deployed on an Amazon EC2 instance. My operating system of choice for production servers is Debian GNU/Linux, but the client insisted that we use Amazon Linux, a proprietary variant of CentOS.

In this tutorial I will try to recreate what I did to deploy the application to EC2 using edeliver.

Prerequisites

For the purpose of writing this article, I set up a minimal CentOS 7 machine on VirtualBox. The overall process was almost the same as on Amazon Linux, except for firewall settings, which in case of EC2 have to be set in the AWS Management Console. Also, as I am just a poor college student, I will be running the DB server locally rather than in the cloud. I assume you are running a *NIX operating system (i.e. OS X, GNU/Linux, or FreeBSD,) and have Erlang/OTP, Elixir 1.5+, Node.js, MariaDB, and Git installed on your development machine.

Creating a new Phoenix application

Let’s start out by creating a new Phoenix 1.4 application. First, install Phoenix 1.4 Release Candidate 3, which is the latest version at the time of this writing.

Run these commands in a terminal on your development machine:

mix archive.uninstall phx_new
mix archive.install hex phx_new 1.4.0-rc.3

Then cd into your working folder and create a new Phoenix application. I think Japanese words are the best pick for code names, so I’ll call the demo application Harajuku (Japanese: 原宿 harajuku,) which is the name of a party district in Tokyo:

mkdir -p ~/working/deployment_demo
cd !$
mix phx.new harajuku --database mysql

The generator will prompt you if it should fetch and install dependencies. Press Enter to install all dependencies of the Elixir application as well as Node.js packages required by Webpack. cd into the project directory and initialize a Git repository:

cd harajuku
git init
git add .
git commit -m "Initial commit"

Now, set your DB credentials in config/dev.exs. You will have to create a MariaDB user with the rights to create databases first, which is covered later on in the article:

# ...
# (Part of file omitted for brevity)

# Configure your database
config :harajuku, Harajuku.Repo,
  username: "harajuku",
  password: "harajuku",
  database: "harajuku_dev",
  hostname: "localhost",
  pool_size: 10

Then create your database:

[~/working/deployment_demo/harajuku] $ mix ecto.create
Compiling 13 files (.ex)
Generated harajuku app
The database for Harajuku.Repo has been created

Then run the development server:

[~/working/deployment_demo/harajuku] $ mix phx.server
Compiling 13 files (.ex)
Generated harajuku app
[info] Running HarajukuWeb.Endpoint with cowboy 2.5.0 at http://localhost:4000

Webpack is watching the files…

Then open your browser and navigate to localhost:4000. You should see the standard Phoenix welcome screen:

Image: 螢幕快照 2018-10-26 16.43.00.png (no description provided)

Phoenix programming is outside the scope of this article, so let’s proceed with deployment.

Set up deployment server

On the deployment machine, you will need to install OTP, Elixir, MariaDB/MySQL, and Node.js.

Compile and install Erlang/OTP 21.1

First, install Erlang/OTP, which is Elixir’s only dependency. The instructions for compiling Erlang/OTP on Amazon Linux come from this article in Japanese.

SSH into your production server and install Erlang’s build dependencies:

sudo yum -y install git ncurses-devel openssl openssl-devel gcc-c++ unixODBC unixODBC-devel fop java-1.6.0-openjdk-devel wget
sudo yum -y groupinstall "Development Tools"

Then, download the latest OTP source tarball (version 21.1 as of this writing):

mkdir ~/erlang && cd ~/erlang
wget http://erlang.org/download/otp_src_21.1.tar.gz

Unpack the source code tarball and cd into the source code folder:

tar xzf otp_src_21.1.tar.gz
cd otp_src_21.1

The installation is very straightforward. The ./configure command will warn you that the wxWidgets library is not available, but since it is a GUI library, you won’t need it on a server.

./configure
make
sudo make install

At this point, you should be able to run the Erlang shell (press Ctrl-C twice to exit):

[deploy@centos otp_src_21.1]$ erl
Erlang/OTP 21 [erts-10.1] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1] [hipe]

Eshell V10.1  (abort with ^G)
1>

When you make sure that Erlang/OTP is up and running, you can delete the source code to save disk space:

cd && rm -rf ~/erlang

Install Elixir 1.7.4

The original article in Japanese instructs to compile Elixir from source. However, I ran into issues due to the fact that the compilation required quite a lot of RAM and the EC2 instance we were running was the cheapest variant with only 1 GB of RAM. I opted for the precompiled package instead.

The following will download and install precompiled packages in /usr/local/elixir. At the time of this writing, the latest version is 1.7.4.

ELIXIR_VERSION="1.7.4"
cd /tmp && \
wget https://github.com/elixir-lang/elixir/releases/download/v${ELIXIR_VERSION}/Precompiled.zip && \
sudo unzip -d /usr/local/elixir -x Precompiled.zip && \
rm -f /tmp/Precompiled.zip

Add /usr/local/elixir/bin to your PATH by appending an export statement to your $HOME/.bashrc file. Then source the file:

echo 'export PATH="$PATH:/usr/local/elixir/bin"' >> ~/.bashrc
. ~/.bashrc

You should now be able to run iex:

[deploy@centos ~]$ iex
Erlang/OTP 21 [erts-10.1] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>

Install Node.js 8.12.0

Node.js is required on the build server to compile and optimize static assets, such as CSS . The easiest way to install an arbitrary version of Node.js is using NVM

wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash
. ~/.bashrc

If you run command -v nvm in your shell now, it should output nvm:

[deploy@centos ~]$ command -v nvm
nvm

Now you can proceed to install Node.js. For production, you should choose the latest LTS version, which at the time of this writing is 8.12.0:

nvm install 8.12.0

Check that node is installed:

[deploy@centos ~]$ node -v
v8.12.0

Install MariaDB 5.5.6

Install MariaDB and start it as a service:

sudo yum -y install mariadb mariadb-server
sudo systemctl enable mariadb
sudo systemctl start mariadb

Now you should be able to use the mysql command client:

[deploy@centos ~]$ mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.60-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

Create production database

Connect to MariaDB as root using sudo mysql. Then paste the following query to create a user and a database.

CREATE USER 'harajuku'@'localhost' IDENTIFIED BY 'harajuku';
GRANT ALL PRIVILEGES ON *.* to 'harajuku'@'localhost' IDENTIFIED BY 'harajuku';
CREATE DATABASE harajuku_prod CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Now that our deployment server is up and running, let’s prepare our application for deployment.

Prepare Phoenix application for deployment

This section is based on the brilliant article Deploy Early and Often: Deploying Phoenix with Edeliver and Distillery (Part Two) by Zek Interactive.

Start by adding Edeliver and Distillery to your application’s dependencies. Although the latest version of Edeliver as of this writing is 1.6.0, I have experienced some issues with the latest version, so for now I’ll stick with the versions that I used to deploy my blog, 2137.io. In mix.exs, add the following lines inside def deps:

{:edeliver, "~> 1.5.0"},
{:distillery, "~> 1.5.3"}

Add .deliver/releases/ folder to the project’s .gitignore:

echo ".deliver/releases/" >> .gitignore

Then init a .deliver/config file. This is where you will be setting most of deployment-related information and this version is based on the file I used to deploy my blog. Don’t just blindly copy and paste this file, but fill in the credentials of your server.

APP="harajuku" # Set this to the name of your application

BUILD_HOST="192.168.56.102" # IP of my VM
BUILD_USER="deploy"
BUILD_AT="/tmp/edeliver/$APP/builds"
AUTO_VERSION=revision

RELEASE_DIR="/tmp/edeliver/$APP/builds/_build/prod/rel/$APP"
RELEASE_STORE="${BUILD_USER}@${BUILD_HOST}:~/releases/"

# prevent re-installing node modules; this defaults to "."
GIT_CLEAN_PATHS="_build rel priv/static"

PRODUCTION_HOSTS="192.168.56.102"
PRODUCTION_USER="deploy"
DELIVER_TO="/home/deploy"

# For *Phoenix* projects, symlink prod.secret.exs to our tmp source
pre_erlang_get_and_update_deps() {
  local _prod_secret_path="/home/deploy/secret/prod.secret.exs"
  if [ "$TARGET_MIX_ENV" = "prod" ]; then
    __sync_remote "
      mkdir -p '$BUILD_AT' /home/deploy/releases
      ln -sfn '$_prod_secret_path' '$BUILD_AT/config/prod.secret.exs'
    "
  fi
}

pre_erlang_clean_compile() {
  status "Running npm install"
    __sync_remote "
      [ -f ~/.profile ] && source ~/.profile
      set -e
      cd '$BUILD_AT'/assets
      npm install
    "

  status "Compiling assets"
    __sync_remote "
      [ -f ~/.profile ] && source ~/.profile
      set -e
      cd '$BUILD_AT'/assets
      node_modules/.bin/webpack --mode production --silent
    "

  status "Running phoenix.digest" # log output prepended with "----->"
  __sync_remote " # runs the commands on the build host
    [ -f ~/.profile ] && source ~/.profile # load profile (optional)
    set -e # fail if any command fails (recommended)
    cd '$BUILD_AT' # enter the build directory on the build host (required)
    # prepare something
    mkdir -p priv/static # required by the phoenix.digest task
    # run your custom task
    APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest $SILENCE
    APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phx.digest.clean $SILENCE
  "
}

Commit these changes:

git add -A
git commit -m "Set up edeliver"

Now hard-code a port for your application’s endpoint in config/prod.exs and uncomment the line that says config :phoenix, :serve_endpoints, true:

config :harajuku, HarajukuWeb.Endpoint,
  http: [port: 2137], # or a port of your choice
  url: [host: "example.com", port: 80],
  cache_static_manifest: "priv/static/cache_manifest.json"
#
config :phoenix, :serve_endpoints, true

Then set up Distillery:

mix release.init
git add -A && git commit -m "Set up distillery"

Create a secrets file

Now, SSH into your server again and create the $HOME/secret/prod.secret.exs file.

mkdir ~/secret
vim ~/secret/prod.secret.exs

In this file, you have to set your application’s secret key base and DB credentials. You can generate a pseudorandom key base using mix phx.gen.secret (you have to run it on a machine with Phoenix installed, e.g. your development machine,) which you can paste into the file.

# File: /home/deploy/secret/prod.secret.exs
use Mix.Config

config :harajuku, HarajukuWeb.Endpoint, secret_key_base: "<<PASTE YOUR SECRET KEY BASE HERE>>"

config :harajuku, Harajuku.Repo,
  username: "harajuku",
  password: "harajuku",
  database: "harajuku_prod",
  pool_size: 20

First release

At this point, you should be able to build your first release.

[~/working/deployment_demo/harajuku] $ mix edeliver build release production                                                                                1

BUILDING RELEASE OF HARAJUKU APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: deploy@192.168.56.102
-----> Resetting remote hosts to 2613f53ec473000bb2babdeb9bd805588a9082d2
-----> Cleaning generated files from last build
-----> Authorizing release store on build host
-----> Fetching / Updating dependencies
-----> Running npm install
-----> Compiling assets
-----> Running phoenix.digest
-----> Compiling sources
-----> Generating release
-----> Copying release 0.1.0+2613f53 to remote release store

RELEASE BUILD OF HARAJUKU WAS SUCCESSFUL!

Then, deploy your first release to production.

[~/working/deployment_demo/harajuku] $ mix edeliver deploy release to production

DEPLOYING RELEASE OF HARAJUKU APP TO PRODUCTION HOSTS

-----> Authorizing hosts
-----> Authorizing release store host
-----> Authorizing deploy hosts on release store
-----> Uploading archive of release 0.1.0+2613f53 from remote release store
-----> Extracting archive harajuku_0.1.0+2613f53.tar.gz

DEPLOYED RELEASE TO PRODUCTION!

Now, start your application server:

[~/working/deployment_demo/harajuku] $ mix edeliver start production

EDELIVER HARAJUKU WITH START COMMAND

-----> starting production servers

production node:

  user    : deploy
  host    : 192.168.56.102
  path    : /home/deploy
  response:

START DONE!

At this point your server should be working on the deployment server. You can check it by running curl localhost:2137 in the SSH shell. However, this port won’t be available to the outside as it is blocked by the system firewall. I unblocked it using:

# This won't work on AWS EC2, use Management Console instead
sudo firewall-cmd --zone=public --add-port=2137/tcp --permanent
sudo firewall-cmd --zone=public --add-service=http --permanent
sudo firewall-cmd --reload

Now, navigate to your server’s IP address on port 2137 in a Web browser:

Image: 螢幕快照 2018-10-26 18.17.46.png (no description provided)

Install Nginx

To get the application to run on ports 80 and 443, you might feel tempted to run the application as root, but obviously, this is not the best option. You could also redirect ports using iptables or your firewall, but I haven’t managed to do it on AWS. The most common option is to use Nginx. Unfortunately, the configuration provided in the article doesn’t work out of the box due to differences in defaults between Debian and CentOS.

On your production server, install and start Nginx:

sudo yum -y install epel-release
sudo yum -y install nginx
sudo systemctl enable nginx
sudo systemctl start nginx

You should now be able to visit the default Nginx landing page:

Image: 螢幕快照 2018-10-26 18.31.54.png (no description provided)

Now create sites-available and sites-enabled folders in /etc/nginx and stub a configuration file:

sudo mkdir -p /etc/nginx/sites-{available,enabled}
sudo touch /etc/nginx/sites-available/harajuku.conf
sudo ln -s /etc/nginx/sites-available/harajuku.conf /etc/nginx/sites-enabled/harajuku.conf

Comment or remove existing server {} section in /etc/nginx/nginx.conf and add these lines within http {} block:

include /etc/nginx/sites-enabled/*.conf;
server_names_hash_bucket_size 64;

Then edit /etc/nginx/sites-available/harajuku.conf to look like this. The Upgrade and Connection headers are necessary for establishing WebSocket connections. I am not an Nginx expert, but this is the minimal config that worked for me:

# File /etc/nginx/sites-available/harajuku.conf
server {
  listen 80 default_server;
  listen [::]:80 default_server;
  server_name _;
  location / {
    proxy_pass http://localhost:2137;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_http_version 1.1;
  }
}

Check if your configuration has any syntax errors:

[deploy@centos ~]$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Then restart Nginx using sudo systemctl restart nginx. You should be able

References

  1. @fullmated. 2018. Amazon Linux ni Elixir+Phoenix wo insutōru Amazon LinuxにElixir+Phoenixをインストール [Installing Elixir+Phoenix on Amazon Linux]. online (accessed 26th Oct. 2018)

  2. https://gist.github.com/Ch4s3/77f5946972f7677b0ab4e3a9d9e22729

  3. https://stackoverflow.com/questions/3513773/change-mysql-default-character-set-to-utf-8-in-my-cnf

  4. https://medium.com/@zek/deploy-early-and-often-deploying-phoenix-with-edeliver-and-distillery-part-one-5e91cac8d4bd

  5. https://medium.com/@zek/deploy-early-and-often-deploying-phoenix-with-edeliver-and-distillery-part-two-f361ef36aa10

  6. https://stackoverflow.com/questions/24729024/open-firewall-port-on-centos-7

  7. https://www.tecmint.com/install-nginx-on-centos-7/