Deployment Guidelines - CleverseAcademy/cd-compose-deployment GitHub Wiki
.dockerignore
Tip
For more help, visit the .dockerignore file reference guide at https://docs.docker.com/engine/reference/builder/#dockerignore-file
**/.DS_Store
**/.dockerignore
**/.env
**/.git
**/.gitignore
**/.project
**/.settings
**/.vs
**/.vscode
**/.next
**/.cache
**/*.*proj.user
**/docker-compose*
**/compose.y*
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/secrets.dev.yaml
**/values.dev.yaml
**/build
**/dist
LICENSE
README.md
# Include any files or directories that you don't want to be copied to your
# container here (e.g., local build artifacts, temporary files, etc.).
-
Dockerfile
-
Front-end:
Vite
, see cohort2-learnhub-vite/Dockerfile for a reference -
Back-end:
Express ↔️ Prisma ↔️ PostgreSQL
, see cohort2-learnhub-api/Dockerfile for a reference
-
Front-end:
-
compose.yml
Compose file must includes both front-end and back-end, along with all necessary databases.
See cd-compose-deployment/showcases/compose.yml for a reference.
-
.env.prod
DOCKERHUB_USERNAME=<your Docker Hub username> PROJECT_NAME=<project name>
docker compose up --build
The result should be look like this:
For sake of simplicity, we'll use shell environment variables extensively from now on.
Before continuing, set the DockerHub username and repository prefix as an environment variable with the following command:
export DOCKERHUB_USERNAME=<your Docker Hub username>
export PROJECT_NAME=<project name>
-
Front-end:
-web
-
Build docker image with the following command
docker build -t $DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.1 \ --build-arg=<necessary build arguments, e.g., VITE_API_HOST.> \ --build-arg=<build-argument-2-to-n> \ <./front-end/root/path>
i.e.
docker build -t $DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.1 \ --build-arg=VITE_API_HOST=http://localhost:8080 \ ./cohort2-learnhub-vite
-
Push to Docker Hub registry with the following command
docker push $DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.1
-
-
Back-end:
-api
-
Build docker image with following command
docker build -t $DOCKERHUB_USERNAME/$PROJECT_NAME-api:0.0.1 <./back-end/root/path>
-
Push to Docker Hub registry with the following command
docker push $DOCKERHUB_USERNAME/$PROJECT_NAME-api:0.0.1
-
Important
For ARM-powered machine, please contact Tiger for multi-platform supports
Or use a following template:
version: "3.8"
services:
webpage:
image: ${DOCKERHUB_USERNAME}/${PROJECT_NAME}-web:0.0.1
ports:
- 80:80
api:
image: ${DOCKERHUB_USERNAME}/${PROJECT_NAME}-api:0.0.1
secrets:
- source: node-env
target: /app/.env
ports:
- 8080:8080
# `depends_on` tells Docker Compose to start the database before your application.
depends_on:
db:
condition: service_healthy
db:
image: postgres:15
restart: always
user: postgres
volumes:
# The `db-data` volume persists the database data between container restarts.
- db-data:/var/lib/postgresql/data
secrets:
- db-password
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/db-password
expose:
- 5432
ports:
- 5432:5432
healthcheck:
test: ["CMD", "pg_isready"]
interval: 3s
timeout: 5s
retries: 5
volumes:
# The `db-data` volume persists the database data between container restarts.
db-data:
secrets:
# The `db-password` secret is used to set the database password.
db-password:
file: secrets/pg-password.txt
node-env:
file: secrets/node-env.txt
Then run:
docker compose --env-file=.env.prod up
The outcome should be the same as a previous run.
- Google: Compute Engine
- Vultr: Deploy New Server
- AWS: Amazon EC2
The most preferred cloud provider is AWS, as they offer a free tier.
For more information, please contact Tiger, Men, or A for further support.
- Launch a new instance.
Tip
If your cloud provider is AWS, click here to create a new instance in ap-southeast-1 region.
- If necessary, generate a 4096-bit RSA key pair using command below:
ssh-keygen -t rsa -b 4096 -C "<your_instance_name>"
Note
Typically, you can request key pairs from AWS by navigating to Network & Security > Key Pairs, so you don't need to manage key pairs from your end.
-
Configure firewall policy to DROP all incoming traffic by default, and allow only traffic matching following rules:
IP version Protocol Port Source Description AWS Type IPv4 TCP 22 0.0.0.0/0 SSH SSH IPv4 TCP 3000 0.0.0.0/0 Compose Deployment Custom TCP IPv4 TCP 80 0.0.0.0/0 Website service (Front-end) HTTP IPv4 TCP 8080 0.0.0.0/0 API service (Back-end) Custom TCP
Tip
For AWS, visit EC2 Security Groups, then navigate to a group matching to your instance.
Typically the security group's name is launch-wizard-1
Caution
❗️❗️❗️ ALWAYS ENSURE THAT FIREWALL RULES ARE PROPERLY CONFIGURED BEFORE PROCEEDING TO THE NEXT STEP
Otherwise, be prepared to say goodbye to your hard work for the previous two weeks with no meaning at all. And in the future, it will become more serious when your customer information gets leaked.
Before continuing, set the SSH private key path and the instance IP address as an environment variable with the following command:
export SSH_PRIVATE_KEY=</path/to/your/private/key/.pem/file>
export INSTANCE_IP=<your.server.I.P>
-
Start the ssh-agent in the backgound
eval "$(ssh-agent -s)"
-
Add your SSH private key to the
ssh-agent
ssh-add $SSH_PRIVATE_KEY
-
Use you SSH private key as an identity to connect to your server
For AWS:
ssh -i $SSH_PRIVATE_KEY admin@$INSTANCE_IP
For Vultr:
ssh -i $SSH_PRIVATE_KEY root@$INSTANCE_IP
After successfully ssh into a server, you should get a welcome message with information like the screenshot below.
-
Setup Docker on the server by follows the official documentation:
mkdir production && cd production
Generate cryptographic elliptic curve P-256 key pair for the deployer
service with the following command:
- Generate a private key
openssl genpkey -algorithm EC -out cd_priv.pem \ -pkeyopt ec_paramgen_curve:P-256 \ -pkeyopt ec_param_enc:named_curve
- Export public key from the previous step
openssl pkey -in cd_priv.pem -pubout -out cd_public.pem
Let's take a look with:
ls
It should be two files placed under a production
folder like a screenshot below:
-
Under a
services:
section, add adeployer
servicedeployer: image: cloudiana/compose-deployment:0.0.5 ports: - 3000:3000 environment: - CD_HOST_COMPOSE_WORKING_DIR=${PRODUCTION_DIR} env_file: - .env.prod volumes: - .env.prod:${PRODUCTION_DIR}/.env.prod - ./compose.yml:/bin/compose.yml - /var/run/docker.sock:/var/run/docker.sock secrets: - pubkey
-
Under a
secrets:
section, add apubkey
secretpubkey: file: cd_public.pem
scp -i $SSH_PRIVATE_KEY -r compose.yml .env.prod secrets admin@$INSTANCE_IP:~/production
set PRODUCTION_DIR
environment variables in remote instance with the following command:
echo "PRODUCTION_DIR=$(pwd)" >> .env.prod
For most relational database, e.g. PostgreSQL, it's necessary to setup schema before running our application.
Use the following command to setup the database schema with prisma db push
docker compose --env-file=.env.prod run api npx --yes prisma db push
The result should be look like screenshot below:
docker compose --env-file=.env.prod up
The result should be look like this:
Let's browse your website to see if things work as expected.
Service | Binding | Protocol + Hostname | Example |
---|---|---|---|
Front-End: webpage
|
TCP/80 |
http://server_ip
|
http://45.76.181.219/ |
Back-End: api
|
TCP/8080 |
http://server_ip :8080 |
http://45.76.181.219:8080/ |
-
Setup deployment private key locally by run the following command:
export CD_CLI_PRIVATE_KEY_PEM=$(ssh -i $SSH_PRIVATE_KEY admin@$INSTANCE_IP "cat ~/production/cd_priv.pem")
-
Use
@cloud-bombard/[email protected]
to test connectivity between the deployment service and client by run the following command:npx @cloud-bombard/[email protected] next-deployment --target=$INSTANCE_IP webpage
The result should be look like this
Let's say we found that cloudiana/learnhub-web:0.0.1
is not properly configured for remote host API. so we need to build a new image again.
We could use the same build command from Part One: Step 3
docker build -t $DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.2-rc0 \
--build-arg=VITE_API_HOST=http://45.76.181.219:8080 \
./cohort2-learnhub-vite
docker push $DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.2-rc0
After built and pushed a new image to Docker Hub repository successfully, we can deploy a new image from our local machine with the following command:
npx @cloud-bombard/[email protected] deploy \
--target=$INSTANCE_IP --priority=0 \
--image=$DOCKERHUB_USERNAME/$PROJECT_NAME-web:0.0.2-rc0 \
webpage
And check a next deployment being executed after roughly 4 minutes with:
npx @cloud-bombard/[email protected] next-deployment --target=$INSTANCE_IP webpage
The result of two npx @cloud-bombard/[email protected]
runs above should look like a screenshot below:
Name | Description | Where to get | Example |
---|---|---|---|
DOCKERHUB_TOKEN |
Docker Hub Personal Access Token | https://hub.docker.com/settings/security | dckr_pat_...secrets... |
CD_HOST |
Server Instance IP Address | Cloud provider, e.g. AWS. | 45.67.171.129 |
CD_CLI_PRIVATE_KEY_PEM |
Deployment private key generated from Part Three: Step 2 | ssh -i $SSH_PRIVATE_KEY admin@$INSTANCE_IP "cat ~/production/cd_priv.pem" | pbcopy |
-----BEGIN PRIVATE KEY----- ...secrets... -----END PRIVATE KEY----- |
Name | Description | Where to get | Example |
---|---|---|---|
DOCKERHUB_USERNAME |
Docker Hub username | https://hub.docker.com/u/`DOCKERHUB_USERNAME` | cloudiana |
Name | Description | Where to get | Example |
---|---|---|---|
CD_SERVICE |
Deployment service name | under services: section, i.e. showcases/compose.yml#L10. |
api |
IMAGE_NAME |
Docker image output name | echo $PROJECT_NAME-$CD_SERVICE |
learnhub-api |
And copy GitHub Workflows below to your .github/workflows
directory
name: Image build and deploy to server
on:
push:
branches: [main]
jobs:
build-docker-image:
runs-on: ubuntu-latest
outputs:
image: ${{ fromJson(steps.build.outputs.metadata)['image.name'] }}
steps:
- name: checkout
uses: actions/checkout@v4
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- run: |
echo "SHORTENED_SHA=${GITHUB_SHA:0:7}" >> $GITHUB_OUTPUT &&
echo "REF_TAG=$(sed -e "s/\//\./g" <<< $REF_NAME)" >> $GITHUB_OUTPUT
id: params
env:
GITHUB_SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
id: build
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ vars.DOCKERHUB_USERNAME }}/${{ vars.IMAGE_NAME }}:${{ steps.params.outputs.REF_TAG }}-${{steps.params.outputs.SHORTENED_SHA}}
request-deployment:
needs: [build-docker-image]
uses: CleverseAcademy/cd-compose-deployment/.github/workflows/deploy.yaml@main
with:
image: ${{ needs.build-docker-image.outputs.image }}
service_name: ${{ vars.CD_SERVICE }}
secrets:
host: ${{ secrets.CD_HOST }}
private_key: ${{ secrets.CD_CLI_PRIVATE_KEY_PEM }}