Production Setup - fnrfarid/excel-rma GitHub Wiki

Start

Muake changes to URLs as required. Configure A records for portainer, traefik, various apps:

• Add A record for stag-portainer.excelbd.com pointing the public IP
• Add A record for stag-traefik.excelbd.com pointing the public IP
• Add A record for stag-erp.com pointing the public IP
• Add A record for stag-rma.excelbd.com pointing the public IP
• Add A record for stag-warranty.excelbd.com pointing the public IP
• Add A record for stag-pos.excelbd.com pointing the public IP

Install Ubuntu Server with Docker Swarm Mode

//Use root user to setup basic things

Starting the docker swarm installation

export USE_HOSTNAME=stag-docker.excelbd.com
echo $USE_HOSTNAME > /etc/hostname
hostname -F /etc/hostname
apt update && apt upgrade -y
apt install curl
curl -fsSL get.docker.com -o get-docker.sh
CHANNEL=stable sh get-docker.sh
rm get-docker.sh
docker swarm init
docker ps
//Post-installation steps for Linux: Manage Docker as a non-root user
sudo groupadd docker
sudo usermod -aG docker $USER
//Log out and log back in so that your group membership is re-evaluated.

Traefik Proxy with HTTPS

docker network create --driver=overlay traefik-public
export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add traefik-public.traefik-public-certificates=true $NODE_ID
export [email protected]
export DOMAIN=stag-traefik.excelbd.com
export USERNAME=admin
//AVOID all sorts of characters [~!@#$%, etc.] as a pass, sometimes gives error
export PASSWORD=Admin123
export HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)
//Deploy the stack
curl -L dockerswarm.rocks/traefik-host.yml -o traefik-host.yml
docker stack deploy -c traefik-host.yml traefik
docker stack ps traefik

Portainer web user interface

export DOMAIN=stag-portainer.excelbd.com 
export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add portainer.portainer-data=true $NODE_ID
//Create the Docker Compose file
curl -L dockerswarm.rocks/portainer.yml -o portainer.yml
//Deploy the stack
// to update/install the portainer to the community edition, edit the yaml and change portainer image tag to portainer/portainer-ce and deploy the stack.
docker stack deploy -c portainer.yml portainer
docker stack ps portainer

Login to portainer[setup password]. Follow the rest of the steps using Portainer console.

Create Config

Configs > Add Config give name with this content > frappe-mariadb-config

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

[mysql]
default-character-set = utf8mb4

Create Secret

Secret > Add Secret > frappe-mariadb-root-password LongSecretPassword // Set your pass here.

Deploy MariaDB Replication

Stacks > Add Stacks > frappe-mariadb

Change the password in the yaml or make it environment variable

Add the yaml configuration from

version: "3.7"

services:
  mariadb-master:
    image: 'bitnami/mariadb:10.6.7'
    deploy:
      restart_policy:
        condition: on-failure
          
    networks:
      - frappe-network
    secrets:
      - frappe-mariadb-root-password
    volumes:
      - 'mariadb_master_data:/bitnami/mariadb'
    environment:
      - MARIADB_REPLICATION_MODE=master
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD_FILE=/run/secrets/frappe-mariadb-root-password
      - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/frappe-mariadb-root-password
      - MARIADB_CHARACTER_SET= utf8mb4
      - MARIADB_COLLATE= utf8mb4_unicode_ci
      - MARIADB_ROOT_PASSWORD= StagAdmin#123
      - MARIADB_EXTRA_FLAGS= --skip-character-set-client-handshake --skip-innodb-read-only-compressed

  mariadb-slave:
    image: 'bitnami/mariadb:10.6.7'
    deploy:
      restart_policy:
        condition: on-failure
    networks:
      - frappe-network
    secrets:
      - frappe-mariadb-root-password
    volumes:
      - 'mariadb_slave_data:/bitnami/mariadb'
    environment:
      - MARIADB_REPLICATION_MODE=slave
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD_FILE=/run/secrets/frappe-mariadb-root-password
      - MARIADB_MASTER_HOST=mariadb-master
      - MARIADB_MASTER_PORT_NUMBER=3306
      - MARIADB_MASTER_ROOT_PASSWORD_FILE=/run/secrets/frappe-mariadb-root-password
      - MARIADB_CHARACTER_SET= utf8mb4
      - MARIADB_COLLATE= utf8mb4_unicode_ci
      - MARIADB_ROOT_PASSWORD= StagAdmin#123
      - MARIADB_EXTRA_FLAGS= --skip-character-set-client-handshake --skip-innodb-read-only-compressed

volumes:
  mariadb_master_data:
  mariadb_slave_data:

configs:
  frappe-mariadb-config:
    external: true

secrets:
  frappe-mariadb-root-password:
    external: true

networks:
  frappe-network:
    name: frappe-network
    attachable: true

Deploy Frappe/ERPNext

Stacks > Add Stacks > frappe-bench-v12

Add the yaml configuration and Use environment variables mentioned below:

• FRAPPE_VERSION= v12.13.0
• EXCEL_ERPNEXT_VERSION= v12.10.0
• MARIADB_HOST= frappe-mariadb_mariadb-master
• SITES variable is list of sites in back tick and separated by comma
• SITES=

`stag-erp.excelbd.com`, `stag-erp2.excelbd.com`

version: "3.7"

services:
  redis-cache:
    image: redis:latest
    volumes:
      - redis-cache-vol:/data
    deploy:
      restart_policy:
        condition: on-failure
    networks:
      - frappe-network

  redis-queue:
    image: redis:latest
    volumes:
      - redis-queue-vol:/data
    deploy:
      restart_policy:
        condition: on-failure
    networks:
      - frappe-network

  redis-socketio:
    image: redis:latest
    volumes:
      - redis-socketio-vol:/data
    deploy:
      restart_policy:
        condition: on-failure
    networks:
      - frappe-network

  erpnext-nginx:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-nginx:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    environment:
      - FRAPPE_PY=erpnext-python
      - FRAPPE_PY_PORT=8000
      - FRAPPE_SOCKETIO=frappe-socketio
      - SOCKETIO_PORT=9000
    volumes:
      - sites-vol:/var/www/html/sites:rw
      - assets-vol:/assets:rw
    networks:
      - frappe-network
      - traefik-public
    deploy:
      restart_policy:
        condition: on-failure
      labels:
        - "traefik.docker.network=traefik-public"
        - "traefik.enable=true"
        - "traefik.constraint-label=traefik-public"
        - "traefik.http.routers.erpnext-nginx.rule=Host(${SITES?Variable SITES not set})"
        - "traefik.http.routers.erpnext-nginx.entrypoints=http"
        - "traefik.http.routers.erpnext-nginx.middlewares=https-redirect"
        - "traefik.http.routers.erpnext-nginx-https.rule=Host(${SITES?Variable SITES not set})"
        - "traefik.http.routers.erpnext-nginx-https.entrypoints=https"
        - "traefik.http.routers.erpnext-nginx-https.tls=true"
        - "traefik.http.routers.erpnext-nginx-https.tls.certresolver=le"
        - "traefik.http.services.erpnext-nginx.loadbalancer.server.port=80"

  erpnext-python:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    deploy:
      restart_policy:
        condition: on-failure
    environment:
      - MARIADB_HOST=${MARIADB_HOST?Variable MARIADB_HOST not set}
      - REDIS_CACHE=redis-cache:6379
      - REDIS_QUEUE=redis-queue:6379
      - REDIS_SOCKETIO=redis-socketio:6379
      - SOCKETIO_PORT=9000
      - AUTO_MIGRATE=1
      - PASS_MAX_DAYS=99999
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
      - assets-vol:/home/frappe/frappe-bench/sites/assets:rw
    networks:
      - frappe-network

  frappe-socketio:
    image: frappe/frappe-socketio:${FRAPPE_VERSION?Variable FRAPPE_VERSION not set}
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
    networks:
      - frappe-network

  erpnext-worker-default:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    environment:
      - PASS_MAX_DAYS=9999
    deploy:
      restart_policy:
        condition: on-failure
    command: worker
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
    networks:
      - frappe-network

  erpnext-worker-short:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    deploy:
      restart_policy:
        condition: on-failure
    command: worker
    environment:
      - WORKER_TYPE=short
      - PASS_MAX_DAYS=9999
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
    networks:
      - frappe-network

  erpnext-worker-long:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    deploy:
      restart_policy:
        condition: on-failure
    command: worker
    environment:
      - WORKER_TYPE=long
      - PASS_MAX_DAYS=9999
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
    networks:
      - frappe-network

  frappe-schedule:
    image: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:${EXCEL_ERPNEXT_VERSION?Variable EXCEL_ERPNEXT_VERSION not set}
    environment:
      - PASS_MAX_DAYS=9999
    deploy:
      restart_policy:
        condition: on-failure
    command: schedule
    volumes:
      - sites-vol:/home/frappe/frappe-bench/sites:rw
    networks:
      - frappe-network

volumes:
  redis-cache-vol:
  redis-queue-vol:
  redis-socketio-vol:
  assets-vol:
  sites-vol:

networks:
  traefik-public:
    external: true
  frappe-network:
    external: true

Create new site job to deploy the ERPNext Site

Containers > Add Container > stag-erp.excelbd.com

Env variables:

• MYSQL_ROOT_PASSWORD=LongSecretPassword
• SITE_NAME=stag-erp.excelbd.com
• ADMIN_PASSWORD= LongSecretPassword
• INSTALL_APPS=erpnext,excel_erpnext

Other variables

• Select Image docker io frappe/erpnext-worker: registry.gitlab.com/castlecraft/excel_erpnext/excel-erpnext-worker:v12.10.0
• Set command : new
• Set Entry Point: docker-entrypoint.sh
• Set Container: /home/frappe/frappe-bench/sites
• Set Volume: frappe-bench-v12_sites-vol - local
• Select network: frappe-network

Start container

Permission Error

Go to the log and see the Installation Progress. If you see permission error go to the container list and open terminal from frappe-bench-v12_erpnext-python. Run

ls-l
cd ..
chown -R frappe:frappe sites

and start Deploy the container again.

Getting started with RMA Server [Custom Apps Installations]

Add mongodb Stack

Add the yaml configuration from mongodb Stack yaml Use Environment variables:

• MONGODB_ROOT_PASSWORD=SecretPassword
• MONGODB_USERNAME=rma-server
• MONGODB_PASSWORD=SecretPassword
• MONGODB_DATABASE-rma-server

version: '3.7'

services:
  global-mongodb:
    image: 'bitnami/mongodb:4.2.8'
    environment:
      - "MONGODB_ROOT_PASSWORD=${MONGODB_ROOT_PASSWORD}"
      - "MONGODB_USERNAME=${MONGODB_USERNAME}"
      - "MONGODB_PASSWORD=${MONGODB_PASSWORD}"
      - "MONGODB_DATABASE=${MONGODB_DATABASE}"
    volumes:
      - 'mongodb-vol:/bitnami/mongodb'
    networks:
      - mongodb-network
    deploy:
      restart_policy:
        condition: on-failure

volumes:
  mongodb-vol:

networks:
  mongodb-network:
    name: mongodb-network
    attachable: true

Let the MongoBD Database run completely and from the terminal:

Add cache-db database credentials:

Execute following command in mongodb container. Change the SecretPassword to desired password.

mongo cache-db --port 27017 -u root -p SecretPassword --authenticationDatabase admin --eval "db.createUser({user: 'cache-db', pwd: 'SecretPassword', roles:[{role:'dbOwner', db: 'cache-db'}]});"

Add rma-server Stack and Deploy

Add the yaml configuration from RMA SERVER Stack yaml, Environment variables:

• DB_NAME=rma-server
• RMA_SERVER_VERSION=latest or 1.2.2
• RMA_WARRANTY_VERSION=latest or 1.1.1
• RMA_FRONTEND_VERSION= latest or 1.2.1
• RMA_POS_VERSION=latest or 1.0.0
• DB_USER=rma-server
• DB_PASSWORD=SecretPassword
• CACHE_DB_NAME=cache-db
• CACHE_DB_USER=cache-db
• CACHE_DB_PASSWORD=SecretPassword
• SITE=stag-rma.excelbd.com
• WARRANTY_SITE=stag-warranty.excelbd.com
• POS_SITE=stag-pos.excelbd.com

version: "3.7"

services:
  rma-frontend:
    image: registry.gitlab.com/castlecraft/excel-rma/rma-frontend:${RMA_FRONTEND_VERSION?Variable RMA_FRONTEND_VERSION not set}
    environment:
      - API_HOST=rma-server
      - API_PORT=8800
    networks:
      - mongodb-network
      - traefik-public
    deploy:
      restart_policy:
        condition: on-failure
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-public"
        - "traefik.constraint-label=traefik-public"
        - "traefik.http.routers.rma-frontend.rule=Host(`${SITE?Variable SITE not set}`)"
        - "traefik.http.routers.rma-frontend.entrypoints=http"
        - "traefik.http.routers.rma-frontend.middlewares=https-redirect"
        - "traefik.http.middlewares.rma-frontend.headers.contentTypeNosniff=true"
        - "traefik.http.routers.rma-frontend-https.rule=Host(`${SITE?Variable SITE not set}`)"
        - "traefik.http.routers.rma-frontend-https.entrypoints=https"
        - "traefik.http.routers.rma-frontend-https.tls=true"
        - "traefik.http.routers.rma-frontend-https.tls.certresolver=le"
        - "traefik.http.services.rma-frontend.loadbalancer.server.port=8080"

  rma-warranty:
    image: registry.gitlab.com/castlecraft/excel-rma/rma-warranty:${RMA_WARRANTY_VERSION?Variable RMA_WARRANTY_VERSION not set}
    environment:
      - API_HOST=rma-server
      - API_PORT=8800
    networks:
      - mongodb-network
      - traefik-public
    deploy:
      restart_policy:
        condition: on-failure
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-public"
        - "traefik.constraint-label=traefik-public"
        - "traefik.http.routers.rma-warranty.rule=Host(`${WARRANTY_SITE?Variable WARRANTY_SITE not set}`)"
        - "traefik.http.routers.rma-warranty.entrypoints=http"
        - "traefik.http.routers.rma-warranty.middlewares=https-redirect"
        - "traefik.http.middlewares.rma-warranty.headers.contentTypeNosniff=true"
        - "traefik.http.routers.rma-warranty-https.rule=Host(`${WARRANTY_SITE?Variable WARRANTY_SITE not set}`)"
        - "traefik.http.routers.rma-warranty-https.entrypoints=https"
        - "traefik.http.routers.rma-warranty-https.tls=true"
        - "traefik.http.routers.rma-warranty-https.tls.certresolver=le"
        - "traefik.http.services.rma-warranty.loadbalancer.server.port=8080"

  rma-pos:
    image: registry.gitlab.com/castlecraft/excel-rma/rma-pos:${RMA_POS_VERSION?Variable RMA_POS_VERSION not set}
    environment:
      - API_HOST=rma-server
      - API_PORT=8800
    networks:
      - mongodb-network
      - traefik-public
    deploy:
      restart_policy:
        condition: on-failure
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-public"
        - "traefik.constraint-label=traefik-public"
        - "traefik.http.routers.rma-pos.rule=Host(`${POS_SITE?Variable POS_SITE not set}`)"
        - "traefik.http.routers.rma-pos.entrypoints=http"
        - "traefik.http.routers.rma-pos.middlewares=https-redirect"
        - "traefik.http.middlewares.rma-pos.headers.contentTypeNosniff=true"
        - "traefik.http.routers.rma-pos-https.rule=Host(`${POS_SITE?Variable POS_SITE not set}`)"
        - "traefik.http.routers.rma-pos-https.entrypoints=https"
        - "traefik.http.routers.rma-pos-https.tls=true"
        - "traefik.http.routers.rma-pos-https.tls.certresolver=le"
        - "traefik.http.services.rma-pos.loadbalancer.server.port=8080"

  rma-server:
    image: registry.gitlab.com/castlecraft/excel-rma/rma-server:${RMA_SERVER_VERSION?Variable RMA_SERVER_VERSION not set}
    deploy:
      restart_policy:
        condition: on-failure
    environment:
      - DB_HOST=global-mongodb
      - DB_NAME=${SERVER_DB}
      - DB_PASSWORD=${SERVER_DB_PASSWORD}
      - DB_USER=${SERVER_USER}
      - CACHE_DB_NAME=${CACHE_DB}
      - CACHE_DB_PASSWORD=${CACHE_DB_PASSWORD}
      - CACHE_DB_USER=${CACHE_USER}
      - NODE_ENV=production
      - NODE_OPTIONS=--max-old-space-size=4096
      - AGENDA_JOBS_CONCURRENCY=1
    networks:
      - mongodb-network

networks:
  traefik-public:
    external: true
  mongodb-network:
    external: true

Configure the rma-server connection to ERPNext

Make a post request using RESTer extension from Google Chrome Store then copy this json on the body and select json from the right corner menu.

{
  "appURL" : "https://stag-rma.excelbd.com",
  "warrantyAppURL" : "https://stag-warranty.excelbd.com",
  "posAppURL" : "https://stag-pos.excelbd.com",
  "frontendClientId" : "XXXXXXXXXXXXXXXXXXX",
  "backendClientId" : "XXXXXXXXXXXXXXXXXXX",
  "serviceAccountUser" : "[email protected]",
  "serviceAccountSecret" : "XXXXXXXXXXXXXXXXXXX",
  "authServerURL" : "https://stag-erp.excelbd.com",
  "serviceAccountApiKey" : "XXXXXXXXXXXXXXXXXXX",
  "serviceAccountApiSecret" : "XXXXXXXXXXXXXXXXXXX"
}

To get the backendClientId: Login to stag-erp.example.com as System Admin user, go to oAuth Client List to add a Client.

• Name: RMA Server
• Skip Authorization
• URL+Redirected URIs: https://stag-rma.excelbd.com/api/direct/callback

The app client ID is the backendClientId.

To get the serviceAccountUser: We need to add a System Admin user named service_account using [email protected] as the email. Whatever the email address name is, that should be there on the json body as the serviceAccountUser. The account password is the serviceAccountSecret.

To get the serviceAccountUser: Below the service account user details, under API Access, generate the key and save the pop up password. This should be the serviceAccountApiSecret. The key is serviceAccountApiKey .

To get the frontendClientID: Create another oAuth client.

• Name: RMA Client
• Skip Authorization
• Redirected URIs:

https://stag-rma.excelbd.com/callback https://stag-rma.excelbd.com/silent-refresh.html https://stag-warranty.excelbd.com/callback https://stag-warranty.excelbd.com/silent-refresh.html https://stag-pos.excelbd.com/callback https://stag-pos.excelbd.com/silent-refresh.html

• Default URI: https://stag-rma.excelbd.com/callback
• Advance Settings: Grant Type> Implicit, Response Type> Token

The app client ID is the frontendClientID.

Once all the variables are collected, the RESTer Method should be POST and the URL should be https://stag-rma.excelbd.com/api/setup

Under the header section it should be Name> Content-Type, Value> application/json

Now, sending the request should be successful. To complete the connection, now we need to add a social login key. Go to Social Login Key list in ERPNext and create a key.

• Not enabled
• Login Provider: Frappe
• Base URL: https://stag-erp.excelbd.com

Now stag-rma.excelbd.com should allow me to login via Frappe.

In case there's any mistake in MongoDB variables/RESTer variables

• We can't run the RESTer setup more than once. So if any mistake
• Delete the stopped Containers/Volumes/Associated Networks/Services of MongoDB
• Delete the Stack mongodb
• From server ssh terminal docker system prune -f && docker volume prune -f
• This will clean things and reset passwords
• Redeploy the stack mongodb & rma-server
• Setup the RESTer connection again as everything was reset.

Setup Webhooks

Create another POST request using RESTer. Make Sure to logout from ERPNext/RMA-Server completly and login again from incognito to avoid errors.

• Method: POST
• URL: https://stag-rma.excelbd.com/api/settings/v1/setup_webhooks
• Header: Authorization
• Value: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Bearer token can be collected via this method:

• Login to rma.excelbd.com,
• Open browser console by ctrl+shift+j then Application tab>Storage>IndexedDB>excel-rma>keyvaluepair
• the access token is the Bearer token variable.

Posting this will create all the webhooks and then we will have to setup the company and standard selling_price only from RMA-Server settings.

After that logout and after 2 minutes login and setup the Debtor Account/Transfer Warehouse/Territory mapping etc.

If we face any error on login/redirect please check your client and API credentials and try to setup again. Check this section.