Containerizing ERPNext with Docker Compose

Some History

A few weeks ago, I worked on installing and setting up an enterprise resource planning system (ERPNext) for a company. I had worked on this a few times previously so I had a couple of pain points that I wanted to resolve this time around. The primary one was to ease the installation, backup, and restoration process. The usual way of installing ERPNext previously involved booting up a linux server, downloading and running a bunch of scripts until finally I could use their bench cli to create a new site. It was a lengthy and error prone process that often required some debugging due to machine differences and odd unautomated things like LetsEncrypt SSL certificates. However, during the latest installation cycle, I was pleased to see that they had upgraded their installation instructions to now use docker containers which would solve the not on my machine problem that previously was painful. Additionally, since their setup utilized docker compose along with a bunch of additional compose files you could tack on, configuration and reproduction was simple (..as long as you understood the docker ecosystem). I had used docker before for a couple of things, but this seemed like the perfect opportunity to learn and make notes while at it so I could share it here. I’m going to write a more abstract blog on docker compose to go with this (and put the link somewhere here when its done) but for the moment heres a short guide to a containerized installation of ERP Next along with brief explanations on how things are working.

Summary

Step 1: Get the Repo

Clone or fork the Frappe Docker and rename it to something recognizable. We are going to start our ERP Next application from within the repo. There’s a lot of unneeded fluff for our purposes so we’ll delete like 80% of the files later on.

Step 2: Understand the Structure

This section will contain some explanation which isn’t necessary to actually install ERP Next so we can hold off on writing or running code and commands till Step 3.

compose.yaml

The compose.yaml file at the root is the main compose file which we are going to be using to start all our smaller containers. I’ve added helpful comments to it below, let’s go through it.

# Defining a reusable chunk of data
#   - Definition: &name_of_data
#   - Usage: *name_of_data

# Any service can adds this reusable default option
x-depends-on-configurator:
  &depends_on_configurator # Only run when the configurator exits with "service_completed_successfully"
  depends_on:
    configurator:
      condition: service_completed_successfully

# Another reusable default
x-backend-defaults:
  &backend_defaults # Use defaults from "depends_on_configurator" i.e. run after "configurator" service exits successfully
  <<: *depends_on_configurator
  # Use the specified image
  image: frappe/frappe-worker:${FRAPPE_VERSION:?No Frappe version set}
  # Mount the "sites" volume to the "/home/frappe/frappe-bench/sites" directory in the container
  volumes:
    - sites:/home/frappe/frappe-bench/sites

# Now we define all the services for this docker compose
# When we use "docker compose ... up", it will start all the services below
services:
  # The configurator service does some environment setup for all other services below
  configurator:
    <<: *backend_defaults
    # Script definition: https://github.com/frappe/frappe_docker/blob/4e63052a5448eda90c0be3ccf9efc39a83f0027c/images/worker/configure.py
    # This script just sets up some files for frappe to work correctly
    command: configure.py
    environment:
      DB_HOST: ${DB_HOST}
      DB_PORT: ${DB_PORT}
      REDIS_CACHE: ${REDIS_CACHE}
      REDIS_QUEUE: ${REDIS_QUEUE}
      REDIS_SOCKETIO: ${REDIS_SOCKETIO}
      SOCKETIO_PORT: 9000
    # Override depends_on so it doesn't use the default behavior of depending on configurator exiting successfully
    # (we are the configurator service!)
    depends_on: {}

  # The backend server which you can bash into to run commands such as "bench"
  backend:
    <<: *backend_defaults
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - assets:/home/frappe/frappe-bench/sites/assets:ro

  # Serves the website publicly
  frontend:
    image: frappe/frappe-nginx:${FRAPPE_VERSION}
    environment:
      BACKEND: backend:8000
      SOCKETIO: websocket:9000
      FRAPPE_SITE_NAME_HEADER: ${FRAPPE_SITE_NAME_HEADER:-$$host}
      UPSTREAM_REAL_IP_ADDRESS: ${UPSTREAM_REAL_IP_ADDRESS:-127.0.0.1}
      UPSTREAM_REAL_IP_HEADER: ${UPSTREAM_REAL_IP_HEADER:-X-Forwarded-For}
      UPSTREAM_REAL_IP_RECURSIVE: ${UPSTREAM_REAL_IP_RECURSIVE:-off}
    volumes:
      - sites:/usr/share/nginx/html/sites
      - assets:/usr/share/nginx/html/assets
    depends_on:
      - backend
      - websocket

  # Enables a two-way session between client and server to send and receive data
  # without the traditional handshake method that occurs for each HTTP request
  websocket:
    <<: *depends_on_configurator
    image: frappe/frappe-socketio:${FRAPPE_VERSION}
    volumes:
      - sites:/home/frappe/frappe-bench/sites

  # Bench has three queues where background jobs can be added
  # Service workers will pickup jobs from these queues one by one

  # Short queue for quick jobs
  queue-short:
    <<: *backend_defaults
    command: bench worker --queue short

  # Default queue
  queue-default:
    <<: *backend_defaults
    command: bench worker --queue default

  # And long queue for long running/lengthy jobs
  queue-long:
    <<: *backend_defaults
    command: bench worker --queue long

  # Scheduler enqueues different jobs which are then placed into the queues
  # above to be executed by the service workers
  scheduler:
    <<: *backend_defaults
    command: bench schedule

# Volumes are docker-controlled persistent storage that can be mounted and shared by many different services.
# We specify two volumes, one which contains all ours bench sites (in our case, only one site)
# and one which contains the assets for our sites.
volumes:
  sites:
  # The assets volume in particular is used by our nginx frontend to serve assets
  assets:

Overrides

In the overrides sub-directory, you will find a couple more YAML files such as compose.erpnext.yaml, compose.mariadb.yaml, etc. You can essentially think of these files as “plugins” with the caveat that not everything is compatible with each other out of the box.

• compose.erpnext.yaml – Replaces the frappe/worker docker images with erpnext/worker.
• compose.https.yaml – Adds support for HTTPS via auto-renewed LetsEncrypt SSL certificates and re-routes HTTP traffic to HTTPS.
• compose.mariadb.yaml – Sets up MariaDB as the SQL database.
• compose.noproxy.yaml – Makes the front-end directly available without the Traefik proxy middleware.
• compose.postgres.yaml – Sets up PostgreSQL as the SQL database.
• compose.proxy.yaml – Enables the Traefik proxy middleware.
• compose.redis.yaml – Enables Redis which is an in-memory cache layer.

To use one of these “plugins,” we can just add it as an additional configuration file when calling docker compose up. For example:

docker compose -p my-erp -f compose.yaml -f plugins/compose.erpnext.yaml -f plugins/compose.https.yaml up

Docker Compose & Orchestration

The idea behind docker compose is that it creates a predictable environment for each service to run along with defining how different services are orchestrated together. In the small bash script above, we are telling docker compose to build a configuration out of three YAML files (-f xyz). We specify a project name .. -p my-erp so that the next time we call the script, it recreates containers/services related to the saved my-erp project instead of creating an entirely new set of containers/services. And finally the command up essentially is a “start” command. To stop all services/containers related to the my-erp project, we would instead just write:

docker compose -p my-erp down

For more information on docker compose, check out their Getting Started with Docker Compose tutorial.

Step 3: Setting up Frappe

Step 3.0: Creating an environment file

The first thing I like to do is create a file called .env (if it doesn’t already exist) which will have information such as frappe version, database passwords, authentication keys for AWS, etc. As the guide focuses on a single server running ERP Next, lets add a SITE_NAME entry to the environment file.

# Reference: https://github.com/frappe/frappe_docker/blob/main/docs/images-and-compose-files.md
# Frappe and ErpNext version
FRAPPE_VERSION=v13.35.0
ERPNEXT_VERSION=v13.35.1

# Make sure both SITE_NAME and FRAPPE_SITE_NAME_HEADER are the same
# They aren't required to be the same but makes it easier in our case
SITE_NAME=my-erp
FRAPPE_SITE_NAME_HEADER=my-erp

# Password of the MariaDB database
DB_PASSWORD=<ENTER PASSWORD HERE>
# Password of the "Adminstrator" user of ErpNext
ADMIN_PASSWORD=<ENTER PASSWORD HERE>
# If there's an issue with the SSL certificate of your site (misconfiguration or expiration)
# LetsEncrypt will send you an email at this address
LETSENCRYPT_EMAIL=<ENTER EMAIL HERE>

# Fill this in with the public IP address of your server when you have one
UPSTREAM_REAL_IP_ADDRESS=

# These environment variables are not required.
DB_HOST=
DB_PORT=
REDIS_CACHE=
REDIS_QUEUE=
REDIS_SOCKETIO=
UPSTREAM_REAL_IP_HEADER=
UPSTREAM_REAL_IP_RECURSIVE=

Step 3.1: Creating your Docker Compose configuration

I like to use a shell script to start and stop all containers. Go ahead and create a file called start.sh in the same directory as your .env file and paste this bash code in.

#!/bin/bash

# Load variables from the .env file
if [ ! -f .env ]; then
  printf "No .env file found!"
  exit
else
  export $(grep -v '^#' .env | xargs)
fi

# Make sure we have all environment variables
if [ -z $SITE_NAME ]; then printf "SITE_NAME is empty."; exit; fi

docker compose 
--project-name $SITE_NAME 
-f compose.yaml 
-f plugins/compose.erpnext.yaml 
-f plugins/compose.mariadb.yaml 
-f plugins/compose.redis.yaml 
-f plugins/compose.https.yaml 
up $@

A few things to note about the script:

1. It requires the SITE_NAME variable to be defined in your .env file as we use it as the project name in docker compose. Specifying a project name allows us to tell docker which containers to restart, recreate, shutdown, etc. For more information, see Compose Project Name
2. It uses four overrides, erpnext, mariadb, redis, and https
3. The $@ at the end allows you to specify additional arguments such as ./start.sh -d or ./start.sh -d --remove-orphans which may be useful in the future.

Additionally, let’s create a stop.sh script to stop all running containers.

#!/bin/bash

# Load variables from the .env file
if [ ! -f .env ]; then
  printf "No .env file found!"
  exit
else
  export $(grep -v '^#' .env | xargs)
fi

# Make sure we have all environment variables
if [ -z $SITE_NAME ]; then printf "SITE_NAME is empty."; exit; fi

docker compose --project $SITE_NAME down $@

Step 3.2: Preliminary Tests

Run the start.sh script. You may need to give permissions to execute the script on your operating system. If everything is configured correctly, you should see multiple services popup on your terminal. As we haven’t yet created an ERP Next site, you won’t see anything if you visit your site on localhost.. well maybe an 404 page.

Step 4: Creating a new ERP Next site

To create the actual ERP Next site, you need to login to the bash terminal of your backend service. While all your containers are running, execute the following script to open a terminal shell in the backend container/service.

docker compose --project-name <THE_SITE_NAME_IN_YOUR_ENV_FILE> exec backend /bin/bash

Your terminal should be logged into the frappe user in the backend service. To create a new ERP Next site, run the command:

bench new-site <THE_SITE_NAME_IN_YOUR_ENV_FILE> 
--mariadb-root-password <THE_DB_PASSWORD_IN_YOUR_ENV_FILE> 
--admin-password <THE_ADMIN_PASSWORD_IN_YOUR_ENV_FILE>

This command creates a frappe site and will take a minute or two to process. After a frappe site is created, we install the ERP Next application on top of it using the command:

bench --site <THE_SITE_NAME_IN_YOUR_ENV_FILE> install-app erpnext

Wait for a minute or two. The install-app command just copies the relevant files over to our frappe site. To finish installation, we need to run a migration so entries in our database are created for the doc types added by frappe and erpnext. Finally, we run and wait:

bench --site <THE_SITE_NAME_IN_YOUR_ENV_FILE> migrate

Step 5: Running and Beyond

If you now navigate to localhost, you should see your ERP Next site up and running. Your browser may warn you about an incorrect SSL certificate, just ignore it for now. Once you this setup running on a publicly accessible server, your containerized installation will auto-negotiate and get a proper SSL certificate.

I’ll post a follow up on how to get your containerized installation running on Google Cloud Platform in the future. It’s mostly straight-forward (not really) but with some small caveats on what image to use, how to install docker, where to enable HTTPS traffic, etc.