* Rename `bench-build` target to `bench` in bake file * Update bake file and break everything * Rename docker-compose.yml to compose.yml to avoid conflicting on `docker buildx bake` * Fix groups in bake file * Update frappe-worker * Update frappe-nginx, erpnext-nginx * Remove old erpnext images * Update frappe-socketio * Fix develop frappe-nginx build on linux/arm64 * Update dockerignore * Update gitignore * Update gitignore * Update .env files * Update installation (overrides) * Update tests * Fix image names * Update compose * Update get-latest-tags * Update CI * Setup and remove .env on tests * Add build bench workflow * Add triggers to main workflow * Add release helm job * Use reusable workflows * Rollback * Print configuration before running tests * Show tests/.env * Revert "Show tests/.env" This reverts commit4bc3bdebaf
. * Fix ci image versions * Remove `frappe-` prefix in build directories * Move requirements-dev.txt * Fix image name in CI * Update gitignore * Update pre-commit config * Drop `version:` in compose files * Add push-backup * Fix postgres CI test * Change .yml to .yaml in compose file to follow compose-spec * Remove prettierignore * Fix dockerignore * Change .yml to .yaml in compose file to follow compose-spec * Don't depend on boto3 while testing (do it in backend) * Update erpnext example version * Don't fail ping on URLError * Move assets volume to main compose file * Fix type annotations for v12 * Fix postgres ci override in tests * Fix spaces in socketio * Reorder stages in nginx image, improve perfomance * Remove unused todo * Optimize worker build * Install Node in worker image * Add 502 error page * Remove unused quiet-pull in tests * Add configurator service to dynamically set common config * Remove unused compose.ci-postgres.yml * Use Python for configurator service: faster and more robust * Add TODO.md * Use python script to get latest tags in CI * Clean up nginx dockerfile * Remove VOLUME declaration https://stackoverflow.com/a/55052682 * Add custom app example * Remove pwd for now * Remove pwd for now * Use jq for parsing config in healthcheck * Take advantage of yaml lang: add defaults in compose file. Also require env vars * Fix CI * Use resusable workflow * Update * Move release_helm job to main.yml * Rename docker-build to docker-build-push * Rename main to build_stable * Rename bench targets * Remove quotes from docker-build-push inputs * Update build develop * Remove HELM_DEPLOY_KEY secret from docker-build-push * Add job names * Remove build_bench workflow * Update version input description in docker-build-push * Print .env in tests, if version is develop, change to latest (for tag) * Fix env setup * Uncomment tests * Parse and set short tags from git tag in bake file * Move devcontainer settings to devcontainer.json * Add db command notice * Fix CI? * Fix inconsistencies in development readme * Remove pwd for now * Remove custom apps for production instruction * Update todos * Add docs for images and compose files * Add variables docs and allow custom frappe site name header * Add notice about internal environment variables * Update site-operations docs * Update todos * Add Overrides header in images-and-compose-files * Update todos * Remove extra docs * Don't log requests in worker image (nginx already does that) * Remove default value of FRAPPE_SITE_NAME_HEADER in example.env * Use file that consistent in v12, v13 and develop to check /assets * Fix paths in CI * Update todos * Remove TODO.md * Update tests/_check_backup_files.py Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com> * Change variables MINIO_ACCESS_KEY and MINIO_SECRET_KEY to S3_ACCESS_KEY, S3_SECRET_KEY in tests * Fix S3 test * Use `nginxinc/nginx-unprivileged` instead of `nginx` image Also use Ngnix 1.20 instead of unstable 1.21 * Fix https override * Update Dockerfile * Mount assets to backend service in read only mode * Touch .build (#307), use scripts from nginx image to generate config and touch .build * Update example env after building stable images * Touch `.build` on develop image (untill https://github.com/frappe/frappe/issues/15396 is resolved) * Add `make` to worker build deps for linux/arm64 * Fix update example.env job * Fix .build creation on develop branch * Move bench CI to different file This way workflow runs only on PRs that relevant to bench build * Fix app name in custom app example * Update erpnext and frappe versions in example.env * Don't install `svg-sprite` and `sass` node modules in nginx image on linux/arm64 (https://github.com/frappe/frappe/pull/15275) * docs: README and docs * docs: add link to site operations from docker swarm * ci: fix tests as per changes to compose.yaml * docs: move wiki articles to docs * docs: fix add custom domain * docs: fix patch code from images * fix: do not expose port 80 for old images * fix: custom domain labels to frontend container/service * Add missing descriptions to envs in example.env * Fix redis depends_on * Fix docker compose in tests when not running on TTY * Set -T flag in `docker compose exec` only if not tty * Run pre-commit on docs * Remove postgres healthcheck (it gets overriden by mariadb) * Refactor test * Update workflow names * Add pip to dependabot config * docs: backup and push (#19) * Beautify changes by @revant (#20) * feat: add gevent to worker image * feat: real_ip configuration for nginx * Return `healthcheck.sh` just for tests Co-authored-by: Lev Vereshchagin <mail@vrslev.com> * Make pretend bench catch unknown commands (closes #666) * Remove debug print in push-backup * Fix typing issues in push-backup * Update file keys in push-backups: from abs path to <site>/<file> * Refactor push-backup * Move gevent installation in Frappe step * Don't pin boto stubs requirement * Cache pip deps on build * Update example env versions * Refactor check backup files * Fix backup test * Fix backup test * Rename build/ dir to images/ * Rename build/ dir to images/ * Fix /build -> /images in docs * Update example.env * Use reusable workflow in frappe user instead of vrslev * Fix compose`s `project` option in docs (https://github.com/frappe/frappe_docker/pull/617#issuecomment-1065178792) * Add note about project option in site-operations doc * Update example env * Rename build arg `USERNAME` to `REGISTRY_USER` * Allow https proxy to access Docker socket * Revert "Use reusable workflow in frappe user instead of vrslev" This reverts commit6062500d0d
. * Revert "Revert "Use reusable workflow in frappe user instead of vrslev"" This reverts commit4680d18ff8
. Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com>
6.0 KiB
Images
There's 4 images that you can find in /images
directory:
bench
. It is used for development. Learn more how to start development.nginx
. This image contains JS and CSS assets. Container using this image also routes incoming requests using nginx.socketio
. Container using this image processes realtime websocket requests using Socket.IO.worker
. Multi-purpose Python backend. Runs Werkzeug server with gunicorn, queues (viabench worker
), or schedule (viabench schedule
).
nginx
,socketio
andworker
images — everything we need to be able to run all processes that Frappe framework requires (take a look at Bench Procfile reference). We follow Docker best practices and split these processes to different containers.
ERPNext images don't have their own Dockerfiles. We use multi-stage builds and Docker Buildx to reuse as much things as possible and make our builds more efficient.
Compose files
After building the images we have to run the containers. The best and simplest way to do this is to use compose files.
We have one main compose file, compose.yaml
. Services described, networking, volumes are also handled there.
Services
All services are described in compose.yaml
configurator
. Updatescommon_site_config.json
so Frappe knows how to access db and redis. It is executed on everydocker-compose up
(and exited immediately). Other services start after this container exits successfully.backend
. Werkzeug server.db
. Optional service that runs MariaDB if you also useoverrides/compose.mariadb.yaml
or Postgres if you also useoverrides/compose.postgres.yaml
.redis
. Optional service that runs Redis server with cache, Socket.IO and queues data.frontend
. nginx server that serves JS/CSS assets and routes incoming requests.proxy
. Traefik proxy. It is here for complicated setups or HTTPS override (withoverrides/compose.https.yaml
).websocket
. Node server that runs Socket.IO.queue-short
,queue-default
,queue-long
. Python servers that run job queues using rq.scheduler
. Python server that runs tasks on schedule using schedule.
Overrides
We have several overrides:
overrides/compose.proxy.yaml
. Adds traefik proxy to setup.overrides/compose.noproxy.yaml
. Publishesfrontend
ports directly without any proxy.overrides/compose.erpnext.yaml
. Replaces all Frappe images with ERPNext ones. ERPNext images are built on top of Frappe ones, so it is safe to replace them.overrides/compose.https.yaml
. Automatically sets up Let's Encrypt certificate and redirects all requests to directed to http, to https.overrides/compose.mariadb.yaml
. Addsdb
service and sets its image to MariaDB.overrides/compose.postgres.yaml
. Addsdb
service and sets its image to Postgres. Note that ERPNext currently doesn't support Postgres.overrides/compose.redis.yaml
. Addsredis
service and sets its image toredis
.overrides/compose.swarm.yaml
. Workaround override for generating swarm stack.
It is quite simple to run overrides. All we need to do is to specify compose files that should be used by docker-compose. For example, we want ERPNext:
# Point to main compose file (compose.yaml) and add one more.
docker-compose -f compose.yaml -f overrides/compose.erpnext.yaml config
That's it! Of course, we also have to setup .env
before all of that, but that's not the point.
Configuration
We use environment variables to configure our setup. docker-compose uses variables from .env
file. To get started, copy example.env
to .env
.
FRAPPE_VERSION
Frappe framework release. You can find all releases here.
DB_PASSWORD
Password for MariaDB (or Postgres) database.
DB_HOST
Hostname for MariaDB (or Postgres) database. Set only if external service for database is used.
DB_PORT
Port for MariaDB (3306) or Postgres (5432) database. Set only if external service for database is used.
REDIS_CACHE
Hostname for redis server to store cache. Set only if external service for redis is used.
REDIS_QUEUE
Hostname for redis server to store queue data. Set only if external service for redis is used.
REDIS_SOCKETIO
Hostname for redis server to store socketio data. Set only if external service for redis is used.
ERPNEXT_VERSION
ERPNext release. This variable is required if you use ERPNext override.
LETSENCRYPT_EMAIL
Email that used to register https certificate. This one is required only if you use HTTPS override.
FRAPPE_SITE_NAME_HEADER
This environment variable is not required. Default value is $$host
which resolves site by host. For example, if your host is example.com
, site's name should be example.com
, or if host is 127.0.0.1
(local debugging), it should be 127.0.0.1
This variable allows to override described behavior. Let's say you create site named mysite
and do want to access it by 127.0.0.1
host. Than you would set this variable to mysite
.
There is other variables not mentioned here. They're somewhat internal and you don't have to worry about them except you want to change main compose file.