2
0
mirror of https://github.com/frappe/frappe_docker.git synced 2024-12-24 11:05:29 +00:00
frappe_docker/docs/images-and-compose-files.md
Lev a9b6b755ce
Global refactoring (#617)
* Rename `bench-build` target to `bench` in bake file

* Update bake file and break everything

* Rename docker-compose.yml to compose.yml to avoid conflicting on `docker buildx bake`

* Fix groups in bake file

* Update frappe-worker

* Update frappe-nginx, erpnext-nginx

* Remove old erpnext images

* Update frappe-socketio

* Fix develop frappe-nginx build on linux/arm64

* Update dockerignore

* Update gitignore

* Update gitignore

* Update .env files

* Update installation (overrides)

* Update tests

* Fix image names

* Update compose

* Update get-latest-tags

* Update CI

* Setup and remove .env on tests

* Add build bench workflow

* Add triggers to main workflow

* Add release helm job

* Use reusable workflows

* Rollback

* Print configuration before running tests

* Show tests/.env

* Revert "Show tests/.env"

This reverts commit 4bc3bdebaf.

* Fix ci image versions

* Remove `frappe-` prefix in build directories

* Move requirements-dev.txt

* Fix image name in CI

* Update gitignore

* Update pre-commit config

* Drop `version:` in compose files

* Add push-backup

* Fix postgres CI test

* Change .yml to .yaml in compose file to follow compose-spec

* Remove prettierignore

* Fix dockerignore

* Change .yml to .yaml in compose file to follow compose-spec

* Don't depend on boto3 while testing (do it in backend)

* Update erpnext example version

* Don't fail ping on URLError

* Move assets volume to main compose file

* Fix type annotations for v12

* Fix postgres ci override in tests

* Fix spaces in socketio

* Reorder stages in nginx image, improve perfomance

* Remove unused todo

* Optimize worker build

* Install Node in worker image

* Add 502 error page

* Remove unused quiet-pull in tests

* Add configurator service to dynamically set common config

* Remove unused compose.ci-postgres.yml

* Use Python for configurator service: faster and more robust

* Add TODO.md

* Use python script to get latest tags in CI

* Clean up nginx dockerfile

* Remove VOLUME declaration

https://stackoverflow.com/a/55052682

* Add custom app example

* Remove pwd for now

* Remove pwd for now

* Use jq for parsing config in healthcheck

* Take advantage of yaml lang: add defaults in compose file. Also require env vars

* Fix CI

* Use resusable workflow

* Update

* Move release_helm job to main.yml

* Rename docker-build to docker-build-push

* Rename main to build_stable

* Rename bench targets

* Remove quotes from docker-build-push inputs

* Update build develop

* Remove HELM_DEPLOY_KEY secret from docker-build-push

* Add job names

* Remove build_bench workflow

* Update version input description in docker-build-push

* Print .env in tests, if version is develop, change to latest (for tag)

* Fix env setup

* Uncomment tests

* Parse and set short tags from git tag in bake file

* Move devcontainer settings to devcontainer.json

* Add db command notice

* Fix CI?

* Fix inconsistencies in development readme

* Remove pwd for now

* Remove custom apps for production instruction

* Update todos

* Add docs for images and compose files

* Add variables docs and allow custom frappe site name header

* Add notice about internal environment variables

* Update site-operations docs

* Update todos

* Add Overrides header in images-and-compose-files

* Update todos

* Remove extra docs

* Don't log requests in worker image (nginx already does that)

* Remove default value of FRAPPE_SITE_NAME_HEADER in example.env

* Use file that consistent in v12, v13 and develop to check /assets

* Fix paths in CI

* Update todos

* Remove TODO.md

* Update tests/_check_backup_files.py

Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com>

* Change variables MINIO_ACCESS_KEY and MINIO_SECRET_KEY to S3_ACCESS_KEY, S3_SECRET_KEY in tests

* Fix S3 test

* Use `nginxinc/nginx-unprivileged` instead of `nginx` image

Also use Ngnix 1.20 instead of unstable 1.21

* Fix https override

* Update Dockerfile

* Mount assets to backend service in read only mode

* Touch .build (#307), use scripts from nginx image to generate config and touch .build

* Update example env after building stable images

* Touch `.build` on develop image (untill https://github.com/frappe/frappe/issues/15396 is resolved)

* Add `make` to worker build deps for linux/arm64

* Fix update example.env job

* Fix .build creation on develop branch

* Move bench CI to different file

This way workflow runs only on PRs that relevant to bench build

* Fix app name in custom app example

* Update erpnext and frappe versions in example.env

* Don't install `svg-sprite` and `sass` node modules in nginx image on linux/arm64 (https://github.com/frappe/frappe/pull/15275)

* docs: README and docs

* docs: add link to site operations from docker swarm

* ci: fix tests as per changes to compose.yaml

* docs: move wiki articles to docs

* docs: fix add custom domain

* docs: fix patch code from images

* fix: do not expose port 80 for old images

* fix: custom domain labels to frontend container/service

* Add missing descriptions to envs in example.env

* Fix redis depends_on

* Fix docker compose in tests when not running on TTY

* Set -T flag in `docker compose exec` only if not tty

* Run pre-commit on docs

* Remove postgres healthcheck (it gets overriden by mariadb)

* Refactor test

* Update workflow names

* Add pip to dependabot config

* docs: backup and push (#19)

* Beautify changes by @revant (#20)

* feat: add gevent to worker image

* feat: real_ip configuration for nginx

* Return `healthcheck.sh` just for tests

Co-authored-by: Lev Vereshchagin <mail@vrslev.com>

* Make pretend bench catch unknown commands (closes #666)

* Remove debug print in push-backup

* Fix typing issues in push-backup

* Update file keys in push-backups: from abs path to <site>/<file>

* Refactor push-backup

* Move gevent installation in Frappe step

* Don't pin boto stubs requirement

* Cache pip deps on build

* Update example env versions

* Refactor check backup files

* Fix backup test

* Fix backup test

* Rename build/ dir to images/

* Rename build/ dir to images/

* Fix /build -> /images in docs

* Update example.env

* Use reusable workflow in frappe user instead of vrslev

* Fix compose`s `project` option in docs (https://github.com/frappe/frappe_docker/pull/617#issuecomment-1065178792)

* Add note about project option in site-operations doc

* Update example env

* Rename build arg `USERNAME` to `REGISTRY_USER`

* Allow https proxy to access Docker socket

* Revert "Use reusable workflow in frappe user instead of vrslev"

This reverts commit 6062500d0d.

* Revert "Revert "Use reusable workflow in frappe user instead of vrslev""

This reverts commit 4680d18ff8.

Co-authored-by: Revant Nandgaonkar <revant.one@gmail.com>
2022-03-14 11:23:03 +05:30

6.0 KiB

Images

There's 4 images that you can find in /images directory:

  • bench. It is used for development. Learn more how to start development.
  • nginx. This image contains JS and CSS assets. Container using this image also routes incoming requests using nginx.
  • socketio. Container using this image processes realtime websocket requests using Socket.IO.
  • worker. Multi-purpose Python backend. Runs Werkzeug server with gunicorn, queues (via bench worker), or schedule (via bench schedule).

nginx, socketio and worker images — everything we need to be able to run all processes that Frappe framework requires (take a look at Bench Procfile reference). We follow Docker best practices and split these processes to different containers.

ERPNext images don't have their own Dockerfiles. We use multi-stage builds and Docker Buildx to reuse as much things as possible and make our builds more efficient.

Compose files

After building the images we have to run the containers. The best and simplest way to do this is to use compose files.

We have one main compose file, compose.yaml. Services described, networking, volumes are also handled there.

Services

All services are described in compose.yaml

  • configurator. Updates common_site_config.json so Frappe knows how to access db and redis. It is executed on every docker-compose up (and exited immediately). Other services start after this container exits successfully.
  • backend. Werkzeug server.
  • db. Optional service that runs MariaDB if you also use overrides/compose.mariadb.yaml or Postgres if you also use overrides/compose.postgres.yaml.
  • redis. Optional service that runs Redis server with cache, Socket.IO and queues data.
  • frontend. nginx server that serves JS/CSS assets and routes incoming requests.
  • proxy. Traefik proxy. It is here for complicated setups or HTTPS override (with overrides/compose.https.yaml).
  • websocket. Node server that runs Socket.IO.
  • queue-short, queue-default, queue-long. Python servers that run job queues using rq.
  • scheduler. Python server that runs tasks on schedule using schedule.

Overrides

We have several overrides:

  • overrides/compose.proxy.yaml. Adds traefik proxy to setup.
  • overrides/compose.noproxy.yaml. Publishes frontend ports directly without any proxy.
  • overrides/compose.erpnext.yaml. Replaces all Frappe images with ERPNext ones. ERPNext images are built on top of Frappe ones, so it is safe to replace them.
  • overrides/compose.https.yaml. Automatically sets up Let's Encrypt certificate and redirects all requests to directed to http, to https.
  • overrides/compose.mariadb.yaml. Adds db service and sets its image to MariaDB.
  • overrides/compose.postgres.yaml. Adds db service and sets its image to Postgres. Note that ERPNext currently doesn't support Postgres.
  • overrides/compose.redis.yaml. Adds redis service and sets its image to redis.
  • overrides/compose.swarm.yaml. Workaround override for generating swarm stack.

It is quite simple to run overrides. All we need to do is to specify compose files that should be used by docker-compose. For example, we want ERPNext:

# Point to main compose file (compose.yaml) and add one more.
docker-compose -f compose.yaml -f overrides/compose.erpnext.yaml config

That's it! Of course, we also have to setup .env before all of that, but that's not the point.

Configuration

We use environment variables to configure our setup. docker-compose uses variables from .env file. To get started, copy example.env to .env.

FRAPPE_VERSION

Frappe framework release. You can find all releases here.

DB_PASSWORD

Password for MariaDB (or Postgres) database.

DB_HOST

Hostname for MariaDB (or Postgres) database. Set only if external service for database is used.

DB_PORT

Port for MariaDB (3306) or Postgres (5432) database. Set only if external service for database is used.

REDIS_CACHE

Hostname for redis server to store cache. Set only if external service for redis is used.

REDIS_QUEUE

Hostname for redis server to store queue data. Set only if external service for redis is used.

REDIS_SOCKETIO

Hostname for redis server to store socketio data. Set only if external service for redis is used.

ERPNEXT_VERSION

ERPNext release. This variable is required if you use ERPNext override.

LETSENCRYPT_EMAIL

Email that used to register https certificate. This one is required only if you use HTTPS override.

FRAPPE_SITE_NAME_HEADER

This environment variable is not required. Default value is $$host which resolves site by host. For example, if your host is example.com, site's name should be example.com, or if host is 127.0.0.1 (local debugging), it should be 127.0.0.1 This variable allows to override described behavior. Let's say you create site named mysite and do want to access it by 127.0.0.1 host. Than you would set this variable to mysite.

There is other variables not mentioned here. They're somewhat internal and you don't have to worry about them except you want to change main compose file.