2020-01-10 09:55:08 +00:00
|
|
|
version: "3.7"
|
2017-07-03 10:39:19 +00:00
|
|
|
services:
|
|
|
|
|
2017-07-24 09:32:50 +00:00
|
|
|
############# External services
|
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MONGODB %}
|
2017-07-03 10:39:19 +00:00
|
|
|
mongodb:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_MONGODB }}
|
2017-07-03 10:39:19 +00:00
|
|
|
# Use WiredTiger in all environments, just like at edx.org
|
2021-10-25 12:22:08 +00:00
|
|
|
command: mongod --nojournal --storageEngine wiredTiger
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
user: "999:999"
|
|
|
|
privileged: false
|
|
|
|
volumes:
|
|
|
|
- ../../data/mongodb:/data/db
|
|
|
|
depends_on:
|
|
|
|
- mongodb-permissions
|
|
|
|
mongodb-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["999", "/data/db"]
|
|
|
|
restart: on-failure
|
2017-07-03 10:39:19 +00:00
|
|
|
volumes:
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/mongodb:/data/db
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2017-07-03 10:39:19 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MYSQL %}
|
2017-07-03 10:39:19 +00:00
|
|
|
mysql:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_MYSQL }}
|
2017-07-03 10:39:19 +00:00
|
|
|
command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
user: "1000:1000"
|
|
|
|
privileged: false
|
2017-07-03 10:39:19 +00:00
|
|
|
volumes:
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/mysql:/var/lib/mysql
|
2020-06-01 15:03:28 +00:00
|
|
|
environment:
|
|
|
|
MYSQL_ROOT_PASSWORD: "{{ MYSQL_ROOT_PASSWORD }}"
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
mysql-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["1000", "/var/lib/mysql"]
|
|
|
|
restart: on-failure
|
|
|
|
volumes:
|
|
|
|
- ../../data/mysql:/var/lib/mysql
|
Improve job running in local and k8s
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:
- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.
Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.
This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.
In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
2020-03-25 17:47:36 +00:00
|
|
|
{% endif %}
|
2017-07-03 10:39:19 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_ELASTICSEARCH %}
|
2018-05-27 11:30:23 +00:00
|
|
|
elasticsearch:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_ELASTICSEARCH }}
|
2021-04-13 20:14:43 +00:00
|
|
|
environment:
|
|
|
|
- cluster.name=openedx
|
|
|
|
- bootstrap.memory_lock=true
|
|
|
|
- discovery.type=single-node
|
|
|
|
- "ES_JAVA_OPTS=-Xms{{ ELASTICSEARCH_HEAP_SIZE }} -Xmx{{ ELASTICSEARCH_HEAP_SIZE }}"
|
2018-05-27 11:30:23 +00:00
|
|
|
ulimits:
|
|
|
|
memlock:
|
|
|
|
soft: -1
|
|
|
|
hard: -1
|
2018-10-30 08:33:11 +00:00
|
|
|
restart: unless-stopped
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
user: "1000:1000"
|
|
|
|
volumes:
|
|
|
|
- ../../data/elasticsearch:/usr/share/elasticsearch/data
|
|
|
|
depends_on:
|
|
|
|
- elasticsearch-permissions
|
|
|
|
elasticsearch-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["1000", "/usr/share/elasticsearch/data"]
|
|
|
|
restart: on-failure
|
2018-05-27 11:30:23 +00:00
|
|
|
volumes:
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/elasticsearch:/usr/share/elasticsearch/data
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2018-05-27 11:30:23 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_REDIS %}
|
|
|
|
redis:
|
|
|
|
image: {{ DOCKER_IMAGE_REDIS }}
|
2021-02-16 11:12:18 +00:00
|
|
|
working_dir: /openedx/redis/data
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
user: "1000:1000"
|
2017-07-24 09:32:50 +00:00
|
|
|
volumes:
|
2021-02-16 11:12:18 +00:00
|
|
|
- ../apps/redis/redis.conf:/openedx/redis/config/redis.conf:ro
|
2020-09-17 10:53:14 +00:00
|
|
|
- ../../data/redis:/openedx/redis/data
|
|
|
|
command: redis-server /openedx/redis/config/redis.conf
|
2018-10-30 08:33:11 +00:00
|
|
|
restart: unless-stopped
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
depends_on:
|
|
|
|
- redis-permissions
|
|
|
|
redis-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["1000", "/openedx/redis/data"]
|
|
|
|
restart: on-failure
|
|
|
|
volumes:
|
|
|
|
- ../../data/redis:/openedx/redis/data
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2017-07-24 09:32:50 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_SMTP %}
|
2017-07-24 09:32:50 +00:00
|
|
|
smtp:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_SMTP }}
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
user: "100:101"
|
|
|
|
environment:
|
|
|
|
HOSTNAME: "{{ LMS_HOST }}"
|
2019-03-22 16:58:41 +00:00
|
|
|
{% endif %}
|
2017-07-24 09:32:50 +00:00
|
|
|
|
|
|
|
############# LMS and CMS
|
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_LMS %}
|
2017-07-03 10:39:19 +00:00
|
|
|
lms:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2018-04-09 17:16:58 +00:00
|
|
|
environment:
|
|
|
|
SERVICE_VARIANT: lms
|
2020-09-17 10:53:14 +00:00
|
|
|
UWSGI_WORKERS: {{ OPENEDX_LMS_UWSGI_WORKERS }}
|
2021-01-24 18:48:01 +00:00
|
|
|
SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production}
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
2017-07-03 10:39:19 +00:00
|
|
|
volumes:
|
2021-11-16 10:42:40 +00:00
|
|
|
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/config:/openedx/config:ro
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/lms:/openedx/data
|
2019-07-07 00:52:47 +00:00
|
|
|
- ../../data/openedx-media:/openedx/media
|
2017-07-03 10:39:19 +00:00
|
|
|
depends_on:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
- lms-permissions
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MYSQL %}- mysql{% endif %}
|
|
|
|
{% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %}
|
|
|
|
{% if RUN_MONGODB %}- mongodb{% endif %}
|
|
|
|
{% if RUN_REDIS %}- redis{% endif %}
|
|
|
|
{% if RUN_SMTP %}- smtp{% endif %}
|
2019-09-03 07:35:55 +00:00
|
|
|
{{ patch("local-docker-compose-lms-dependencies")|indent(6) }}
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
lms-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["1000", "/openedx/data", "/openedx/media"]
|
|
|
|
restart: on-failure
|
|
|
|
volumes:
|
|
|
|
- ../../data/lms:/openedx/data
|
|
|
|
- ../../data/openedx-media:/openedx/media
|
2019-03-22 19:15:50 +00:00
|
|
|
{% endif %}
|
2017-07-03 10:39:19 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_CMS %}
|
2017-07-24 09:32:50 +00:00
|
|
|
cms:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2018-04-09 17:16:58 +00:00
|
|
|
environment:
|
|
|
|
SERVICE_VARIANT: cms
|
2020-09-17 10:53:14 +00:00
|
|
|
UWSGI_WORKERS: {{ OPENEDX_CMS_UWSGI_WORKERS }}
|
2021-01-24 18:48:01 +00:00
|
|
|
SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production}
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
2017-07-24 09:32:50 +00:00
|
|
|
volumes:
|
2021-11-16 10:42:40 +00:00
|
|
|
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/config:/openedx/config:ro
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/cms:/openedx/data
|
2019-07-07 00:52:47 +00:00
|
|
|
- ../../data/openedx-media:/openedx/media
|
2017-07-24 09:32:50 +00:00
|
|
|
depends_on:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
- cms-permissions
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MYSQL %}- mysql{% endif %}
|
|
|
|
{% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %}
|
|
|
|
{% if RUN_MONGODB %}- mongodb{% endif %}
|
|
|
|
{% if RUN_REDIS %}- redis{% endif %}
|
|
|
|
{% if RUN_SMTP %}- smtp{% endif %}
|
|
|
|
{% if RUN_LMS %}- lms{% endif %}
|
2019-09-03 07:35:55 +00:00
|
|
|
{{ patch("local-docker-compose-cms-dependencies")|indent(6) }}
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
cms-permissions:
|
|
|
|
image: {{ DOCKER_IMAGE_PERMISSIONS }}
|
|
|
|
command: ["1000", "/openedx/data", "/openedx/media"]
|
|
|
|
restart: on-failure
|
|
|
|
volumes:
|
|
|
|
- ../../data/cms:/openedx/data
|
|
|
|
- ../../data/openedx-media:/openedx/media
|
2019-03-22 19:15:50 +00:00
|
|
|
{% endif %}
|
2017-07-24 09:32:50 +00:00
|
|
|
|
|
|
|
############# LMS and CMS workers
|
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_LMS %}
|
2020-03-10 09:05:53 +00:00
|
|
|
lms-worker:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2018-04-09 17:16:58 +00:00
|
|
|
environment:
|
|
|
|
SERVICE_VARIANT: lms
|
2021-01-24 18:48:01 +00:00
|
|
|
SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production}
|
2021-02-20 11:45:17 +00:00
|
|
|
command: celery worker --app=lms.celery --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild=100 --exclude-queues=edx.cms.core.default
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
2017-07-24 09:32:50 +00:00
|
|
|
volumes:
|
2021-11-16 10:42:40 +00:00
|
|
|
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/config:/openedx/config:ro
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/lms:/openedx/data
|
2019-07-07 00:52:47 +00:00
|
|
|
- ../../data/openedx-media:/openedx/media
|
2017-07-24 09:32:50 +00:00
|
|
|
depends_on:
|
2019-05-29 06:42:13 +00:00
|
|
|
- lms
|
2019-03-22 19:15:50 +00:00
|
|
|
{% endif %}
|
2017-12-26 00:16:35 +00:00
|
|
|
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_CMS %}
|
2020-03-10 09:05:53 +00:00
|
|
|
cms-worker:
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2018-04-09 17:16:58 +00:00
|
|
|
environment:
|
|
|
|
SERVICE_VARIANT: cms
|
2021-01-24 18:48:01 +00:00
|
|
|
SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production}
|
2020-09-17 10:53:14 +00:00
|
|
|
command: celery worker --app=cms.celery --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.lms.core.default
|
2018-02-08 06:40:18 +00:00
|
|
|
restart: unless-stopped
|
2017-12-26 00:16:35 +00:00
|
|
|
volumes:
|
2021-11-16 10:42:40 +00:00
|
|
|
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
|
|
|
|
- ../apps/openedx/config:/openedx/config:ro
|
2018-12-03 18:59:09 +00:00
|
|
|
- ../../data/cms:/openedx/data
|
2019-07-07 00:52:47 +00:00
|
|
|
- ../../data/openedx-media:/openedx/media
|
2017-12-26 00:16:35 +00:00
|
|
|
depends_on:
|
2019-05-29 06:42:13 +00:00
|
|
|
- cms
|
2019-03-22 19:15:50 +00:00
|
|
|
{% endif %}
|
2018-12-03 18:59:09 +00:00
|
|
|
|
2020-07-30 21:59:01 +00:00
|
|
|
{{ patch("local-docker-compose-services")|indent(2) }}
|