2020-09-17 10:53:14 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: caddy
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: caddy
|
|
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
|
|
|
|
app.kubernetes.io/name: caddy
|
|
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: caddy
|
|
|
|
|
spec:
|
fix: Enable rolling updates for the Caddy deployment in multi-node Kubernetes
When a Pod associated with a Deployment is updated (for example, due
to a change to its ConfigMap, or an updated image reference),
Kubernetes uses a ReplicaSet to spin up a Pod with the new
configuration, and once it is up, it tears down the old one.
In case of the Caddy Deployment, this is complicated by the fact that
it uses a Persistent Volume Claim (PVC), whose corresponding volume
uses a Read/Write-Once (RWO) configuration. This means that it can
only be used by multiple Pods if all those Pods all run on the same
Kubernetes worker node.
In order to enable rolling upgrades for the Caddy Deployment, we need
to ensure that its replacement Pod is scheduled on the same node as
the original Pod.
Thus, add a pod affinity rule that will force exactly that behavior.
Reference:
https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
The other Tutor services that use volumes (MySQL, Redis, Elasticsearch
and MongoDB) do not need this fix, since they all use the "Recreate"
deployment strategy: their Pods are all automatically torn down before
being replaced. This strategy is not needed for Caddy, and using a pod
affinity rule is less disruptive to the learner experience.
2022-05-10 12:38:39 +00:00
|
|
|
{%- if ENABLE_WEB_PROXY %}
|
|
|
|
|
# This Deployment uses a persistent volume claim. This requires
|
|
|
|
|
# that in order to enable rolling updates (i.e. use a deployment
|
|
|
|
|
# strategy other than Replace), we schedule the new Pod to the
|
|
|
|
|
# same node as the original Pod.
|
|
|
|
|
affinity:
|
|
|
|
|
podAffinity:
|
|
|
|
|
requiredDuringSchedulingIgnoredDuringExecution:
|
|
|
|
|
- labelSelector:
|
|
|
|
|
matchExpressions:
|
|
|
|
|
- key: app.kubernetes.io/name
|
|
|
|
|
operator: In
|
|
|
|
|
values:
|
|
|
|
|
- caddy
|
|
|
|
|
topologyKey: "kubernetes.io/hostname"
|
|
|
|
|
{%- endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
containers:
|
|
|
|
|
- name: caddy
|
|
|
|
|
image: {{ DOCKER_IMAGE_CADDY }}
|
2021-11-09 10:30:00 +00:00
|
|
|
env:
|
|
|
|
|
- name: default_site_port
|
|
|
|
|
value: "{% if not ENABLE_HTTPS or not ENABLE_WEB_PROXY %}:80{% endif %}"
|
2020-09-17 10:53:14 +00:00
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /etc/caddy/
|
|
|
|
|
name: config
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- if ENABLE_WEB_PROXY %}
|
2020-09-17 10:53:14 +00:00
|
|
|
- mountPath: /data/
|
|
|
|
|
name: data
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 80
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- if ENABLE_WEB_PROXY %}
|
2020-09-17 10:53:14 +00:00
|
|
|
- containerPort: 443
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: caddy-config
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- if ENABLE_WEB_PROXY %}
|
2020-09-17 10:53:14 +00:00
|
|
|
- name: data
|
|
|
|
|
persistentVolumeClaim:
|
|
|
|
|
claimName: caddy
|
2021-11-01 08:10:36 +00:00
|
|
|
{%- endif %}
|
2019-01-22 20:25:04 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: cms
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: cms
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: cms
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: cms
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
|
|
|
|
- name: cms
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2019-01-22 20:25:04 +00:00
|
|
|
env:
|
|
|
|
|
- name: SERVICE_VARIANT
|
|
|
|
|
value: cms
|
2022-04-12 14:07:48 +00:00
|
|
|
- name: DJANGO_SETTINGS_MODULE
|
|
|
|
|
value: cms.envs.tutor.production
|
2023-03-10 09:04:39 +00:00
|
|
|
- name: UWSGI_WORKERS
|
|
|
|
|
value: "{{ OPENEDX_CMS_UWSGI_WORKERS }}"
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 8000
|
|
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /openedx/edx-platform/lms/envs/tutor/
|
|
|
|
|
name: settings-lms
|
|
|
|
|
- mountPath: /openedx/edx-platform/cms/envs/tutor/
|
|
|
|
|
name: settings-cms
|
|
|
|
|
- mountPath: /openedx/config
|
|
|
|
|
name: config
|
2023-02-25 03:01:35 +00:00
|
|
|
- mountPath: /openedx/edx-platform/uwsgi.ini
|
|
|
|
|
name: uwsgi-config
|
|
|
|
|
subPath: uwsgi.ini
|
2019-06-06 19:58:21 +00:00
|
|
|
resources:
|
|
|
|
|
requests:
|
|
|
|
|
memory: 2Gi
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-06-06 19:58:21 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: settings-lms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-lms
|
|
|
|
|
- name: settings-cms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-cms
|
|
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-config
|
2023-02-25 03:01:35 +00:00
|
|
|
- name: uwsgi-config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-uwsgi-config
|
|
|
|
|
items:
|
|
|
|
|
- key: uwsgi.ini
|
|
|
|
|
path: uwsgi.ini
|
2019-06-06 19:58:21 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: cms-worker
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: cms-worker
|
|
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
|
|
|
|
app.kubernetes.io/name: cms-worker
|
|
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: cms-worker
|
|
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
2019-06-06 19:58:21 +00:00
|
|
|
containers:
|
|
|
|
|
- name: cms-worker
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2022-04-11 15:26:17 +00:00
|
|
|
args: ["celery", "--app=cms.celery", "worker", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--max-tasks-per-child", "100", "--exclude-queues=edx.lms.core.default"]
|
2019-06-06 19:58:21 +00:00
|
|
|
env:
|
|
|
|
|
- name: SERVICE_VARIANT
|
|
|
|
|
value: cms
|
2022-04-12 14:07:48 +00:00
|
|
|
- name: DJANGO_SETTINGS_MODULE
|
|
|
|
|
value: cms.envs.tutor.production
|
2019-06-06 19:58:21 +00:00
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /openedx/edx-platform/lms/envs/tutor/
|
|
|
|
|
name: settings-lms
|
|
|
|
|
- mountPath: /openedx/edx-platform/cms/envs/tutor/
|
|
|
|
|
name: settings-cms
|
|
|
|
|
- mountPath: /openedx/config
|
|
|
|
|
name: config
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: settings-lms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-lms
|
|
|
|
|
- name: settings-cms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-cms
|
|
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-config
|
|
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: lms
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: lms
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: lms
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: lms
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
|
|
|
|
- name: lms
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2022-04-12 14:07:48 +00:00
|
|
|
env:
|
|
|
|
|
- name: SERVICE_VARIANT
|
|
|
|
|
value: lms
|
|
|
|
|
- name: DJANGO_SETTINGS_MODULE
|
|
|
|
|
value: lms.envs.tutor.production
|
2023-03-10 09:04:39 +00:00
|
|
|
- name: UWSGI_WORKERS
|
|
|
|
|
value: "{{ OPENEDX_LMS_UWSGI_WORKERS }}"
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 8000
|
|
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /openedx/edx-platform/lms/envs/tutor/
|
|
|
|
|
name: settings-lms
|
|
|
|
|
- mountPath: /openedx/edx-platform/cms/envs/tutor/
|
|
|
|
|
name: settings-cms
|
|
|
|
|
- mountPath: /openedx/config
|
|
|
|
|
name: config
|
2023-02-25 03:01:35 +00:00
|
|
|
- mountPath: /openedx/edx-platform/uwsgi.ini
|
|
|
|
|
name: uwsgi-config
|
|
|
|
|
subPath: uwsgi.ini
|
2019-06-06 19:58:21 +00:00
|
|
|
resources:
|
|
|
|
|
requests:
|
|
|
|
|
memory: 2Gi
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-06-06 19:58:21 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: settings-lms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-lms
|
|
|
|
|
- name: settings-cms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-cms
|
|
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-config
|
2023-02-25 03:01:35 +00:00
|
|
|
- name: uwsgi-config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-uwsgi-config
|
|
|
|
|
items:
|
|
|
|
|
- key: uwsgi.ini
|
|
|
|
|
path: uwsgi.ini
|
2019-06-06 19:58:21 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: lms-worker
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: lms-worker
|
|
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
|
|
|
|
app.kubernetes.io/name: lms-worker
|
|
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: lms-worker
|
|
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
2019-06-06 19:58:21 +00:00
|
|
|
containers:
|
|
|
|
|
- name: lms-worker
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_OPENEDX }}
|
2022-04-11 15:26:17 +00:00
|
|
|
args: ["celery", "--app=lms.celery", "worker", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--max-tasks-per-child=100", "--exclude-queues=edx.cms.core.default"]
|
2019-06-06 19:58:21 +00:00
|
|
|
env:
|
|
|
|
|
- name: SERVICE_VARIANT
|
|
|
|
|
value: lms
|
2022-04-12 14:07:48 +00:00
|
|
|
- name: DJANGO_SETTINGS_MODULE
|
|
|
|
|
value: lms.envs.tutor.production
|
2019-06-06 19:58:21 +00:00
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /openedx/edx-platform/lms/envs/tutor/
|
|
|
|
|
name: settings-lms
|
|
|
|
|
- mountPath: /openedx/edx-platform/cms/envs/tutor/
|
|
|
|
|
name: settings-cms
|
|
|
|
|
- mountPath: /openedx/config
|
|
|
|
|
name: config
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: settings-lms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-lms
|
|
|
|
|
- name: settings-cms
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-settings-cms
|
|
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: openedx-config
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_ELASTICSEARCH %}
|
2019-01-22 20:25:04 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: elasticsearch
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
2020-03-25 16:08:23 +00:00
|
|
|
app.kubernetes.io/name: elasticsearch
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: elasticsearch
|
2019-06-06 19:58:21 +00:00
|
|
|
strategy:
|
|
|
|
|
type: Recreate
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: elasticsearch
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
|
|
|
|
fsGroup: 1000
|
|
|
|
|
fsGroupChangePolicy: "OnRootMismatch"
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
|
|
|
|
- name: elasticsearch
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_ELASTICSEARCH }}
|
2019-01-22 20:25:04 +00:00
|
|
|
env:
|
2021-04-13 20:14:43 +00:00
|
|
|
- name: cluster.name
|
|
|
|
|
value: "openedx"
|
|
|
|
|
- name: bootstrap.memory_lock
|
2019-01-22 20:25:04 +00:00
|
|
|
value: "true"
|
2021-04-13 20:14:43 +00:00
|
|
|
- name: discovery.type
|
|
|
|
|
value: "single-node"
|
|
|
|
|
- name: ES_JAVA_OPTS
|
|
|
|
|
value: "-Xms{{ ELASTICSEARCH_HEAP_SIZE }} -Xmx{{ ELASTICSEARCH_HEAP_SIZE }}"
|
|
|
|
|
- name: TAKE_FILE_OWNERSHIP
|
|
|
|
|
value: "1"
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 9200
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /usr/share/elasticsearch/data
|
|
|
|
|
name: data
|
|
|
|
|
volumes:
|
|
|
|
|
- name: data
|
|
|
|
|
persistentVolumeClaim:
|
|
|
|
|
claimName: elasticsearch
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MONGODB %}
|
2019-01-22 20:25:04 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: mongodb
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: mongodb
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: mongodb
|
2019-06-06 19:58:21 +00:00
|
|
|
strategy:
|
|
|
|
|
type: Recreate
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: mongodb
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 999
|
|
|
|
|
runAsGroup: 999
|
|
|
|
|
fsGroup: 999
|
|
|
|
|
fsGroupChangePolicy: "OnRootMismatch"
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
|
|
|
|
- name: mongodb
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_MONGODB }}
|
2021-10-25 12:22:08 +00:00
|
|
|
args: ["mongod", "--nojournal", "--storageEngine", "wiredTiger"]
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 27017
|
|
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /data/db
|
|
|
|
|
name: data
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: data
|
2019-06-06 19:58:21 +00:00
|
|
|
persistentVolumeClaim:
|
|
|
|
|
claimName: mongodb
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_MYSQL %}
|
2019-01-22 20:25:04 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: mysql
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: mysql
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: mysql
|
2019-06-06 19:58:21 +00:00
|
|
|
strategy:
|
|
|
|
|
type: Recreate
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: mysql
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
2022-01-03 08:04:12 +00:00
|
|
|
runAsUser: 999
|
|
|
|
|
runAsGroup: 999
|
|
|
|
|
fsGroup: 999
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
fsGroupChangePolicy: "OnRootMismatch"
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
|
|
|
|
- name: mysql
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_MYSQL }}
|
fix: Reduce MySQL binlog expiry from 30 days to 3
MySQL 8 defaults to a binlog expiry period of 2592000 seconds
(30 days), which for Tutor/Open edX purposes can be considered
excessive.
On the one hand, it is unlikely that a MySQL server configured for
Tutor uses MySQL replication at all (considering that up until Tutor
15 and MySQL 5.7, the binlog was disabled by default, rendering
replication impossible). Even if it does, a replica lagging more than
two days behind the primary server would be unacceptable.
Likewise, it is unlikely that an Open edX database is backed up less
than once a day, thus is is unlikely that Open edX admins would
benefit from the ability to do point-in-time restore over a 30-day
period.
On the other hand, having a 30-day binlog expiry period can
considerably increase the storage space requirements for the MySQL
container, particularly on busy Open edX platforms. When left
unchecked, this can even cause the MySQL container to run into "No
space left on device" situations, disabling the MySQL database
altogether. Thus, the MySQL default settings are likely to be a net
disadvantage for Open edX admins.
Finally, all of the above considerations apply only if the Open edX
administrator has chosen to run their own MySQL and not opted for a
DBaaS solution like AWS RDS.
Thus, it should be acceptable to run with a reduced binlog expiry
period of 3 days (rather than 30) by default.
Therefore, inject the --binlog-expire-logs-seconds=259200 argument
into the Tutor-generated command to start mysqld.
Reference:
https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_expire_logs_seconds
2023-10-23 13:59:44 +00:00
|
|
|
args:
|
|
|
|
|
- "mysqld"
|
|
|
|
|
- "--character-set-server=utf8mb3"
|
|
|
|
|
- "--collation-server=utf8mb3_general_ci"
|
|
|
|
|
- "--binlog-expire-logs-seconds=259200"
|
2019-01-22 20:25:04 +00:00
|
|
|
env:
|
|
|
|
|
- name: MYSQL_ROOT_PASSWORD
|
2020-06-01 15:03:28 +00:00
|
|
|
value: "{{ MYSQL_ROOT_PASSWORD }}"
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
|
|
|
|
- containerPort: 3306
|
|
|
|
|
volumeMounts:
|
|
|
|
|
- mountPath: /var/lib/mysql
|
|
|
|
|
name: data
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumes:
|
|
|
|
|
- name: data
|
|
|
|
|
persistentVolumeClaim:
|
|
|
|
|
claimName: mysql
|
Improve job running in local and k8s
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:
- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.
Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.
This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.
In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
2020-03-25 17:47:36 +00:00
|
|
|
{% endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_SMTP %}
|
2019-03-20 17:35:09 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
|
|
|
|
name: smtp
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
|
|
|
|
app.kubernetes.io/name: smtp
|
2019-03-20 17:35:09 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: smtp
|
2019-03-20 17:35:09 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2019-05-09 07:51:06 +00:00
|
|
|
app.kubernetes.io/name: smtp
|
2019-03-20 17:35:09 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 100
|
|
|
|
|
runAsGroup: 101
|
2019-03-20 17:35:09 +00:00
|
|
|
containers:
|
|
|
|
|
- name: smtp
|
2020-07-21 07:13:00 +00:00
|
|
|
image: {{ DOCKER_IMAGE_SMTP }}
|
2019-03-20 17:35:09 +00:00
|
|
|
ports:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
- containerPort: 8025
|
2019-03-20 17:45:09 +00:00
|
|
|
{% endif %}
|
2020-09-17 10:53:14 +00:00
|
|
|
{% if RUN_REDIS %}
|
2019-01-22 20:25:04 +00:00
|
|
|
---
|
|
|
|
|
apiVersion: apps/v1
|
|
|
|
|
kind: Deployment
|
|
|
|
|
metadata:
|
2020-09-17 10:53:14 +00:00
|
|
|
name: redis
|
2019-05-09 07:51:06 +00:00
|
|
|
labels:
|
2020-09-17 10:53:14 +00:00
|
|
|
app.kubernetes.io/name: redis
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
|
|
|
|
selector:
|
|
|
|
|
matchLabels:
|
2020-09-17 10:53:14 +00:00
|
|
|
app.kubernetes.io/name: redis
|
2019-06-06 19:58:21 +00:00
|
|
|
strategy:
|
|
|
|
|
type: Recreate
|
2019-01-22 20:25:04 +00:00
|
|
|
template:
|
|
|
|
|
metadata:
|
|
|
|
|
labels:
|
2020-09-17 10:53:14 +00:00
|
|
|
app.kubernetes.io/name: redis
|
2019-01-22 20:25:04 +00:00
|
|
|
spec:
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
runAsUser: 1000
|
|
|
|
|
runAsGroup: 1000
|
|
|
|
|
fsGroup: 1000
|
|
|
|
|
fsGroupChangePolicy: "OnRootMismatch"
|
2019-01-22 20:25:04 +00:00
|
|
|
containers:
|
2020-09-17 10:53:14 +00:00
|
|
|
- name: redis
|
|
|
|
|
image: {{ DOCKER_IMAGE_REDIS }}
|
2021-02-16 11:12:18 +00:00
|
|
|
args: ["redis-server", "/openedx/redis/config/redis.conf"]
|
|
|
|
|
workingDir: /openedx/redis/data
|
2019-01-22 20:25:04 +00:00
|
|
|
ports:
|
2020-09-17 10:53:14 +00:00
|
|
|
- containerPort: {{ REDIS_PORT }}
|
2019-01-22 20:25:04 +00:00
|
|
|
volumeMounts:
|
2020-09-17 10:53:14 +00:00
|
|
|
- mountPath: /openedx/redis/config/
|
|
|
|
|
name: config
|
|
|
|
|
- mountPath: /openedx/redis/data
|
2019-01-22 20:25:04 +00:00
|
|
|
name: data
|
feat: run all services as unprivileged containers
With this change, containers are no longer run as "root" but as unprivileged
users. This is necessary in some environments, notably some Kubernetes
clusters.
To make this possible, we need to manually fix bind-mounted volumes in
docker-compose. This is pretty much equivalent to the behaviour in Kubernetes,
where permissions are fixed at runtime if the volume owner is incorrect. Thus,
we have a consistent behaviour between docker-compose and Kubernetes.
We achieve this by bind-mounting some repos inside "*-permissions" services.
These services run as root user on docker-compose and will fix the required
permissions, as per build/permissions/setowner.sh These services simply do not
run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make
use of Kubernete's built-in volume ownership feature.
With this change, we get rid of the "openedx-dev" Docker image, in the sense
that it no longer has its own Dockerfile. Instead, the dev image is now simply
a different target in the multi-layer openedx Docker image. This makes it much
faster to build the openedx-dev image.
Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need
to pass the user ID from the host there. The only way to achieve that is with a
tutor config variable. The downside of this approach is that the
dev/docker-compose.yml file is no longer portable from one machine to the next.
We consider that this is not such a big issue, as it affects the development
environment only.
We take this opportunity to replace the base image of the "forum" image. There
is now no need to re-install ruby inside the image. The total image size is
only decreased by 10%, but re-building the image is faster.
In order to run the smtp service as non-root, we switch from namshi/smtp to
devture/exim-relay. This change should be backward-compatible.
Note that the nginx container remains privileged. We could switch to
nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are
considering to get rid of the nginx container altogether.
Close #323.
2021-09-23 10:04:19 +00:00
|
|
|
securityContext:
|
|
|
|
|
allowPrivilegeEscalation: false
|
2019-01-22 20:25:04 +00:00
|
|
|
volumes:
|
2020-09-17 10:53:14 +00:00
|
|
|
- name: config
|
|
|
|
|
configMap:
|
|
|
|
|
name: redis-config
|
2019-01-22 20:25:04 +00:00
|
|
|
- name: data
|
|
|
|
|
persistentVolumeClaim:
|
2020-09-17 10:53:14 +00:00
|
|
|
claimName: redis
|
2019-03-20 17:59:09 +00:00
|
|
|
{% endif %}
|
2019-06-05 13:43:51 +00:00
|
|
|
{{ patch("k8s-deployments") }}
|
