From 01b58d9d75471b93aa4b5864edef6fcabd56712a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 23 Sep 2021 12:04:19 +0200 Subject: [PATCH] feat: run all services as unprivileged containers With this change, containers are no longer run as "root" but as unprivileged users. This is necessary in some environments, notably some Kubernetes clusters. To make this possible, we need to manually fix bind-mounted volumes in docker-compose. This is pretty much equivalent to the behaviour in Kubernetes, where permissions are fixed at runtime if the volume owner is incorrect. Thus, we have a consistent behaviour between docker-compose and Kubernetes. We achieve this by bind-mounting some repos inside "*-permissions" services. These services run as root user on docker-compose and will fix the required permissions, as per build/permissions/setowner.sh These services simply do not run on Kubernetes, where we don't rely on bind-mounted volumes. There, we make use of Kubernete's built-in volume ownership feature. With this change, we get rid of the "openedx-dev" Docker image, in the sense that it no longer has its own Dockerfile. Instead, the dev image is now simply a different target in the multi-layer openedx Docker image. This makes it much faster to build the openedx-dev image. Because we declare the APP_USER_ID in the dev/docker-compose.yml file, we need to pass the user ID from the host there. The only way to achieve that is with a tutor config variable. The downside of this approach is that the dev/docker-compose.yml file is no longer portable from one machine to the next. We consider that this is not such a big issue, as it affects the development environment only. We take this opportunity to replace the base image of the "forum" image. There is now no need to re-install ruby inside the image. The total image size is only decreased by 10%, but re-building the image is faster. In order to run the smtp service as non-root, we switch from namshi/smtp to devture/exim-relay. This change should be backward-compatible. Note that the nginx container remains privileged. We could switch to nginxinc/nginx-unprivileged, but it's probably not worth the effort, as we are considering to get rid of the nginx container altogether. Close #323. --- CHANGELOG-nightly.md | 5 +- docs/configuration.rst | 2 +- docs/dev.rst | 2 +- docs/tutorials/theming.rst | 4 -- tutor/bindmounts.py | 1 + tutor/commands/images.py | 11 +-- tutor/env.py | 1 + tutor/jobs.py | 1 + tutor/templates/build/forum/Dockerfile | 37 +++++----- tutor/templates/build/openedx-dev/Dockerfile | 34 ---------- .../build/openedx-dev/bin/create-user.sh | 11 --- .../openedx-dev/bin/docker-entrypoint.sh | 19 ------ tutor/templates/build/openedx/Dockerfile | 68 ++++++++++++++----- tutor/templates/build/permissions/Dockerfile | 7 ++ tutor/templates/build/permissions/setowner.sh | 14 ++++ tutor/templates/config.yml | 7 +- tutor/templates/dev/docker-compose.yml | 11 +++ tutor/templates/k8s/deployments.yml | 63 +++++++++++++++-- tutor/templates/k8s/services.yml | 4 +- tutor/templates/local/docker-compose.yml | 60 +++++++++++++++- 20 files changed, 230 insertions(+), 132 deletions(-) delete mode 100644 tutor/templates/build/openedx-dev/Dockerfile delete mode 100755 tutor/templates/build/openedx-dev/bin/create-user.sh delete mode 100644 tutor/templates/build/openedx-dev/bin/docker-entrypoint.sh create mode 100644 tutor/templates/build/permissions/Dockerfile create mode 100644 tutor/templates/build/permissions/setowner.sh diff --git a/CHANGELOG-nightly.md b/CHANGELOG-nightly.md index 4844fda..8e6ffe1 100644 --- a/CHANGELOG-nightly.md +++ b/CHANGELOG-nightly.md @@ -2,7 +2,10 @@ Note: Breaking changes between versions are indicated by "💥". +- 💥[Improvement] Run all services as unprivileged containers, for better security. This has multiple consequences: + - The "openedx-dev" image is now built with `tutor dev dc build lms`. + - The "smtp" service now runs the "devture/exim-relay" Docker image, which is unprivileged. Also, the default SMTP port is now 8025. - 💥[Feature] Get rid of the nginx container and service, which is now replaced by Caddy. this has the following consequences: - Patches "nginx-cms", "nginx-lms", "nginx-extra", "local-docker-compose-nginx-aliases" are replaced by "caddyfile-cms", "caddyfile-lms", "caddyfile", " local-docker-compose-caddy-aliases". - Patches "k8s-deployments-nginx-volume-mounts", "k8s-deployments-nginx-volumes" were obsolete and are removed. - - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`. \ No newline at end of file + - The `NGINX_HTTP_PORT` setting is renamed to `CADDY_HTTP_PORT`. diff --git a/docs/configuration.rst b/docs/configuration.rst index 5408dc9..b681016 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -352,7 +352,7 @@ And djangojs.po:: Then you will have to re-build the openedx Docker image:: - tutor images build openedx openedx-dev + tutor images build openedx Beware that this will take a long time! Unfortunately it's difficult to accelerate this process, as translation files need to be compiled prior to collecting the assets. In development it's possible to accelerate the iteration loop -- but that exercise is left to the reader. diff --git a/docs/dev.rst b/docs/dev.rst index 1209c72..9dcb3a1 100644 --- a/docs/dev.rst +++ b/docs/dev.rst @@ -19,7 +19,7 @@ Once the local platform has been configured, you should stop it so that it does Finally, you should build the ``openedx-dev`` docker image:: - tutor images build openedx-dev + tutor dev dc build lms This ``openedx-dev`` development image differs from the ``openedx`` production image: diff --git a/docs/tutorials/theming.rst b/docs/tutorials/theming.rst index 4f6cfb2..bc514b9 100644 --- a/docs/tutorials/theming.rst +++ b/docs/tutorials/theming.rst @@ -50,10 +50,6 @@ The LMS can then be accessed at http://local.overhang.io:8000. You will then hav tutor dev settheme mythemename -Re-build development docker image (and compile assets):: - - tutor images build openedx-dev - Watch the themes folders for changes (in a different terminal):: tutor dev run watchthemes diff --git a/tutor/bindmounts.py b/tutor/bindmounts.py index 0fa1898..bbefca7 100644 --- a/tutor/bindmounts.py +++ b/tutor/bindmounts.py @@ -34,6 +34,7 @@ chown -R {user_id} {volumes_path}/{volume_name}""".format( "run", "--rm", "--no-deps", + "--user=0", "--volume", "{}:{}".format(volumes_root_path, container_volumes_root_path), service, diff --git a/tutor/commands/images.py b/tutor/commands/images.py index bae0ad5..0e6da5a 100644 --- a/tutor/commands/images.py +++ b/tutor/commands/images.py @@ -8,11 +8,9 @@ from .. import exceptions from .. import images from .. import plugins from ..types import Config -from .. import utils from .context import Context -BASE_IMAGE_NAMES = ["openedx", "forum"] -DEV_IMAGE_NAMES = ["openedx-dev"] +BASE_IMAGE_NAMES = ["openedx", "forum", "permissions"] VENDOR_IMAGES = [ "caddy", "elasticsearch", @@ -136,13 +134,6 @@ def build_image(root: str, config: Config, image: str, *args: str) -> None: (tutor_env.pathjoin(root, "plugins", plugin, "build", img), tag, args) ) - # Build dev images with user id argument - dev_build_arg = ("--build-arg", "USERID={}".format(utils.get_user_id())) - for img, tag in iter_images(config, image, DEV_IMAGE_NAMES): - to_build.append( - (tutor_env.pathjoin(root, "build", img), tag, dev_build_arg + args) - ) - if not to_build: raise ImageNotFoundError(image) diff --git a/tutor/env.py b/tutor/env.py index f591db3..7639415 100644 --- a/tutor/env.py +++ b/tutor/env.py @@ -59,6 +59,7 @@ class Renderer: environment.globals["rsa_import_key"] = utils.rsa_import_key environment.filters["rsa_private_key"] = utils.rsa_private_key environment.filters["walk_templates"] = self.walk_templates + environment.globals["HOST_USER_ID"] = utils.get_user_id() environment.globals["TUTOR_APP"] = __app__.replace("-", "_") environment.globals["TUTOR_VERSION"] = __version__ self.environment = environment diff --git a/tutor/jobs.py b/tutor/jobs.py index 87b7396..2238bad 100644 --- a/tutor/jobs.py +++ b/tutor/jobs.py @@ -50,6 +50,7 @@ class BaseComposeJobRunner(BaseJobRunner): def initialise(runner: BaseJobRunner, limit_to: Optional[str] = None) -> None: fmt.echo_info("Initialising all services...") if limit_to is None or limit_to == "mysql": + fmt.echo_info("Initialising mysql...") runner.run_job_from_template("mysql", "hooks", "mysql", "init") for plugin_name, hook in runner.iter_plugin_hooks("pre-init"): if limit_to is None or limit_to == plugin_name: diff --git a/tutor/templates/build/forum/Dockerfile b/tutor/templates/build/forum/Dockerfile index 0bbbbd2..cdb059e 100644 --- a/tutor/templates/build/forum/Dockerfile +++ b/tutor/templates/build/forum/Dockerfile @@ -1,4 +1,4 @@ -FROM docker.io/ubuntu:20.04 +FROM docker.io/ruby:2.5.7-slim-stretch MAINTAINER Overhang.io ENV DEBIAN_FRONTEND=noninteractive @@ -12,32 +12,27 @@ RUN wget -O /tmp/dockerize.tar.gz https://github.com/jwilder/dockerize/releases/ && tar -C /usr/local/bin -xzvf /tmp/dockerize.tar.gz \ && rm /tmp/dockerize.tar.gz -RUN mkdir /openedx +# Create unprivileged "app" user +RUN useradd --home-dir /app --create-home --shell /bin/bash --uid 1000 app -# Install ruby-build for building specific version of ruby -# The ruby-build version should be periodically updated to reflect the latest release -ARG RUBY_BUILD_VERSION=v20200401 -RUN git clone https://github.com/rbenv/ruby-build.git --branch $RUBY_BUILD_VERSION /openedx/ruby-build -WORKDIR /openedx/ruby-build -RUN PREFIX=/usr/local ./install.sh +# Copy custom scripts +COPY ./bin /app/bin +RUN chmod a+x /app/bin/* +ENV PATH :${PATH} -# Install ruby and some specific dependencies -ARG RUBY_VERSION=2.5.7 -ARG BUNDLER_VERSION=1.17.3 -ARG RAKE_VERSION=13.0.1 -RUN ruby-build $RUBY_VERSION /openedx/ruby -ENV PATH "/openedx/ruby/bin:$PATH" -RUN gem install bundler -v $BUNDLER_VERSION -RUN gem install rake -v $RAKE_VERSION +# From then on, run as unprivileged app user +USER app + +# Install rake and bundler +ENV PATH "/app/bin:/app/.gem/ruby/2.5.0/bin:$PATH" +RUN gem install --user-install bundler --version 1.17.3 +RUN gem install --user-install rake --version 13.0.1 # Install forum -RUN git clone https://github.com/edx/cs_comments_service.git --branch {{ OPENEDX_COMMON_VERSION }} --depth 1 /openedx/cs_comments_service -WORKDIR /openedx/cs_comments_service +RUN git clone https://github.com/edx/cs_comments_service.git --branch {{ OPENEDX_COMMON_VERSION }} --depth 1 /app/cs_comments_service +WORKDIR /app/cs_comments_service RUN bundle install --deployment -COPY ./bin /openedx/bin -RUN chmod a+x /openedx/bin/* -ENV PATH /openedx/bin:${PATH} ENTRYPOINT ["docker-entrypoint.sh"] ENV SINATRA_ENV staging diff --git a/tutor/templates/build/openedx-dev/Dockerfile b/tutor/templates/build/openedx-dev/Dockerfile deleted file mode 100644 index 8e55797..0000000 --- a/tutor/templates/build/openedx-dev/Dockerfile +++ /dev/null @@ -1,34 +0,0 @@ -FROM {{ DOCKER_IMAGE_OPENEDX }} as base -MAINTAINER Overhang.io - -# Install useful system requirements -RUN apt update && \ - apt install -y vim iputils-ping dnsutils telnet \ - && rm -rf /var/lib/apt/lists/* - -# Install dev python requirements -RUN pip install -r requirements/edx/development.txt -RUN pip install ipdb==0.13.4 ipython==7.27.0 - -{{ patch("openedx-dev-dockerfile-post-python-requirements") }} - -# Recompile static assets: in development mode all static assets are stored in edx-platform, -# and the location of these files is stored in webpack-stats.json. If we don't recompile -# static assets, then production assets will be served instead. -RUN rm -r /openedx/staticfiles && \ - mkdir /openedx/staticfiles && \ - openedx-assets webpack --env=dev - -# Copy new entrypoint (to take care of permission issues at runtime) -COPY ./bin /openedx/bin -RUN chmod a+x /openedx/bin/* - -# Configure new user -ARG USERID=1000 -RUN create-user.sh $USERID - -######## Development image -FROM base as dev - -# Default django settings -ENV SETTINGS tutor.development diff --git a/tutor/templates/build/openedx-dev/bin/create-user.sh b/tutor/templates/build/openedx-dev/bin/create-user.sh deleted file mode 100755 index 4b91302..0000000 --- a/tutor/templates/build/openedx-dev/bin/create-user.sh +++ /dev/null @@ -1,11 +0,0 @@ -#! /bin/sh -e -USERID=$1 - -if [ "$USERID" != "" ] && [ "$USERID" != "0" ] -then - echo "Creating 'openedx' user with id $USERID" - useradd --home-dir /openedx --uid $USERID openedx - chown -R openedx:openedx /openedx -else - echo "Running as root" -fi \ No newline at end of file diff --git a/tutor/templates/build/openedx-dev/bin/docker-entrypoint.sh b/tutor/templates/build/openedx-dev/bin/docker-entrypoint.sh deleted file mode 100644 index 7c10997..0000000 --- a/tutor/templates/build/openedx-dev/bin/docker-entrypoint.sh +++ /dev/null @@ -1,19 +0,0 @@ -#!/bin/sh -e -export DJANGO_SETTINGS_MODULE=$SERVICE_VARIANT.envs.$SETTINGS - -if id -u openedx > /dev/null 2>&1; then - # Change owners of mounted volumes - echo "Setting file permissions for user openedx..." - find /openedx \ - -not -path "/openedx/edx-platform/*" \ - -not -user openedx \ - -writable \ - -exec chown openedx:openedx {} \+ - echo "File permissions set." - - # Run CMD as user openedx - exec chroot --userspec="openedx:openedx" --skip-chdir / env HOME=/openedx "$@" -else - echo "Running openedx-dev as root user" - exec "$@" -fi diff --git a/tutor/templates/build/openedx/Dockerfile b/tutor/templates/build/openedx/Dockerfile index b4e728b..7ea82cd 100644 --- a/tutor/templates/build/openedx/Dockerfile +++ b/tutor/templates/build/openedx/Dockerfile @@ -127,14 +127,19 @@ RUN apt update && \ apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng-dev libsqlite3-dev libxmlsec1-dev lynx ntp pkg-config rdfind && \ rm -rf /var/lib/apt/lists/* -COPY --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize -COPY --from=code /openedx/edx-platform /openedx/edx-platform -COPY --from=locales /openedx/locale/contrib/locale /openedx/locale/contrib/locale -COPY --from=python /opt/pyenv /opt/pyenv -COPY --from=python-requirements /openedx/venv /openedx/venv -COPY --from=python-requirements /openedx/requirements /openedx/requirements -COPY --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv -COPY --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/edx-platform/node_modules +# From then on, run as unprivileged "app" user +ARG APP_USER_ID=1000 +RUN useradd --home-dir /openedx --create-home --shell /bin/bash --uid ${APP_USER_ID} app +USER ${APP_USER_ID} + +COPY --chown=app:app --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize +COPY --chown=app:app --from=code /openedx/edx-platform /openedx/edx-platform +COPY --chown=app:app --from=locales /openedx/locale /openedx/locale +COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv +COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv +COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements +COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv +COPY --chown=app:app --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/edx-platform/node_modules ENV PATH /openedx/venv/bin:./node_modules/.bin:/openedx/nodeenv/bin:${PATH} ENV VIRTUAL_ENV /openedx/venv/ @@ -146,16 +151,16 @@ RUN pip install -r requirements/edx/local.in # Create folder that will store lms/cms.env.json files, as well as # the tutor-specific settings files. RUN mkdir -p /openedx/config ./lms/envs/tutor ./cms/envs/tutor -COPY revisions.yml /openedx/config/ +COPY --chown=app:app revisions.yml /openedx/config/ ENV LMS_CFG /openedx/config/lms.env.json ENV STUDIO_CFG /openedx/config/cms.env.json ENV REVISION_CFG /openedx/config/revisions.yml -COPY settings/lms/*.py ./lms/envs/tutor/ -COPY settings/cms/*.py ./cms/envs/tutor/ +COPY --chown=app:app settings/lms/*.py ./lms/envs/tutor/ +COPY --chown=app:app settings/cms/*.py ./cms/envs/tutor/ # Copy user-specific locales to /openedx/locale/user/locale and compile them -RUN mkdir -p /openedx/locale/user -COPY ./locale/ /openedx/locale/user/locale/ +RUN mkdir /openedx/locale/user +COPY --chown=app:app ./locale/ /openedx/locale/user/locale/ RUN cd /openedx/locale/user && \ django-admin.py compilemessages -v1 @@ -166,7 +171,7 @@ RUN ./manage.py lms --settings=tutor.i18n compilejsi18n RUN ./manage.py cms --settings=tutor.i18n compilejsi18n # Copy scripts -COPY ./bin /openedx/bin +COPY --chown=app:app ./bin /openedx/bin RUN chmod a+x /openedx/bin/* ENV PATH /openedx/bin:${PATH} @@ -188,7 +193,7 @@ RUN openedx-assets xmodule \ && openedx-assets npm \ && openedx-assets webpack --env=prod \ && openedx-assets common -COPY ./themes/ /openedx/themes/ +COPY --chown=app:app ./themes/ /openedx/themes/ RUN openedx-assets themes \ && openedx-assets collect --settings=tutor.assets \ # De-duplicate static assets with symlinks @@ -205,9 +210,40 @@ ENV SETTINGS tutor.production # Entrypoint will set right environment variables ENTRYPOINT ["docker-entrypoint.sh"] +EXPOSE 8000 + +###### Intermediate image with dev/test dependencies +FROM production as development + +# Install useful system requirements (as root) +USER root +RUN apt update && \ + apt install -y vim iputils-ping dnsutils telnet \ + && rm -rf /var/lib/apt/lists/* +USER app + +# Install dev python requirements +RUN pip install -r requirements/edx/development.txt +RUN pip install ipdb==0.13.4 ipython==7.27.0 + +# Recompile static assets: in development mode all static assets are stored in edx-platform, +# and the location of these files is stored in webpack-stats.json. If we don't recompile +# static assets, then production assets will be served instead. +RUN rm -r /openedx/staticfiles && \ + mkdir /openedx/staticfiles && \ + openedx-assets webpack --env=dev + +{{ patch("openedx-dev-dockerfile-post-python-requirements") }} + +# Default django settings +ENV SETTINGS tutor.development + +CMD ./manage.py $SERVICE_VARIANT runserver 0.0.0.0:8000 + +###### Final image with production cmd +FROM production as final # Run server -EXPOSE 8000 CMD uwsgi \ --static-map /static=/openedx/staticfiles/ \ --static-map /media=/openedx/media/ \ diff --git a/tutor/templates/build/permissions/Dockerfile b/tutor/templates/build/permissions/Dockerfile new file mode 100644 index 0000000..1ddcc5f --- /dev/null +++ b/tutor/templates/build/permissions/Dockerfile @@ -0,0 +1,7 @@ +from docker.io/alpine:3.13.6 +MAINTAINER Overhang.io + +COPY ./setowner.sh /usr/local/bin/setowner +RUN chmod a+x /usr/local/bin/setowner + +ENTRYPOINT ["setowner"] diff --git a/tutor/templates/build/permissions/setowner.sh b/tutor/templates/build/permissions/setowner.sh new file mode 100644 index 0000000..f0a3ea9 --- /dev/null +++ b/tutor/templates/build/permissions/setowner.sh @@ -0,0 +1,14 @@ +#! /bin/sh +set -e +user_id="$1" +shift +for path in $@; do + path_user_id="$(stat -c '%u' $path)" + if [ "$path_user_id" != "$user_id" ] + then + echo "$path changing UID from $path_user_id to $user_id..." + chown --recursive $user_id $path + else + echo "$path already owned by $user_id" + fi +done diff --git a/tutor/templates/config.yml b/tutor/templates/config.yml index fdd488d..ce0ec6a 100644 --- a/tutor/templates/config.yml +++ b/tutor/templates/config.yml @@ -26,7 +26,7 @@ OPENEDX_AWS_SECRET_ACCESS_KEY: "" DEV_PROJECT_NAME: "tutor_dev" DOCKER_REGISTRY: "docker.io/" DOCKER_IMAGE_OPENEDX: "{{ DOCKER_REGISTRY }}overhangio/openedx:{{ TUTOR_VERSION }}" -DOCKER_IMAGE_OPENEDX_DEV: "{{ DOCKER_REGISTRY }}overhangio/openedx-dev:{{ TUTOR_VERSION }}" +DOCKER_IMAGE_OPENEDX_DEV: "openedx-dev" DOCKER_IMAGE_CADDY: "{{ DOCKER_REGISTRY }}caddy:2.3.0" DOCKER_IMAGE_ELASTICSEARCH: "{{ DOCKER_REGISTRY }}elasticsearch:7.10.1" DOCKER_IMAGE_FORUM: "{{ DOCKER_REGISTRY }}overhangio/openedx-forum:{{ TUTOR_VERSION }}" @@ -34,8 +34,9 @@ DOCKER_IMAGE_MONGODB: "{{ DOCKER_REGISTRY }}mongo:4.2.17" DOCKER_IMAGE_MYSQL: "{{ DOCKER_REGISTRY }}mysql:5.7.35" DOCKER_IMAGE_ELASTICSEARCH: "{{ DOCKER_REGISTRY }}elasticsearch:7.10.1" DOCKER_IMAGE_NGINX: "{{ DOCKER_REGISTRY }}nginx:1.21.1" +DOCKER_IMAGE_PERMISSIONS: "{{ DOCKER_REGISTRY }}alpine:3.13.6" DOCKER_IMAGE_REDIS: "{{ DOCKER_REGISTRY }}redis:6.2.6" -DOCKER_IMAGE_SMTP: "{{ DOCKER_REGISTRY }}namshi/smtp:latest" +DOCKER_IMAGE_SMTP: "{{ DOCKER_REGISTRY }}devture/exim-relay:4.94.2-r0-4" LOCAL_PROJECT_NAME: "{{ TUTOR_APP }}_local" ELASTICSEARCH_HOST: "elasticsearch" ELASTICSEARCH_PORT: 9200 @@ -77,7 +78,7 @@ REDIS_PORT: 6379 REDIS_USERNAME: "" REDIS_PASSWORD: "" SMTP_HOST: "smtp" -SMTP_PORT: 25 +SMTP_PORT: 8025 SMTP_USERNAME: "" SMTP_PASSWORD: "" SMTP_USE_TLS: false diff --git a/tutor/templates/dev/docker-compose.yml b/tutor/templates/dev/docker-compose.yml index 634c44e..c984398 100644 --- a/tutor/templates/dev/docker-compose.yml +++ b/tutor/templates/dev/docker-compose.yml @@ -3,6 +3,11 @@ version: "3.7" x-openedx-service: &openedx-service image: {{ DOCKER_IMAGE_OPENEDX_DEV }} + build: + context: ../build/openedx/ + target: development + args: + APP_USER_ID: "{{ HOST_USER_ID }}" environment: SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.development} volumes: @@ -16,6 +21,12 @@ x-openedx-service: - ../build/openedx/requirements:/openedx/requirements services: + lms-permissions: + command: ["{{ HOST_USER_ID }}", "/openedx/data", "/openedx/media"] + + cms-permissions: + command: ["{{ HOST_USER_ID }}", "/openedx/data", "/openedx/media"] + lms: <<: *openedx-service command: ./manage.py lms runserver 0.0.0.0:8000 diff --git a/tutor/templates/k8s/deployments.yml b/tutor/templates/k8s/deployments.yml index 9541ebc..ddfa4f1 100644 --- a/tutor/templates/k8s/deployments.yml +++ b/tutor/templates/k8s/deployments.yml @@ -51,6 +51,9 @@ spec: labels: app.kubernetes.io/name: cms spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 containers: - name: cms image: {{ DOCKER_IMAGE_OPENEDX }} @@ -69,6 +72,8 @@ spec: resources: requests: memory: 2Gi + securityContext: + allowPrivilegeEscalation: false volumes: - name: settings-lms configMap: @@ -95,6 +100,9 @@ spec: labels: app.kubernetes.io/name: cms-worker spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 containers: - name: cms-worker image: {{ DOCKER_IMAGE_OPENEDX }} @@ -102,8 +110,6 @@ spec: env: - name: SERVICE_VARIANT value: cms - - name: C_FORCE_ROOT - value: "1" volumeMounts: - mountPath: /openedx/edx-platform/lms/envs/tutor/ name: settings-lms @@ -111,6 +117,8 @@ spec: name: settings-cms - mountPath: /openedx/config name: config + securityContext: + allowPrivilegeEscalation: false volumes: - name: settings-lms configMap: @@ -139,6 +147,9 @@ spec: labels: app.kubernetes.io/name: forum spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 containers: - name: forum image: {{ DOCKER_IMAGE_FORUM }} @@ -155,6 +166,8 @@ spec: value: "{{ MONGODB_PORT }}" - name: MONGODB_DATABASE value: "{{ FORUM_MONGODB_DATABASE }}" + securityContext: + allowPrivilegeEscalation: false {% endif %} {% if RUN_LMS %} --- @@ -173,6 +186,9 @@ spec: labels: app.kubernetes.io/name: lms spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 containers: - name: lms image: {{ DOCKER_IMAGE_OPENEDX }} @@ -188,6 +204,8 @@ spec: resources: requests: memory: 2Gi + securityContext: + allowPrivilegeEscalation: false volumes: - name: settings-lms configMap: @@ -214,6 +232,9 @@ spec: labels: app.kubernetes.io/name: lms-worker spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 containers: - name: lms-worker image: {{ DOCKER_IMAGE_OPENEDX }} @@ -221,8 +242,6 @@ spec: env: - name: SERVICE_VARIANT value: lms - - name: C_FORCE_ROOT - value: "1" volumeMounts: - mountPath: /openedx/edx-platform/lms/envs/tutor/ name: settings-lms @@ -230,6 +249,8 @@ spec: name: settings-cms - mountPath: /openedx/config name: config + securityContext: + allowPrivilegeEscalation: false volumes: - name: settings-lms configMap: @@ -260,6 +281,11 @@ spec: labels: app.kubernetes.io/name: elasticsearch spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + fsGroupChangePolicy: "OnRootMismatch" containers: - name: elasticsearch image: {{ DOCKER_IMAGE_ELASTICSEARCH }} @@ -276,6 +302,8 @@ spec: value: "1" ports: - containerPort: 9200 + securityContext: + allowPrivilegeEscalation: false volumeMounts: - mountPath: /usr/share/elasticsearch/data name: data @@ -303,6 +331,11 @@ spec: labels: app.kubernetes.io/name: mongodb spec: + securityContext: + runAsUser: 999 + runAsGroup: 999 + fsGroup: 999 + fsGroupChangePolicy: "OnRootMismatch" containers: - name: mongodb image: {{ DOCKER_IMAGE_MONGODB }} @@ -312,7 +345,8 @@ spec: volumeMounts: - mountPath: /data/db name: data - + securityContext: + allowPrivilegeEscalation: false volumes: - name: data persistentVolumeClaim: @@ -337,6 +371,11 @@ spec: labels: app.kubernetes.io/name: mysql spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + fsGroupChangePolicy: "OnRootMismatch" containers: - name: mysql image: {{ DOCKER_IMAGE_MYSQL }} @@ -351,6 +390,8 @@ spec: volumeMounts: - mountPath: /var/lib/mysql name: data + securityContext: + allowPrivilegeEscalation: false volumes: - name: data persistentVolumeClaim: @@ -373,11 +414,14 @@ spec: labels: app.kubernetes.io/name: smtp spec: + securityContext: + runAsUser: 100 + runAsGroup: 101 containers: - name: smtp image: {{ DOCKER_IMAGE_SMTP }} ports: - - containerPort: 25 + - containerPort: 8025 {% endif %} {% if RUN_REDIS %} --- @@ -398,6 +442,11 @@ spec: labels: app.kubernetes.io/name: redis spec: + securityContext: + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + fsGroupChangePolicy: "OnRootMismatch" containers: - name: redis image: {{ DOCKER_IMAGE_REDIS }} @@ -410,6 +459,8 @@ spec: name: config - mountPath: /openedx/redis/data name: data + securityContext: + allowPrivilegeEscalation: false volumes: - name: config configMap: diff --git a/tutor/templates/k8s/services.yml b/tutor/templates/k8s/services.yml index 2abcd71..68e78ec 100644 --- a/tutor/templates/k8s/services.yml +++ b/tutor/templates/k8s/services.yml @@ -121,9 +121,9 @@ metadata: spec: type: NodePort ports: - - port: 25 + - port: 8025 protocol: TCP selector: app.kubernetes.io/name: smtp {% endif %} -{{ patch("k8s-services") }} \ No newline at end of file +{{ patch("k8s-services") }} diff --git a/tutor/templates/local/docker-compose.yml b/tutor/templates/local/docker-compose.yml index cb130d8..4d27c24 100644 --- a/tutor/templates/local/docker-compose.yml +++ b/tutor/templates/local/docker-compose.yml @@ -9,6 +9,16 @@ services: # Use WiredTiger in all environments, just like at edx.org command: mongod --nojournal --storageEngine wiredTiger restart: unless-stopped + user: "999:999" + privileged: false + volumes: + - ../../data/mongodb:/data/db + depends_on: + - mongodb-permissions + mongodb-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["999", "/data/db"] + restart: on-failure volumes: - ../../data/mongodb:/data/db {% endif %} @@ -18,10 +28,18 @@ services: image: {{ DOCKER_IMAGE_MYSQL }} command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci restart: unless-stopped + user: "1000:1000" + privileged: false volumes: - ../../data/mysql:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: "{{ MYSQL_ROOT_PASSWORD }}" + mysql-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["1000", "/var/lib/mysql"] + restart: on-failure + volumes: + - ../../data/mysql:/var/lib/mysql {% endif %} {% if RUN_ELASTICSEARCH %} @@ -32,12 +50,20 @@ services: - bootstrap.memory_lock=true - discovery.type=single-node - "ES_JAVA_OPTS=-Xms{{ ELASTICSEARCH_HEAP_SIZE }} -Xmx{{ ELASTICSEARCH_HEAP_SIZE }}" - - TAKE_FILE_OWNERSHIP=1 ulimits: memlock: soft: -1 hard: -1 restart: unless-stopped + user: "1000:1000" + volumes: + - ../../data/elasticsearch:/usr/share/elasticsearch/data + depends_on: + - elasticsearch-permissions + elasticsearch-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["1000", "/usr/share/elasticsearch/data"] + restart: on-failure volumes: - ../../data/elasticsearch:/usr/share/elasticsearch/data {% endif %} @@ -46,17 +72,29 @@ services: redis: image: {{ DOCKER_IMAGE_REDIS }} working_dir: /openedx/redis/data + user: "1000:1000" volumes: - ../apps/redis/redis.conf:/openedx/redis/config/redis.conf:ro - ../../data/redis:/openedx/redis/data command: redis-server /openedx/redis/config/redis.conf restart: unless-stopped + depends_on: + - redis-permissions + redis-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["1000", "/openedx/redis/data"] + restart: on-failure + volumes: + - ../../data/redis:/openedx/redis/data {% endif %} {% if RUN_SMTP %} smtp: image: {{ DOCKER_IMAGE_SMTP }} restart: unless-stopped + user: "100:101" + environment: + HOSTNAME: "{{ LMS_HOST }}" {% endif %} ############# Forum @@ -91,6 +129,7 @@ services: - ../../data/lms:/openedx/data - ../../data/openedx-media:/openedx/media depends_on: + - lms-permissions {% if RUN_MYSQL %}- mysql{% endif %} {% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %} {% if RUN_FORUM %}- forum{% endif %} @@ -98,6 +137,14 @@ services: {% if RUN_REDIS %}- redis{% endif %} {% if RUN_SMTP %}- smtp{% endif %} {{ patch("local-docker-compose-lms-dependencies")|indent(6) }} + lms-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["1000", "/openedx/data", "/openedx/media"] + restart: on-failure + volumes: + - ../../data/redis:/openedx/redis/data + - ../../data/lms:/openedx/data + - ../../data/openedx-media:/openedx/media {% endif %} {% if RUN_CMS %} @@ -115,6 +162,7 @@ services: - ../../data/cms:/openedx/data - ../../data/openedx-media:/openedx/media depends_on: + - cms-permissions {% if RUN_MYSQL %}- mysql{% endif %} {% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %} {% if RUN_MONGODB %}- mongodb{% endif %} @@ -122,6 +170,14 @@ services: {% if RUN_SMTP %}- smtp{% endif %} {% if RUN_LMS %}- lms{% endif %} {{ patch("local-docker-compose-cms-dependencies")|indent(6) }} + cms-permissions: + image: {{ DOCKER_IMAGE_PERMISSIONS }} + command: ["1000", "/openedx/data", "/openedx/media"] + restart: on-failure + volumes: + - ../../data/redis:/openedx/redis/data + - ../../data/cms:/openedx/data + - ../../data/openedx-media:/openedx/media {% endif %} ############# LMS and CMS workers @@ -132,7 +188,6 @@ services: environment: SERVICE_VARIANT: lms SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production} - C_FORCE_ROOT: "1" # run celery tasks as root #nofear command: celery worker --app=lms.celery --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild=100 --exclude-queues=edx.cms.core.default restart: unless-stopped volumes: @@ -151,7 +206,6 @@ services: environment: SERVICE_VARIANT: cms SETTINGS: ${TUTOR_EDX_PLATFORM_SETTINGS:-tutor.production} - C_FORCE_ROOT: "1" # run celery tasks as root #nofear command: celery worker --app=cms.celery --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.lms.core.default restart: unless-stopped volumes: