From 6980550b7c2c46e14245f007f662db51d035ab40 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Sun, 16 Aug 2020 16:05:06 +0200 Subject: [PATCH] Clarify image downloading in k8s docs Close #362 --- docs/configuration.rst | 16 ++++++++++------ docs/k8s.rst | 35 ++++++++++++++++++++++------------- docs/local.rst | 22 +++++++++++----------- 3 files changed, 43 insertions(+), 30 deletions(-) diff --git a/docs/configuration.rst b/docs/configuration.rst index ab42d84..a944110 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -186,10 +186,12 @@ To renew the certificate, run this command once per month:: .. _customise: +.. _custom_openedx_docker_image: + Custom Open edX docker image ---------------------------- -There are different ways you can customise your Open edX platform. For instance, optional features can be activated during configuration. But if you want to add unique features to your Open edX platform, you are going to have to modify and re-build the ``openedx`` docker image. This is the image that contains the ``edx-platform`` repository: it is in charge of running the web application for the Open edX "core". Both the LMS and the CMS run from the ``openedx`` docker image. +There are different ways you can customise your Open edX platform. For instance, optional features can be activated during configuration. But if you want to add unique features to your Open edX platform, you are going to have to modify and re-build the ``openedx`` docker image. This is the image that contains the ``edx-platform`` repository: it is in charge of running the web application for the Open edX "core". Both the LMS and the CMS run from the ``openedx`` docker image. On a vanilla platform deployed by Tutor, the image that is run is downloaded from the `overhangio/openedx repository on Docker Hub `_. This is also the image that is downloaded whenever we run ``tutor local pullimages``. But you can decide to build the image locally instead of downloading it. To do so, build and tag the ``openedx`` image:: @@ -212,7 +214,7 @@ When building the "openedx" Docker image, it is possible to specify a few `argum - ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``) These arguments can be specified from the command line, `very much like Docker `__. For instance:: - + tutor images build -a EDX_PLATFORM_VERSION=customsha1 openedx Adding custom themes @@ -241,6 +243,8 @@ Then you must rebuild the openedx Docker image:: Finally, you should enable your theme with the :ref:`settheme command `. +.. _custom_extra_xblocks: + Installing extra xblocks and requirements ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -281,13 +285,13 @@ Adding custom translations If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform `__ as well as those from `openedx-i18n `__. Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance:: - + mkdir -p "$(tutor config printroot)/env/build/openedx/locale/fr/LC_MESSAGES" The language code should be similar to those used in edx-platform or openedx-i18n (see links above). Then, add a "django.po" file there that will contain your custom translations:: - + msgid "String to translate" msgstr "你翻译的东西 la traduction de votre bidule" @@ -296,7 +300,7 @@ The "String to translate" part should match *exactly* the string that you would If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well `__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory. To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy:: - + $(tutor config printroot)/env/build/openedx/locale/ fr/ LC_MESSAGES/ @@ -304,7 +308,7 @@ To recap, here is an example. To translate a few strings in French, both from dj djangojs.po With django.po containing:: - + msgid "It works! Powered by Open edX{registered_trademark}" msgstr "Ça marche ! Propulsé by Open edX{registered_trademark}" diff --git a/docs/k8s.rst b/docs/k8s.rst index bcf8d31..3889da3 100644 --- a/docs/k8s.rst +++ b/docs/k8s.rst @@ -40,11 +40,11 @@ See the `official instructions `__ Issuer that delivers `Let's Encrypt `__ certificates. You may examine the configuration in ``$(tutor config printroot)/env/k8s/ingress.yml``. If you are happy with this configuration, you may apply it with:: - + kubectl apply -k $(tutor config printroot)/env --selector="app.kubernetes.io/component in (ingress, issuer)" On Minikube, run:: - + minikube addons enable ingress With Kubernetes, your Open edX platform will *not* be available at localhost. Instead, you will have to access your platform with the domain names you specified for the LMS and the CMS. To do so on a local computer, you will need to add the following line to /etc/hosts:: @@ -61,19 +61,19 @@ Tutor relies on `cert-manager `_ to generate TLS kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.0/cert-manager.yaml If you decide to enable HTTPS certificates, you will also have to set ``WEB_PROXY=true`` in the platform configuration, because the SSL/TLS termination will not occur in the Nginx container, but in the Ingress controller. To do so, run:: - + tutor config save --set WEB_PROXY=true Note that this configuration might conflict with a local installation. .. warning:: On DigitalOcean, there is currently a bug that prevents certificate issuers from successfully fetching TLS certificates from Let's Encrypt. A workaround consists in adding a custom annotation to the "ingress-nginx" service:: - + kubectl -n ingress-nginx patch service ingress-nginx -p \ '{"metadata": {"annotations": {"service.beta.kubernetes.io/do-loadbalancer-hostname": "YOURLMSHOSTHERE"}}}' - + Sources: - + * https://www.digitalocean.com/community/questions/how-do-i-correct-a-connection-timed-out-error-during-http-01-challenge-propagation-with-cert-manager * https://www.digitalocean.com/community/questions/pod-unable-to-curl-loadbalancer * https://github.com/jetstack/cert-manager/issues/863#issuecomment-567062996 @@ -85,7 +85,7 @@ S3-like object storage with `MinIO `_ Like many web applications, Open edX needs to persist data. In particular, it needs to persist files uploaded by students and course designers. In the local installation, these files are persisted to disk, on the host filesystem. But on Kubernetes, it is difficult to share a single filesystem between different pods. This would require persistent volume claims with `ReadWriteMany` access mode, and these are difficult to setup. Luckily, there is another solution: at `edx.org `_, uploaded files are persisted on AWS S3: Open edX is compatible out-of-the-box with the S3 API for storing user-generated files. The problem with S3 is that it introduces a dependency on AWS. To solve this problem, Tutor comes with a plugin that emulates the S3 API but stores files on premises. This is achieved thanks to `MinIO `_. If you want to deploy a production platform to Kubernetes, you will most certainly need to enable the ``minio`` plugin:: - + tutor plugins enable minio The "minio.LMS_HOST" domain name will have to point to your Kubernetes cluster. This will not be necessary if you have a CNAME from "\*.LMS_HOST" to "LMS_HOST", of course. @@ -96,7 +96,7 @@ Kubernetes dashboard This is not a requirement per se, but it's very convenient to have a visual interface of the Kubernetes cluster. We suggest the official `Kubernetes dashboard `_. Depending on your Kubernetes provider, you may need to install a dashboard yourself. There are generic instructions on the `project's README `_. AWS provides `specific instructions `_. On Minikube, the dashboard is already installed. To access the dashboard, run:: - + minikube dashboard Technical details @@ -107,7 +107,7 @@ Under the hood, Tutor wraps ``kubectl`` commands to interact with the cluster. T Basically, the whole platform is described in manifest files stored in ``$(tutor config printroot)/env/k8s``. There is also a ``kustomization.yml`` file at the project root for `declarative application management `_. This allows us to start and update resources with commands similar to ``kubectl apply -k $(tutor config printroot) --selector=...`` (see the ``kubectl apply`` `official documentation `_). The other benefit of ``kubectl apply`` is that it allows you to customise the Kubernetes resources as much as you want. For instance, the default Tutor configuration can be extended by a ``kustomization.yml`` file stored in ``$(tutor config printroot)/env-custom/`` and which would start with:: - + apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: @@ -134,11 +134,11 @@ Other commands -------------- As with the :ref:`local installation `, there are multiple commands to run operations on your Open edX platform. To view those commands, run:: - + tutor k8s -h - + In particular, the `tutor k8s start` command restarts and reconfigures all services by running ``kubectl apply``. That means that you can delete containers, deployments or just any other kind of resources, and Tutor will re-create them automatically. You should just beware of not deleting any persistent data stored in persistent volume claims. For instance, to restart from a "blank slate", run:: - + tutor k8s stop tutor k8s start @@ -147,11 +147,20 @@ All non-persisting data will be deleted, and then re-created. Guides ------ +Running a custom "openedx" Docker image +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some Tutor plugins and customization procedures require that the "openedx" image be rebuilt (see :ref:`customization `). This is for instance the case if you want to :ref:`install a custom XBlock ` or :ref:`run an edx-platform fork `. When running Open edX on Kubernetes, your custom images will have to be downloaded from a custom registry. You should define a custom image name, build the image and then push them to your custom registry. For instance, for the "openedx" image:: + + tutor config save --set "DOCKER_IMAGE_OPENEDX=docker.io/myusername/openedx:{{ TUTOR_VERSION }}" + tutor images build openedx + tutor images push openedx + Updating docker images ~~~~~~~~~~~~~~~~~~~~~~ Kubernetes does not provide a single command for updating docker images out of the box. A `commonly used trick `_ is to modify an innocuous label on all resources:: - + kubectl patch -k "$(tutor config printroot)/env" --patch "{\"spec\": {\"template\": {\"metadata\": {\"labels\": {\"date\": \"`date +'%Y%m%d-%H%M%S'`\"}}}}}" diff --git a/docs/local.rst b/docs/local.rst index 5541416..0b89505 100644 --- a/docs/local.rst +++ b/docs/local.rst @@ -6,7 +6,7 @@ Local deployment This method is for deploying Open edX locally on a single server, where docker images are orchestrated with `docker-compose `_. In the following, environment and data files will be generated in a user-specific project folder which will be referred to as the "**project root**". On Linux, the default project root is ``~/.local/share/tutor``. An alternative project root can be defined by passing the ``--root=...`` option to the ``tutor`` command, or defining the ``TUTOR_ROOT=...`` environment variable:: - + tutor --root=/path/to/tutorroot run ... # Or equivalently: export TUTOR_ROOT=/path/to/tutorroot @@ -69,7 +69,7 @@ In production, you will probably want to daemonize the services. To do so, run:: And then, to stop all services:: tutor local stop - + Service initialisation ~~~~~~~~~~~~~~~~~~~~~~ @@ -87,7 +87,7 @@ Logging By default, logs from all containers are forwarded to the `default Docker logging driver `_: this means that logs are printed to the standard output when running in non-daemon mode (``tutor local start``). In daemon mode, logs can still be accessed with ``tutor local logs`` commands (see :ref:`logging `). In addition, all LMS and CMS logs are persisted to disk by default in the following files:: - + $(tutor config printroot)/data/lms/logs/all.log $(tutor config printroot)/data/cms/logs/all.log @@ -126,7 +126,7 @@ Setting a new theme ~~~~~~~~~~~~~~~~~~~ The default Open edX theme is rather bland, so Tutor makes it easy to switch to a different theme:: - + tutor local settheme mytheme $(tutor config printvalue LMS_HOST) $(tutor config printvalue CMS_HOST) Notice that we pass the hostnames of the LMS and the CMS to the ``settheme`` command: this is because in Open edX, themes are assigned per domain name. @@ -137,7 +137,7 @@ Running arbitrary ``manage.py`` commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Any ``./manage.py`` command provided by Open edX can be run in a local platform deployed with Tutor. For instance, to delete a course, run:: - + tutor local run cms ./manage.py cms delete_course To update the course search index, run:: @@ -149,7 +149,7 @@ Reloading Open edX settings ~~~~~~~~~~~~~~~~~~~~~~~~~~~ After modifying Open edX settings, for instance when running ``tutor config save``, you will want to restart the web processes of the LMS and the CMS to take into account those new settings. It is possible to simply restart the whole platform (with ``tutor local reboot``) or just a single service (``tutor local restart lms``) but that is overkill. A quicker alternative is to send the HUP signal to the gunicorn processes running inside the containers. The "openedx" Docker image comes with a convenient script that does just that. To run it, execute:: - + tutor local exec lms reload-gunicorn .. _portainer: @@ -264,7 +264,7 @@ Then, install Tutor using one of the :ref:`installation methods `. Then mv config.json data/ "$(tutor config printroot)" Finally, launch your platform with:: - + tutor local quickstart Backups/Migrating to a different server @@ -274,19 +274,19 @@ With Tutor, all data are stored in a single folder. This means that it's extreme 1. Make sure `tutor` is installed on both servers with the same version. 2. Stop any running platform on server 1:: - + tutor local stop 3. Transfer the configuration, environment and platform data from server 1 to server 2:: - rsync -avr "$(tutor config printroot)/" username@server2:/tmp/tutor/ + rsync -avr "$(tutor config printroot)/" username@server2:/tmp/tutor/ 4. On server 2, move the data to the right location:: - + mv /tmp/tutor "$(tutor config printroot)" 5. Start the instance with:: - + tutor local start -d Making database dumps