Configurable docker images

Docker images for each service can now be configured one by one. Close #122.
2025-01-06 07:30:40 +00:00 · 2019-03-06 17:21:47 +01:00 · 2019-03-06 17:21:47 +01:00 · 6d19b99b66
commit 6d19b99b66
parent 976453c838
11 changed files with 75 additions and 36 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -2,6 +2,7 @@

 ## Latest

+- [Improvement] Configurable docker images (#122)
 - [Bugfix] Fix "android pullimage" command
 - [Improvement] Do not upgrade images as part of quickstart
 - [Bugfix] Fix USERID setup in development mode and various dev-related docs (#177)
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@ -1,12 +1,44 @@
-.. _options:
+.. _configuration:

-Optional features
-=================
+Configuration
+=============
+
+With Tutor, all Open edX deployment parameters are stored in a single ``config.yml`` file. This is the file that is generated when you run ``tutor local quickstart`` or ``tutor config interactive``. To view the content of this file, run::
+
+    cat $(tutor config printroot)/config.yml
+
+By default, this file contains only the required configuration parameters for running the platform. Optional configuration parameters may also be specified to modify the default behaviour. To do so, you can edit the ``config.yml`` file manually::
+
+    vim $(tutor config printroot)/config.yml
+
+Alternatively, you can set each parameter from the command line::
+
+    tutor config noninteractive --set PARAM1=VALUE1 --set PARAM2=VALUE2
+
+After changing a configuration parameter, it will be taken into account next time the environment is generated. For instance, in a local installation::
+
+    tutor local env
+
+.. _docker_images:
+
+``DOCKER_IMAGE_*`` Custom Docker images
+---------------------------------------
+
+These configuration parameters define which image to run for each service.
+
+- ``DOCKER_IMAGE_OPENEDX`` (default: ``regis/openedx:hawthorn``)
+- ``DOCKER_IMAGE_ANDROID`` (default: ``regis/openedx-android:hawthorn``)
+- ``DOCKER_IMAGE_FORUM`` (default: ``regis/openedx-forum:hawthorn``)
+- ``DOCKER_IMAGE_NOTES`` (default: ``regis/openedx-notes:hawthorn``)
+- ``DOCKER_IMAGE_XQUEUE`` (default: ``regis/openedx-xqueue:hawthorn``)
+
+``ACTIVATE_*`` Optional features
+--------------------------------

 Some optional features may be activated or deactivated during the interactive configuration step. These features change configuration files (during the ``configure`` step) as well as make targets.

-SSL/TLS certificates for HTTPS access
-------------------------------------
+``ACTIVATE_HTTPS`` SSL/TLS certificates for HTTPS access
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 By activating this feature, a free SSL/TLS certificate from the `Let's Encrypt <https://letsencrypt.org/>`_ certificate authority will be created for your platform. With this feature, **your platform will no longer be accessible in HTTP**. Calls to http urls will be redirected to https url.

@ -26,8 +58,8 @@ To renew the certificate, run this command once per month::

    tutor local https renew

-Student notes
-------------
+``ACTIVATE_NOTES`` Student notes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 With `notes <https://edx.readthedocs.io/projects/open-edx-building-and-running-a-course/en/open-release-hawthorn.master/exercises_tools/notes.html?highlight=notes>`_, students can annotate portions of the courseware. 

@ -36,7 +68,7 @@ With `notes <https://edx.readthedocs.io/projects/open-edx-building-and-running-a

 You should beware that the ``notes.<LMS_HOST>`` domain name should be activated and point to your server. For instance, if your LMS is hosted at http://myopenedx.com, the notes service should be found at http://notes.myopenedx.com.

-Xqueue
------
+``ACTIVATE_XQUEUE`` Xqueue
+~~~~~~~~~~~~~~~~~~~~~~~~~~

 `Xqueue <https://github.com/edx/xqueue>`_ is for grading problems with external services. If you don't know what it is, you probably don't need it.
--- a/docs/customise.rst
+++ b/docs/customise.rst
@ -84,7 +84,9 @@ By default, Tutor runs the `regis/openedx <https://hub.docker.com/r/regis/opened

 Then add the following value to your ``config.yml``::

-    OPENEDX_DOCKER_IMAGE: myusername/openedx:mytag
+    DOCKER_IMAGE_OPENEDX: myusername/openedx:mytag
+
+See the relevant :ref:`configuration parameters <docker_images>`.

 This value will then be used by Tutor when generating your environment. For instance, this is how you would use your image in a local deployment::

--- a/docs/index.rst
+++ b/docs/index.rst
@ -49,8 +49,8 @@ But there's a lot more to Tutor than that! For more advanced usage, please refer

   install
   quickstart
+   configuration
   local
-   options
   customise
   dev
   k8s
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@ -15,7 +15,7 @@ Quickstart

 Yes :) This is what happens when you run ``tutor local quickstart``:

-1. You answer a few questions about the configuration of your Open edX platform and your :ref:`selected options <options>`
+1. You answer a few questions about the :ref:`configuration` of your Open edX platform.
 2. Configuration files are generated from templates.
 3. Docker images are downloaded.
 4. Docker containers are provisioned.
--- a/tutor/android.py
+++ b/tutor/android.py
@ -7,8 +7,6 @@ from . import opts
 from . import utils


-DOCKER_IMAGE = "regis/openedx-android:hawthorn"
-
@click.group(
    help="Build an Android app for your Open edX platform [BETA FEATURE]"
 )
@ -54,13 +52,15 @@ def release(root):
 )
@opts.root
 def pullimage(root):
-    utils.execute("docker", "pull", DOCKER_IMAGE)
+    config = tutor_config.load(root)
+    utils.execute("docker", "pull", config['DOCKER_IMAGE_ANDROID'])

 def docker_run(root, *command):
+    config = tutor_config.load(root)
    utils.docker_run(
        "--volume={}:/openedx/config/".format(tutor_env.pathjoin(root, "android")),
        "--volume={}:/openedx/data/".format(tutor_env.data_path(root, "android")),
-        DOCKER_IMAGE,
+        config['DOCKER_IMAGE_ANDROID'],
        *command
    )

--- a/tutor/config.py
+++ b/tutor/config.py
@ -23,10 +23,10 @@ def config(root):
 )
@opts.root
@opts.key_value
-def interactive(root, s):
+def interactive(root, set_):
    config = {}
    load_files(config, root)
-    for k, v in s:
+    for k, v in set_:
        config[k] = v
    load_interactive(config)
    save(config, root)
@ -36,10 +36,10 @@ def interactive(root, s):
 )
@opts.root
@opts.key_value
-def noninteractive(root, s):
+def noninteractive(root, set_):
    config = {}
    load_files(config, root)
-    for k, v in s:
+    for k, v in set_:
        config[k] = v
    save(config, root)

--- a/tutor/opts.py
+++ b/tutor/opts.py
@ -38,6 +38,6 @@ class YamlParamType(click.ParamType):
        return (k, v)

 key_value = click.option(
-    "-s", type=YamlParamType(), multiple=True, metavar="KEY=VAL",
+    "-s", "--set", "set_", type=YamlParamType(), multiple=True, metavar="KEY=VAL",
    help="Set a configuration value (can be used multiple times)"
 )
--- a/tutor/templates/config-defaults.yml
+++ b/tutor/templates/config-defaults.yml
@ -1,4 +1,9 @@
 ---
+DOCKER_IMAGE_OPENEDX: "regis/openedx:hawthorn"
+DOCKER_IMAGE_ANDROID: "regis/openedx-android:hawthorn"
+DOCKER_IMAGE_FORUM: "regis/openedx-forum:hawthorn"
+DOCKER_IMAGE_NOTES: "regis/openedx-notes:hawthorn"
+DOCKER_IMAGE_XQUEUE: "regis/openedx-xqueue:hawthorn"
 MYSQL_DATABASE: "openedx"
 MYSQL_USERNAME: "openedx"
 MONGODB_DATABASE: "openedx"
@ -7,7 +12,6 @@ NOTES_MYSQL_USERNAME: "notes"
 XQUEUE_AUTH_USERNAME: "lms"
 XQUEUE_MYSQL_DATABASE: "xqueue"
 XQUEUE_MYSQL_USERNAME: "xqueue"
-OPENEDX_DOCKER_IMAGE: "regis/openedx:hawthorn"
 NGINX_HTTP_PORT: 80
 NGINX_HTTPS_PORT: 443
 ANDROID_RELEASE_STORE_PASSWORD: "android store password"
--- a/tutor/templates/k8s/deployments.yml
+++ b/tutor/templates/k8s/deployments.yml
@ -15,7 +15,7 @@ spec:
    spec:
      containers:
        - name: cms
-          image: regis/openedx:hawthorn
+          image: {{ DOCKER_IMAGE_OPENEDX }}
          env:
          - name: SERVICE_VARIANT
            value: cms
@ -62,7 +62,7 @@ spec:
    spec:
      containers:
        - name: forum
-          image: regis/openedx-forum:hawthorn
+          image: {{ DOCKER_IMAGE_FORUM }}
          ports:
            - containerPort: 4567
          imagePullPolicy: Always
@ -84,7 +84,7 @@ spec:
    spec:
      containers:
        - name: lms
-          image: regis/openedx:hawthorn
+          image: {{ DOCKER_IMAGE_OPENEDX }}
          ports:
            - containerPort: 8000
          volumeMounts:
@ -246,14 +246,14 @@ spec:
    spec:
      initContainers:
        - name: clean-openedx-staticfiles
-          image: regis/openedx:hawthorn
+          image: {{ DOCKER_IMAGE_OPENEDX }}
          command: ['rm', '-rf', '/var/www/openedx/staticfiles']
          volumeMounts:
            - mountPath: /var/www/openedx/
              name: openedx-staticfiles
          imagePullPolicy: Always
        - name: init-openedx-staticfiles
-          image: regis/openedx:hawthorn
+          image: {{ DOCKER_IMAGE_OPENEDX }}
          command: ['cp', '-r', '/openedx/staticfiles', '/var/www/openedx/']
          volumeMounts:
            - mountPath: /var/www/openedx/
--- a/tutor/templates/local/docker-compose.yml
+++ b/tutor/templates/local/docker-compose.yml
@ -38,7 +38,7 @@ services:
      - ../../data/elasticsearch:/usr/share/elasticsearch/data

  openedx-assets:
-    image: {{ OPENEDX_DOCKER_IMAGE }}
+    image: {{ DOCKER_IMAGE_OPENEDX }}
    volumes:
      - ../../data/openedx:/var/www/openedx
    command: bash -c "rm -rf /var/www/openedx/staticfiles && cp -r /openedx/staticfiles/ /var/www/openedx/"
@ -74,7 +74,7 @@ services:
  ############# Forum

  forum:
-    image: regis/openedx-forum:hawthorn
+    image: {{ DOCKER_IMAGE_FORUM }}
    restart: unless-stopped
    depends_on:
      - elasticsearch
@ -83,7 +83,7 @@ services:
  ############# LMS and CMS

  lms:
-    image: {{ OPENEDX_DOCKER_IMAGE }}
+    image: {{ DOCKER_IMAGE_OPENEDX }}
    environment:
      SERVICE_VARIANT: lms
      SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
@ -103,7 +103,7 @@ services:
      - smtp

  cms:
-    image: {{ OPENEDX_DOCKER_IMAGE }}
+    image: {{ DOCKER_IMAGE_OPENEDX }}
    environment:
      SERVICE_VARIANT: cms
      SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
@ -125,7 +125,7 @@ services:

  # We could probably create one service per queue here. For small instances, it is not necessary.
  lms_worker:
-    image: {{ OPENEDX_DOCKER_IMAGE }}
+    image: {{ DOCKER_IMAGE_OPENEDX }}
    environment:
      SERVICE_VARIANT: lms
      SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
@ -141,7 +141,7 @@ services:
      - lms

  cms_worker:
-    image: {{ OPENEDX_DOCKER_IMAGE }}
+    image: {{ DOCKER_IMAGE_OPENEDX }}
    environment:
      SERVICE_VARIANT: cms
      SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
@ -159,7 +159,7 @@ services:
  {% if ACTIVATE_NOTES %}
  ############# Notes: backend store for edX Student Notes
  notes:
-    image: regis/openedx-notes:hawthorn
+    image: {{ DOCKER_IMAGE_NOTES }}
    networks:
      default:
        aliases:
@ -177,7 +177,7 @@ services:
  {% if ACTIVATE_XQUEUE %}
  ############# Xqueue: external grading of Open edX problems
  xqueue:
-    image: regis/openedx-xqueue:hawthorn
+    image: {{ DOCKER_IMAGE_XQUEUE }}
    volumes:
      - ../apps/xqueue/settings/tutor.py:/openedx/xqueue/xqueue/tutor.py
      - ../../data/xqueue:/openedx/data
@ -188,7 +188,7 @@ services:
      - mysql

  xqueue_consumer:
-    image: regis/openedx-xqueue:hawthorn
+    image: {{ DOCKER_IMAGE_XQUEUE }}
    volumes:
      - ../apps/xqueue/settings/tutor.py:/openedx/xqueue/xqueue/tutor.py
      - ../../data/xqueue:/openedx/data