diff --git a/docs/configuration.rst b/docs/configuration.rst index bfdd533..8ba5d1d 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -25,21 +25,26 @@ Once the base configuration is created or updated, the environment is automatica With an up-to-date environment, Tutor is ready to launch an Open edX platform and perform usual operations. Below, we document some of the configuration parameters. +Docker +------ + .. _docker_images: -``DOCKER_IMAGE_*`` Custom Docker images ---------------------------------------- +Custom images +------------- + +- ``DOCKER_IMAGE_OPENEDX`` (default: ``"regis/openedx:hawthorn"``) +- ``DOCKER_IMAGE_ANDROID`` (default: ``"regis/openedx-android:hawthorn"``) +- ``DOCKER_IMAGE_FORUM`` (default: ``"regis/openedx-forum:hawthorn"``) +- ``DOCKER_IMAGE_NOTES`` (default: ``"regis/openedx-notes:hawthorn"``) +- ``DOCKER_IMAGE_XQUEUE`` (default: ``"regis/openedx-xqueue:hawthorn"``) These configuration parameters define which image to run for each service. -- ``DOCKER_IMAGE_OPENEDX`` (default: ``regis/openedx:hawthorn``) -- ``DOCKER_IMAGE_ANDROID`` (default: ``regis/openedx-android:hawthorn``) -- ``DOCKER_IMAGE_FORUM`` (default: ``regis/openedx-forum:hawthorn``) -- ``DOCKER_IMAGE_NOTES`` (default: ``regis/openedx-notes:hawthorn``) -- ``DOCKER_IMAGE_XQUEUE`` (default: ``regis/openedx-xqueue:hawthorn``) +Custom registry +--------------- -``DOCKER_REGISTRY`` Custom Docker registry ------------------------------------------- +- ``DOCKER_REGISTRY`` (default: ``""``) You may want to pull/push images from/to a custom docker registry. For instance, for a registry running on ``localhost:5000``, define:: @@ -47,13 +52,58 @@ You may want to pull/push images from/to a custom docker registry. For instance, (the trailing ``/`` is important) -``ACTIVATE_*`` Optional features --------------------------------- +Vendor services +--------------- -Some optional features may be activated or deactivated during the interactive configuration step. These features change configuration files (during the ``configure`` step) as well as make targets. +MySQL +~~~~~ -``ACTIVATE_HTTPS`` SSL/TLS certificates for HTTPS access -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +- ``ACTIVATE_MYSQL`` (default: ``true``) +- ``MYSQL_HOST`` (default: ``"mysql"``) +- ``MYSQL_PORT`` (default: ``3306``) +- ``MYSQL_DATABASE`` (default: ``"openedx"``) +- ``MYSQL_USERNAME`` (default: ``"openedx"``) +- ``MYSQL_PASSWORD`` (default: randomly generated): root user password. Note that you are responsible for creating the root user if you are using a managed database. + +By default, a running Open edX platform deployed with Tutor includes all necessary 3rd-party services, such as MySQL, MongoDb, etc. But it's also possible to store data on a separate database, such as `Amazon RDS `_. For instance, to store data on an external MySQL database, set the following configuration:: + + ACTIVATE_MYSQL: false + MYSQL_HOST: yourhost + MYSQL_PASSWORD: + +Elasticsearch +~~~~~~~~~~~~~ + +- ``ELASTICSEARCH_HOST`` (default: ``"elasticsearch"``) +- ``ELASTICSEARCH_PORT`` (default: ``9200``) + +Memcached +~~~~~~~~~ + +- ``MEMCACHED_HOST`` (default: ``"memcached"``) +- ``MEMCACHED_PORT`` (default: ``11211``) + +Mongodb +~~~~~~~ + +- ``ACTIVATE_MONGODB`` (default: ``true``) +- ``MONGODB_HOST`` (default: ``"mongodb"``) +- ``MONGODB_DATABASE`` (default: ``"openedx"``) + +Rabbitmq +~~~~~~~~ + +- ``RABBITMQ_HOST`` (default: ``"rabbitmq"``) + +Optional features +----------------- + +Some optional features may be activated or deactivated during the interactive configuration step (``tutor config save``). + +SSL/TLS certificates for HTTPS access +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- ``ACTIVATE_HTTPS`` (default: ``false``) By activating this feature, a free SSL/TLS certificate from the `Let's Encrypt `_ certificate authority will be created for your platform. With this feature, **your platform will no longer be accessible in HTTP**. Calls to http urls will be redirected to https url. @@ -73,8 +123,10 @@ To renew the certificate, run this command once per month:: tutor local https renew -``ACTIVATE_NOTES`` Student notes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Student notes +~~~~~~~~~~~~~ + +- ``ACTIVATE_NOTES`` (default: ``false``) With `notes `_, students can annotate portions of the courseware. @@ -83,7 +135,9 @@ With `notes `` domain name should be activated and point to your server. For instance, if your LMS is hosted at http://myopenedx.com, the notes service should be found at http://notes.myopenedx.com. -``ACTIVATE_XQUEUE`` Xqueue -~~~~~~~~~~~~~~~~~~~~~~~~~~ +Xqueue +~~~~~~ + +- ``ACTIVATE_XQUEUE`` (default: ``false``) `Xqueue `_ is for grading problems with external services. If you don't know what it is, you probably don't need it. diff --git a/tutor/images.py b/tutor/images.py index 1d60026..8ea6bc0 100644 --- a/tutor/images.py +++ b/tutor/images.py @@ -85,7 +85,19 @@ def openedx_image_names(config, image): return [image] def vendor_image_names(config, image): - return VENDOR_IMAGES if image == "all" else [image] + if image == "all": + images = VENDOR_IMAGES[:] + if not config['ACTIVATE_ELASTICSEARCH']: + images.remove('elasticsearch') + if not config['ACTIVATE_MEMCACHED']: + images.remove('memcached') + if not config['ACTIVATE_MONGODB']: + images.remove('mongodb') + if not config['ACTIVATE_MYSQL']: + images.remove('mysql') + if not config['ACTIVATE_RABBITMQ']: + images.remove('rabbitmq') + return [image] images.add_command(build) images.add_command(pull) diff --git a/tutor/k8s.py b/tutor/k8s.py index 6cb56bf..e774e56 100644 --- a/tutor/k8s.py +++ b/tutor/k8s.py @@ -21,7 +21,6 @@ def k8s(): def quickstart(root): click.echo(fmt.title("Interactive platform configuration")) tutor_config.save.callback(root, False, []) - click.echo(fmt.title("Environment generation")) click.echo(fmt.title("Stopping any existing platform")) stop.callback() click.echo(fmt.title("Starting the platform")) @@ -32,10 +31,12 @@ def quickstart(root): @click.command(help="Run all configured Open edX services") @opts.root def start(root): + config = tutor_config.load(root) kubectl_no_fail("create", "-f", tutor_env.pathjoin(root, "k8s", "namespace.yml")) kubectl("create", "configmap", "nginx-config", "--from-file", tutor_env.pathjoin(root, "apps", "nginx")) - kubectl("create", "configmap", "mysql-config", "--from-env-file", tutor_env.pathjoin(root, "apps", "mysql", "auth.env")) + if config['ACTIVATE_MYSQL']: + kubectl("create", "configmap", "mysql-config", "--from-env-file", tutor_env.pathjoin(root, "apps", "mysql", "auth.env")) kubectl("create", "configmap", "openedx-settings-lms", "--from-file", tutor_env.pathjoin(root, "apps", "openedx", "settings", "lms")) kubectl("create", "configmap", "openedx-settings-cms", "--from-file", tutor_env.pathjoin(root, "apps", "openedx", "settings", "cms")) kubectl("create", "configmap", "openedx-config", "--from-file", tutor_env.pathjoin(root, "apps", "openedx", "config")) diff --git a/tutor/local.py b/tutor/local.py index f7df5a3..65c400e 100644 --- a/tutor/local.py +++ b/tutor/local.py @@ -124,6 +124,9 @@ def databases(root): ops.migrate(root, run_bash) def init_mysql(root): + config = tutor_config.load(root) + if not config['ACTIVATE_MYSQL']: + return mysql_data_path = tutor_env.data_path(root, "mysql", "mysql") if os.path.exists(mysql_data_path): return diff --git a/tutor/templates/config-defaults.yml b/tutor/templates/config-defaults.yml index 3417061..ca8179a 100644 --- a/tutor/templates/config-defaults.yml +++ b/tutor/templates/config-defaults.yml @@ -1,4 +1,9 @@ --- +ACTIVATE_ELASTICSEARCH: true +ACTIVATE_MEMCACHED: true +ACTIVATE_MONGODB: true +ACTIVATE_MYSQL: true +ACTIVATE_RABBITMQ: true ANDROID_RELEASE_STORE_PASSWORD: "android store password" ANDROID_RELEASE_KEY_PASSWORD: "android release key password" ANDROID_RELEASE_KEY_ALIAS: "android release key alias" diff --git a/tutor/templates/k8s/deployments.yml b/tutor/templates/k8s/deployments.yml index d12f0b1..2c7ed8b 100644 --- a/tutor/templates/k8s/deployments.yml +++ b/tutor/templates/k8s/deployments.yml @@ -116,6 +116,7 @@ spec: persistentVolumeClaim: claimName: lms-data +{% if ACTIVATE_ELASTICSEARCH %} --- apiVersion: apps/v1 kind: Deployment @@ -150,7 +151,9 @@ spec: - name: data persistentVolumeClaim: claimName: elasticsearch +{% endif %} +{% if ACTIVATE_MEMCACHED %} --- apiVersion: apps/v1 kind: Deployment @@ -171,7 +174,9 @@ spec: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_MEMCACHED }} ports: - containerPort: 11211 +{% endif %} +{% if ACTIVATE_MONGODB %} --- apiVersion: apps/v1 kind: Deployment @@ -199,7 +204,9 @@ spec: volumes: - name: data emptyDir: {} +{% endif %} +{% if ACTIVATE_MYSQL %} --- apiVersion: apps/v1 kind: Deployment @@ -233,6 +240,7 @@ spec: - name: data persistentVolumeClaim: claimName: mysql +{% endif %} --- apiVersion: apps/v1 @@ -290,6 +298,7 @@ spec: persistentVolumeClaim: claimName: lms-data +{% if ACTIVATE_RABBITMQ %} --- apiVersion: apps/v1 kind: Deployment @@ -317,4 +326,4 @@ spec: - name: data persistentVolumeClaim: claimName: rabbitmq - +{% endif %} diff --git a/tutor/templates/k8s/services.yml b/tutor/templates/k8s/services.yml index fc5ce4f..c553076 100644 --- a/tutor/templates/k8s/services.yml +++ b/tutor/templates/k8s/services.yml @@ -37,6 +37,7 @@ spec: selector: app: lms +{% if ACTIVATE_ELASTICSEARCH %} --- apiVersion: v1 kind: Service @@ -49,7 +50,9 @@ spec: protocol: TCP selector: app: elasticsearch +{% endif %} +{% if ACTIVATE_MEMCACHED %} --- apiVersion: v1 kind: Service @@ -62,7 +65,9 @@ spec: protocol: TCP selector: app: memcached +{% endif %} +{% if ACTIVATE_MONGODB %} --- apiVersion: v1 kind: Service @@ -75,7 +80,9 @@ spec: protocol: TCP selector: app: mongodb +{% endif %} +{% if ACTIVATE_MYSQL %} --- apiVersion: v1 kind: Service @@ -88,6 +95,7 @@ spec: protocol: TCP selector: app: mysql +{% endif %} --- apiVersion: v1 @@ -108,6 +116,7 @@ spec: selector: app: nginx +{% if ACTIVATE_RABBITMQ %} --- apiVersion: v1 kind: Service @@ -120,4 +129,4 @@ spec: protocol: TCP selector: app: rabbitmq - +{% endif %} diff --git a/tutor/templates/k8s/volumes.yml b/tutor/templates/k8s/volumes.yml index 332f31b..e009e8c 100644 --- a/tutor/templates/k8s/volumes.yml +++ b/tutor/templates/k8s/volumes.yml @@ -22,6 +22,7 @@ spec: requests: storage: 2Gi +{% if ACTIVATE_ELASTICSEARCH %} --- apiVersion: v1 kind: PersistentVolumeClaim @@ -33,7 +34,9 @@ spec: resources: requests: storage: 2Gi +{% endif %} +{% if ACTIVATE_MYSQL %} --- apiVersion: v1 kind: PersistentVolumeClaim @@ -45,6 +48,7 @@ spec: resources: requests: storage: 5Gi +{% endif %} --- apiVersion: v1 @@ -58,6 +62,7 @@ spec: requests: storage: 1Gi +{% if ACTIVATE_RABBITMQ %} --- apiVersion: v1 kind: PersistentVolumeClaim @@ -69,3 +74,4 @@ spec: resources: requests: storage: 1Gi +{% endif %} diff --git a/tutor/templates/local/docker-compose.yml b/tutor/templates/local/docker-compose.yml index 8f12f6b..5c3a7be 100644 --- a/tutor/templates/local/docker-compose.yml +++ b/tutor/templates/local/docker-compose.yml @@ -3,10 +3,13 @@ services: ############# External services + {% if ACTIVATE_MEMCACHED %} memcached: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_MEMCACHED }} restart: unless-stopped + {% endif %} + {% if ACTIVATE_MONGODB %} mongodb: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_MONGODB }} # Use WiredTiger in all environments, just like at edx.org @@ -14,7 +17,9 @@ services: restart: unless-stopped volumes: - ../../data/mongodb:/data/db + {% endif %} + {% if ACTIVATE_MYSQL %} mysql: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_MYSQL }} command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci @@ -22,7 +27,9 @@ services: volumes: - ../../data/mysql:/var/lib/mysql env_file: ../apps/mysql/auth.env + {% endif %} + {% if ACTIVATE_ELASTICSEARCH %} elasticsearch: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_ELASTICSEARCH }} environment: @@ -36,6 +43,7 @@ services: restart: unless-stopped volumes: - ../../data/elasticsearch:/usr/share/elasticsearch/data + {% endif %} openedx-assets: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_OPENEDX }} @@ -60,11 +68,13 @@ services: - cms {% if ACTIVATE_NOTES %}- notes{% endif %} + {% if ACTIVATE_RABBITMQ %} rabbitmq: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_RABBITMQ }} volumes: - ../../data/rabbitmq:/var/lib/rabbitmq restart: unless-stopped + {% endif %} # Simple SMTP server smtp: @@ -80,8 +90,8 @@ services: MONGOHQ_URL: "mongodb://{{ MONGODB_HOST }}/cs_comments_service" restart: unless-stopped depends_on: - - elasticsearch - - mongodb + {% if ACTIVATE_ELASTICSEARCH %}- elasticsearch{% endif %} + {% if ACTIVATE_MONGODB %}- mongodb{% endif %} ############# LMS and CMS @@ -97,12 +107,12 @@ services: - ../apps/openedx/config/:/openedx/config/ - ../../data/lms:/openedx/data depends_on: - - elasticsearch + {% if ACTIVATE_ELASTICSEARCH %}- elasticsearch{% endif %} - forum - - memcached - - mongodb - - mysql - - rabbitmq + {% if ACTIVATE_MEMCACHED %}- memcached{% endif %} + {% if ACTIVATE_MONGODB %}- mongodb{% endif %} + {% if ACTIVATE_MYSQL %}- mysql{% endif %} + {% if ACTIVATE_RABBITMQ %}- rabbitmq{% endif %} - smtp cms: @@ -117,11 +127,11 @@ services: - ../apps/openedx/config/:/openedx/config/ - ../../data/cms:/openedx/data depends_on: - - elasticsearch - - memcached - - mongodb - - mysql - - rabbitmq + {% if ACTIVATE_ELASTICSEARCH %}- elasticsearch{% endif %} + {% if ACTIVATE_MEMCACHED %}- memcached{% endif %} + {% if ACTIVATE_MONGODB %}- mongodb{% endif %} + {% if ACTIVATE_MYSQL %}- mysql{% endif %} + {% if ACTIVATE_RABBITMQ %}- rabbitmq{% endif %} - smtp ############# LMS and CMS workers @@ -174,7 +184,7 @@ services: - ../../data/notes:/openedx/data restart: unless-stopped depends_on: - - mysql + {% if ACTIVATE_MYSQL %}- mysql{% endif %} {% endif %} {% if ACTIVATE_XQUEUE %} @@ -188,7 +198,7 @@ services: DJANGO_SETTINGS_MODULE: xqueue.tutor restart: unless-stopped depends_on: - - mysql + {% if ACTIVATE_MYSQL %}- mysql{% endif %} xqueue_consumer: image: {{ DOCKER_REGISTRY }}{{ DOCKER_IMAGE_XQUEUE }} @@ -200,5 +210,5 @@ services: restart: unless-stopped command: ./manage.py run_consumer depends_on: - - mysql + {% if ACTIVATE_MYSQL %}- mysql{% endif %} {% endif %}