v11.0.0 (2020-12-09)

- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
    - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
    - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
    - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
    - The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
    - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
	- Nginx no longer listens to port 443 for https traffic
	- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
	- Configuration files for web proxies are no longer provided.
	- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
    - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
    - Replace memcached and rabbitmq by redis.
- Additional features:
    - Make it possible to disable all plugins at once with ``plugins disable all``.
    - Add ``tutor k8s wait`` command to wait for a pod to become ready
    - Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
    - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
    - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
This commit is contained in:
Régis Behmo 2020-09-17 12:53:14 +02:00
parent d3c842c8da
commit 728ef966dc
61 changed files with 659 additions and 755 deletions

View File

@ -4,6 +4,31 @@ Note: Breaking changes between versions are indicated by "💥".
## Unreleased
## v11.0.0 (2020-12-09)
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
- The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
- Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
- The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
- The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
- Use Caddy as a web proxy for automated SSL/TLS certificate generation:
- Nginx no longer listens to port 443 for https traffic
- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
- Configuration files for web proxies are no longer provided.
- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
- Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
- Replace memcached and rabbitmq by redis.
- Additional features:
- Make it possible to disable all plugins at once with ``plugins disable all``.
- Add ``tutor k8s wait`` command to wait for a pod to become ready
- Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
- "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
- "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
## v10.5.3 (2020-12-09)
- [Security] Apply upstream edx-platform [security patch](https://github.com/edx/edx-platform/pull/25782)

View File

@ -89,10 +89,10 @@ ci-test-bundle: ## Run basic tests on bundle
yes "" | ./dist/tutor config save --interactive
./dist/tutor config save
./dist/tutor plugins list
# ./dist/tutor plugins enable discovery ecommerce figures lts minio notes xqueue
./dist/tutor plugins enable discovery ecommerce lts minio notes xqueue
# ./dist/tutor plugins enable discovery ecommerce figures license minio notes xqueue
./dist/tutor plugins enable discovery ecommerce license minio notes xqueue
./dist/tutor plugins list
./dist/tutor lts --help
./dist/tutor license --help
./releases/github-release: ## Download github-release binary
mkdir -p releases/

View File

@ -34,7 +34,7 @@ Tutor: the docker-based Open edX distribution designed for peace of mind
**Tutor** is a docker-based `Open edX <https://openedx.org>`_ distribution, both for production and local development. The goal of Tutor is to make it easy to deploy, customize, upgrade and scale Open edX. Tutor is reliable, fast, extensible, and it is already used by dozens of Open edX platforms around the world.
Do you need professional assistance setting up or managing your Open edX platform? Overhang.IO provides online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/lts>`__.
Do you need professional assistance setting up or managing your Open edX platform? Overhang.IO provides online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/pricing>`__.
Features
--------
@ -46,7 +46,7 @@ Features
* Extensible architecture with `plugins <https://docs.tutor.overhang.io/plugins.html>`__
* Works with `Kubernetes <https://docs.tutor.overhang.io/k8s.html>`__
* No technical skill required with the `1-click Tutor AWS image <https://docs.tutor.overhang.io/install.html#cloud-deployment>`__
* Professional support and premium plugins available with `Tutor Long Term Support (LTS) <https://overhang.io/tutor/lts>`__
* Amazing plugins available with `Tutor Wizard Edition <https://overhang.io/tutor>`__
.. _readme_intro_end:
@ -71,7 +71,7 @@ Extensive documentation is available online: https://docs.tutor.overhang.io/
Support
-------
To get community support, go to the official discussion forums: https://discuss.overhang.io. For official support, please subscribe to a Long Term Support (LTS) license at https://overhang.io/tutor/lts.
To get community support, go to the official discussion forums: https://discuss.overhang.io. For official support, please subscribe to a Long Term Support (LTS) license at https://overhang.io/tutor/pricing.
.. _readme_support_end:

View File

@ -6,7 +6,7 @@ for plugin_name in [
"discovery",
"ecommerce",
# "figures",
"lts",
"license",
"minio",
"notes",
"xqueue",

View File

@ -40,16 +40,15 @@ With an up-to-date environment, Tutor is ready to launch an Open edX platform an
Individual service activation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- ``ACTIVATE_LMS`` (default: ``true``)
- ``ACTIVATE_CMS`` (default: ``true``)
- ``ACTIVATE_FORUM`` (default: ``true``)
- ``ACTIVATE_ELASTICSEARCH`` (default: ``true``)
- ``ACTIVATE_MEMCACHED`` (default: ``true``)
- ``ACTIVATE_MONGODB`` (default: ``true``)
- ``ACTIVATE_MYSQL`` (default: ``true``)
- ``ACTIVATE_RABBITMQ`` (default: ``true``)
- ``ACTIVATE_SMTP`` (default: ``true``)
- ``ACTIVATE_HTTPS`` (default: ``false``)
- ``RUN_LMS`` (default: ``true``)
- ``RUN_CMS`` (default: ``true``)
- ``RUN_FORUM`` (default: ``true``)
- ``RUN_ELASTICSEARCH`` (default: ``true``)
- ``RUN_MONGODB`` (default: ``true``)
- ``RUN_MYSQL`` (default: ``true``)
- ``RUN_REDIS`` (default: ``true``)
- ``RUN_SMTP`` (default: ``true``)
- ``ENABLE_HTTPS`` (default: ``false``)
Every single Open edX service may be (de)activated at will by these configuration parameters. This is useful if you want, for instance, to distribute the various Open edX services on different servers.
@ -81,32 +80,37 @@ You may want to pull/push images from/to a custom docker registry. For instance,
Open edX customisation
~~~~~~~~~~~~~~~~~~~~~~
- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/juniper.3"``)
- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/koa.1"``)
This defines the default version that will be pulled from all Open edX git repositories.
- ``OPENEDX_CMS_GUNICORN_WORKERS`` (default: ``2``)
- ``OPENEDX_LMS_GUNICORN_WORKERS`` (default: ``2``)
- ``OPENEDX_CMS_UWSGI_WORKERS`` (default: ``2``)
- ``OPENEDX_LMS_UWSGI_WORKERS`` (default: ``2``)
By default there are 2 `gunicorn worker processes <https://docs.gunicorn.org/en/stable/settings.html#worker-processes>`__ to serve requests for the LMS and the CMS. However, each workers requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.
By default there are 2 `uwsgi worker processes <https://uwsgi-docs.readthedocs.io/en/latest/Options.html#processes>`__ to serve requests for the LMS and the CMS. However, each workers requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.
Vendor services
~~~~~~~~~~~~~~~
Caddy
*****
- ``RUN_CADDY`` (default: ``true``)
`Caddy <https://caddyserver.com>`__ is a web server used in Tutor as a web proxy for the generation of SSL/TLS certificates at runtime. If ``RUN_CADDY`` is set to ``false`` then we assume that SSL termination does not occur in the Caddy container, and thus the ``caddy`` container is not started.
Nginx
*****
- ``NGINX_HTTP_PORT`` (default: ``80``)
- ``NGINX_HTTPS_PORT`` (default: ``443``)
- ``WEB_PROXY`` (default: ``false``)
Nginx is used to route web traffic to the various applications and to serve static assets. In case there is another web server in front of the Nginx container (for instance, a web server running on the host or an Ingress controller on Kubernetes), the container exposed ports can be modified. If ``WEB_PROXY`` is set to ``true`` then we assume that SSL termination does not occur in the Nginx container.
Nginx is used to route web traffic to the various applications and to serve static assets. When ``RUN_CADDY`` is false, the ``NGINX_HTTP_PORT`` is exposed on the host.
MySQL
*****
- ``ACTIVATE_MYSQL`` (default: ``true``)
- ``RUN_MYSQL`` (default: ``true``)
- ``MYSQL_HOST`` (default: ``"mysql"``)
- ``MYSQL_PORT`` (default: ``3306``)
- ``MYSQL_ROOT_USERNAME`` (default: ``"root"``)
@ -114,7 +118,7 @@ MySQL
By default, a running Open edX platform deployed with Tutor includes all necessary 3rd-party services, such as MySQL, MongoDb, etc. But it's also possible to store data on a separate database, such as `Amazon RDS <https://aws.amazon.com/rds/>`_. For instance, to store data on an external MySQL database, set the following configuration::
ACTIVATE_MYSQL: false
RUN_MYSQL: false
MYSQL_HOST: yourhost
MYSQL_ROOT_USERNAME: <root user name>
MYSQL_ROOT_PASSWORD: <root user password>
@ -127,34 +131,31 @@ Elasticsearch
- ``ELASTICSEARCH_PORT`` (default: ``9200``)
- ``ELASTICSEARCH_HEAP_SIZE`` (default: ``"1g"``)
Memcached
*********
- ``MEMCACHED_HOST`` (default: ``"memcached"``)
- ``MEMCACHED_PORT`` (default: ``11211``)
Mongodb
*******
- ``ACTIVATE_MONGODB`` (default: ``true``)
- ``RUN_MONGODB`` (default: ``true``)
- ``MONGODB_HOST`` (default: ``"mongodb"``)
- ``MONGODB_DATABASE`` (default: ``"openedx"``)
- ``MONGODB_PORT`` (default: ``27017``)
- ``MONGODB_USERNAME`` (default: ``""``)
- ``MONGODB_PASSWORD`` (default: ``""``)
Rabbitmq
********
Redis
*****
- ``ACTIVATE_RABBITMQ`` (default: ``true``)
- ``RABBITMQ_HOST`` (default: ``"rabbitmq"``)
- ``RABBITMQ_USERNAME`` (default: ``""``)
- ``RABBITMQ_PASSWORD`` (default: ``""``)
- ``RUN_REDIS`` (default: ``true``)
- ``REDIS_HOST`` (default: ``"redis"``)
- ``REDIS_HOST`` (default: ``6379``)
- ``REDIS_USERNAME`` (default: ``""``)
- ``REDIS_PASSWORD`` (default: ``""``)
Note that Redis has replaced Rabbitmq as the Celery message broker since Tutor v11.0.0.
SMTP
****
- ``ACTIVATE_SMTP`` (default: ``true``)
- ``RUN_SMTP`` (default: ``true``)
- ``SMTP_HOST`` (default: ``"smtp"``)
- ``SMTP_PORT`` (default: ``25``)
- ``SMTP_USERNAME`` (default: ``""``)
@ -167,7 +168,7 @@ Note that the SMTP server shipped with Tutor by default does not implement TLS.
SSL/TLS certificates for HTTPS access
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- ``ACTIVATE_HTTPS`` (default: ``false``)
- ``ENABLE_HTTPS`` (default: ``false``)
By activating this feature, a free SSL/TLS certificate from the `Let's Encrypt <https://letsencrypt.org/>`_ certificate authority will be created for your platform. With this feature, **your platform will no longer be accessible in HTTP**. Calls to http urls will be redirected to https url.
@ -179,15 +180,7 @@ The following DNS records must exist and point to your server::
Thus, **this feature will (probably) not work in development** because the DNS records will (probably) not point to your development machine.
To create the certificate manually, run::
tutor local https create
To renew the certificate, run this command once per month::
tutor local stop nginx
tutor local https renew
tutor local start -d
The SSL/TLS certificates will automatically be generated and updated by the Caddy proxy server container at runtime. Thus, as of v11.0.0 you no longer have to generate the certificates manually.
.. _customise:
@ -214,7 +207,7 @@ openedx Docker Image build arguments
When building the "openedx" Docker image, it is possible to specify a few `arguments <https://docs.docker.com/engine/reference/builder/#arg>`__:
- ``EDX_PLATFORM_REPOSITORY`` (default: ``"https://github.com/edx/edx-platform.git"``)
- ``EDX_PLATFORM_VERSION`` (default: ``"open-release/juniper.3"``)
- ``EDX_PLATFORM_VERSION`` (default: ``"open-release/koa.1"``)
- ``EDX_PLATFORM_VERSION_DATE`` (default: ``"20200227"``)
- ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``)
@ -286,16 +279,16 @@ Note that your edx-platform version must be a fork of the latest release **tag**
If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular:
- Do not try to run a fork from an older (pre-Juniper) version of edx-platform: this will simply not work.
- Do not try to run a fork from an older (pre-Koa) version of edx-platform: this will simply not work.
- Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail.
- Do not try to run a fork from the open-release/juniper.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/juniper.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/juniper.master branch.
- Do not try to run a fork from the open-release/koa.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/koa.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/koa.master branch.
.. _i18n:
Adding custom translations
~~~~~~~~~~~~~~~~~~~~~~~~~~
If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/juniper.3/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/koa.1/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance::
@ -308,9 +301,9 @@ Then, add a "django.po" file there that will contain your custom translations::
msgid "String to translate"
msgstr "你翻译的东西 la traduction de votre bidule"
The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/juniper.3/conf/locale/en/LC_MESSAGES/django.po>`__.
The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/koa.1/conf/locale/en/LC_MESSAGES/django.po>`__.
If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/juniper.3/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/koa.1/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy::

View File

@ -25,7 +25,7 @@ This ``openedx-dev`` development image differs from the ``openedx`` production i
- The user that runs inside the container has the same UID as the user on the host, in order to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository).
- Additional python and system requirements are installed for convenient debugging: `ipython <https://ipython.org/>`__, `ipdb <https://pypi.org/project/ipdb/>`__, vim, telnet.
- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/juniper.3/requirements/edx/development.in>`__ are installed.
- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/koa.1/requirements/edx/development.in>`__ are installed.
Since the ``openedx-dev`` is based upon the ``openedx`` docker image, it should be re-built every time the ``openedx`` docker image is modified.
@ -68,11 +68,11 @@ If you don't want to rewrite this option every time, you can define a command al
alias tutor-dev-run-lms="tutor dev run -v /path/to/edx-platform:/openedx/edx-platform lms"
For technical reasons, the ``-v`` option is only supported for the ``run`` and ``runserver`` commands. With these commands, only one service is started. But there are cases where you may want to launch and debug a complete Open edX platform with ``tutor dev start`` and mount a custom edx-platform fork. For instance, this might be needed when testing the interaction between multiple services. To do so, you should create a ``docker-compose.override.yml`` file that will specify a custom volume to be used with all ``dev`` commands::
vim "$(tutor config printroot)/env/dev/docker-compose.override.yml"
Then, add the following content::
version: "3.7"
services:
lms:
@ -90,7 +90,7 @@ Then, add the following content::
This override file will be loaded when running any ``tutor dev ..`` command. The edx-platform repo mounted at the specified path will be automaticall mounted inside all LMS and CMS containers. With this file, you should no longer specify the ``-v`` option from the command line with the ``run`` or ``runserver`` commands.
**Note:** containers are built on the Juniper release. If you are working on a different version of Open edX, you will have to rebuild the ``openedx`` docker images with the version. See the :ref:`fork edx-platform section <edx_platform_fork>`.
**Note:** containers are built on the Koa release. If you are working on a different version of Open edX, you will have to rebuild the ``openedx`` docker images with the version. See the :ref:`fork edx-platform section <edx_platform_fork>`.
Prepare the edx-platform repo
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -63,7 +63,7 @@ Installing from source
pip install -e .
.. _cloud_install:
Zero-click AWS installation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -78,9 +78,9 @@ Upgrading
With Tutor, it is very easy to upgrade to a more recent Open edX or Tutor release. Just install the latest ``tutor`` version (using either methods above) and run the ``quickstart`` command again. If you have :ref:`customised <configuration_customisation>` your docker images, you will have to re-build them prior to running ``quickstart``.
``quickstart`` should take care of automatically running the upgrade process. If for some reason you need to *manually* upgrade from an Open edX release to the next, you should run ``tutor local upgrade``. For instance, to upgrade from Ironwood to Juniper, run::
``quickstart`` should take care of automatically running the upgrade process. If for some reason you need to *manually* upgrade from an Open edX release to the next, you should run ``tutor local upgrade``. For instance, to upgrade from Juniper to Koa, run::
tutor local upgrade --from=ironwood
tutor local upgrade --from=juniper
.. _autocomplete:

View File

@ -27,57 +27,10 @@ The Kubernetes cluster should have at least 4Gb of RAM on each node. When runnin
.. image:: img/virtualbox-minikube-system.png
:alt: Virtualbox memory settings for Minikube
Ingress controller
~~~~~~~~~~~~~~~~~~
Ingress controller and SSL/TLS certificates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to access your platform, you will have to setup an Ingress controller. Instructions vary for each cloud provider. To deploy an Nginx Ingress controller, it might be as simple as running::
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/cloud-generic.yaml
See the `official instructions <https://kubernetes.github.io/ingress-nginx/deploy/>`_ for more details.
.. warning::
By default, Tutor does *not* launch an Ingress resource or TLS/SSL certificate issuer for you. There are many different ways to create an Ingress resource and issue certificates in a Kubernetes cluster, and it's not the responsibility of Tutor to make this decision. However, Tutor comes with a ready-to-run configuration for an Nginx-based Ingress ressource and a `cert-manager <https://cert-manager.io/docs/>`__ Issuer that delivers `Let's Encrypt <https://letsencrypt.org/>`__ certificates. You may examine the configuration in ``$(tutor config printroot)/env/k8s/ingress.yml``. If you are happy with this configuration, you may apply it with::
kubectl apply -k $(tutor config printroot)/env --selector="app.kubernetes.io/component in (ingress, issuer)"
On Minikube, run::
minikube addons enable ingress
With Kubernetes, your Open edX platform will *not* be available at localhost. Instead, you will have to access your platform with the domain names you specified for the LMS and the CMS. To do so on a local computer, you will need to add the following line to /etc/hosts::
MINIKUBEIP yourdomain.com studio.yourdomain.com preview.yourdomain.com notes.yourdomain.com
where ``MINIKUBEIP`` should be replaced by the result of the command ``minikube ip``.
cert-manager for TLS certificates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tutor relies on `cert-manager <https://docs.cert-manager.io/>`_ to generate TLS certificates for HTTPS access. In order to activate HTTPS support, you will have to install cert-manager yourself. To do so, follow the `instructions from the official documentation <https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html>`_. It might be as simple as running::
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.0/cert-manager.yaml
If you decide to enable HTTPS certificates, you will also have to set ``WEB_PROXY=true`` in the platform configuration, because the SSL/TLS termination will not occur in the Nginx container, but in the Ingress controller. To do so, run::
tutor config save --set WEB_PROXY=true
Note that this configuration might conflict with a local installation.
.. warning::
On DigitalOcean, there is currently a bug that prevents certificate issuers from successfully fetching TLS certificates from Let's Encrypt. A workaround consists in adding a custom annotation to the "ingress-nginx" service::
kubectl -n ingress-nginx patch service ingress-nginx -p \
'{"metadata": {"annotations": {"service.beta.kubernetes.io/do-loadbalancer-hostname": "YOURLMSHOSTHERE"}}}'
Sources:
* https://www.digitalocean.com/community/questions/how-do-i-correct-a-connection-timed-out-error-during-http-01-challenge-propagation-with-cert-manager
* https://www.digitalocean.com/community/questions/pod-unable-to-curl-loadbalancer
* https://github.com/jetstack/cert-manager/issues/863#issuecomment-567062996
* https://github.com/digitalocean/digitalocean-cloud-controller-manager/blob/master/docs/controllers/services/examples/README.md#accessing-pods-over-a-managed-load-balancer-from-inside-the-cluster
As of Tutor v11, it is no longer required to setup an Ingress controller to access your platform. Instead Caddy exposes a LoadBalancer service and SSL/TLS certificates are transparently generated at runtime.
S3-like object storage with `MinIO <https://www.minio.io/>`_
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -148,9 +148,9 @@ To update the course search index, run::
Reloading Open edX settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~
After modifying Open edX settings, for instance when running ``tutor config save``, you will want to restart the web processes of the LMS and the CMS to take into account those new settings. It is possible to simply restart the whole platform (with ``tutor local reboot``) or just a single service (``tutor local restart lms``) but that is overkill. A quicker alternative is to send the HUP signal to the gunicorn processes running inside the containers. The "openedx" Docker image comes with a convenient script that does just that. To run it, execute::
After modifying Open edX settings, for instance when running ``tutor config save``, you will want to restart the web processes of the LMS and the CMS to take into account those new settings. It is possible to simply restart the whole platform (with ``tutor local reboot``) or just a single service (``tutor local restart lms``) but that is overkill. A quicker alternative is to send the HUP signal to the uwsgi processes running inside the containers. The "openedx" Docker image comes with a convenient script that does just that. To run it, execute::
tutor local exec lms reload-gunicorn
tutor local exec lms reload-uwsgi
.. _portainer:
@ -182,26 +182,9 @@ Running Open edX behind a web proxy
The containerized web server (nginx) needs to listen to ports 80 and 443 on the host. If there is already a webserver running on the host, such as Apache or Nginx, the nginx container will not be able to start. Tutor supports running behind a web proxy. To do so, add the following configuration::
tutor config save --set WEB_PROXY=true --set NGINX_HTTP_PORT=81 --set NGINX_HTTPS_PORT=444
tutor config save --set RUN_CADDY=false --set NGINX_HTTP_PORT=81
In this example, the nginx container ports would be mapped to 81 and 444, instead of 80 and 443. You must then configure the web proxy on the host. Basic configuration files are provided by Tutor which can be used directly by your web proxy.
For nginx::
sudo ln -s "$(tutor config printroot)/env/local/proxy/nginx/openedx.conf" /etc/nginx/sites-enabled/
sudo systemctl reload nginx
For apache::
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo ln -s "$(tutor config printroot)/env/local/proxy/apache2/openedx.conf" /etc/apache2/sites-enabled/
sudo systemctl reload apache2
If you have configured your platform to use SSL/TLS certificates for HTTPS access, the generation and renewal of certificates will not be managed by Tutor: you are supposed to take care of it yourself. Suggestions for generating and renewing these certificates with `Let's Encrypt <https://letsencrypt.org/>`_ are given by::
tutor local https create
tutor local https renew
In this example, the nginx container port would be mapped to 81 instead of 80. You must then configure the web proxy on the host. As of v11.0.0, configuration files are no longer provided for automatic configuration of your web proxy. Basically, you should setup a reverse proxy to `localhost:NGINX_HTTP_PORT` from the following hosts: LMS_HOST, preview.LMS_HOST, CMS_HOST, as well as any additional host exposed by your plugins.
Running multiple Open edX platforms on a single server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -210,7 +193,7 @@ With Tutor, it is easy to run multiple Open edX instances on a single server. To
- ``TUTOR_ROOT``: so that configuration, environment and data are not mixed up between platforms.
- ``LOCAL_PROJECT_NAME``: the various docker-compose projects cannot share the same name.
- ``NGINX_HTTP_PORT``, ``NGINX_HTTPS_PORT``: ports cannot be shared by two different containers.
- ``NGINX_HTTP_PORT``: ports cannot be shared by two different containers.
- ``LMS_HOST``, ``CMS_HOST``: the different platforms must be accessible from different domain (or subdomain) names.
In addition, a web proxy must be setup on the host, as described :ref:`above <web_proxy>`.
@ -219,13 +202,13 @@ As an example, here is how to launch two different platforms, with nginx running
# platform 1
export TUTOR_ROOT=~/openedx/site1
tutor config save --interactive --set WEB_PROXY=true --set LOCAL_PROJECT_NAME=tutor_site1 --set NGINX_HTTP_PORT=81 --set NGINX_HTTPS_PORT=481
tutor config save --interactive --set RUN_CADDY=false --set LOCAL_PROJECT_NAME=tutor_site1 --set NGINX_HTTP_PORT=81
tutor local quickstart
sudo ln -s "$(tutor config printroot)/env/local/proxy/nginx/openedx.conf" /etc/nginx/sites-enabled/site1.conf
# platform 2
export TUTOR_ROOT=~/openedx/site2
tutor config save --interactive --set WEB_PROXY=true --set LOCAL_PROJECT_NAME=tutor_site2 --set NGINX_HTTP_PORT=82 --set NGINX_HTTPS_PORT=482
tutor config save --interactive --set RUN_CADDY=false --set LOCAL_PROJECT_NAME=tutor_site2 --set NGINX_HTTP_PORT=82
tutor local quickstart
sudo ln -s "$(tutor config printroot)/env/local/proxy/nginx/openedx.conf" /etc/nginx/sites-enabled/site2.conf

View File

@ -18,7 +18,7 @@ Yes :) This is what happens when you run ``tutor local quickstart``:
2. Configuration files are generated from templates.
3. Docker images are downloaded.
4. Docker containers are provisioned.
5. A full, production-ready Open edX platform (`Juniper <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-juniper.master/platform_releases/juniper.html>`__ release) is run with docker-compose.
5. A full, production-ready Open edX platform (`Koa <https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/open-release-koa.master/platform_releases/koa.html>`__ release) is run with docker-compose.
The whole procedure should require less than 10 minutes, on a server with a good bandwidth. Note that your host environment will not be affected in any way, since everything runs inside docker containers. Root access is not even necessary.

View File

@ -17,7 +17,7 @@ What should you do if you have a problem?
6. If you have a technical background, you may try to decide if the issue is related to Open edX or if it's specific to Tutor. In the latter case, you are most welcome to open an `issue on Github <https://github.com/overhangio/tutor/issues/new>`_. **Please follow the instructions from the issue template!!!** Your issue will be examined in all cases, but you can make our life much easier by giving us as much background information as possible.
Do you need professional assistance with your tutor-managed Open edX platform? Overhang.IO offers online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/lts>`__.
Do you need professional assistance with your tutor-managed Open edX platform? Overhang.IO offers online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/pricing>`__.
.. _logging:

View File

@ -1,7 +1,7 @@
tutor-discovery
tutor-ecommerce
#tutor-figures
tutor-lts
tutor-license
tutor-minio
tutor-notes
tutor-xqueue

View File

@ -74,7 +74,7 @@ class ConfigTests(unittest.TestCase):
self.assertEqual("studio.{{ LMS_HOST }}", defaults["CMS_HOST"])
def test_is_service_activated(self):
config = {"ACTIVATE_SERVICE1": True, "ACTIVATE_SERVICE2": False}
config = {"RUN_SERVICE1": True, "RUN_SERVICE2": False}
self.assertTrue(tutor_config.is_service_activated(config, "service1"))
self.assertFalse(tutor_config.is_service_activated(config, "service2"))

View File

@ -85,11 +85,11 @@ class EnvTests(unittest.TestCase):
with tempfile.TemporaryDirectory() as root:
config = tutor_config.load_current(root, defaults)
tutor_config.merge(config, defaults)
config["ACTIVATE_HTTPS"] = True
config["ENABLE_HTTPS"] = True
with unittest.mock.patch.object(fmt, "STDOUT"):
env.save(root, config)
with open(os.path.join(root, "env", "apps", "nginx", "lms.conf")) as f:
self.assertIn("ssl", f.read())
with open(os.path.join(root, "env", "apps", "caddy", "Caddyfile")) as f:
self.assertIn("www.myopenedx.com {", f.read())
def test_patch(self):
patches = {"plugin1": "abcd", "plugin2": "efgh"}

View File

@ -23,7 +23,7 @@ for entrypoint in pkg_resources.iter_entry_points("tutor.plugin.v0"):
path = os.path.join(plugin_root, folder)
if os.path.exists(path):
datas.append((path, os.path.join(plugin_root_module_name, folder)))
# Fix lts import: if we don't declare some modules, pyinstaller does not find them
# Fix license import: if we don't declare some modules, pyinstaller does not find them
hidden_imports.append("tutorlts.__about__")
hidden_imports.append("Crypto.Cipher.AES")
hidden_imports.append("Crypto.Cipher.PKCS1_OAEP")

View File

@ -1 +1 @@
__version__ = "10.5.3"
__version__ = "11.0.0"

View File

@ -1,5 +1,7 @@
import click
from .compose import ScriptRunner
from .local import LocalContext
from .. import config as tutor_config
from .. import env as tutor_env
from .. import fmt
@ -42,12 +44,8 @@ cp OpenEdXMobile/build/outputs/apk/prod/{apk_folder}/*.apk /openedx/data/"""
def docker_run(root, command):
config = tutor_config.load(root)
utils.docker_run(
"--volume={}:/openedx/config/".format(tutor_env.pathjoin(root, "android")),
"--volume={}:/openedx/data/".format(tutor_env.data_path(root, "android")),
config["DOCKER_IMAGE_ANDROID"],
command,
)
runner = ScriptRunner(root, config, LocalContext.docker_compose)
runner.run_job("android", command)
android.add_command(build)

View File

@ -36,6 +36,8 @@ def main():
cli.add_command(plugins_command)
add_plugin_commands(cli)
cli() # pylint: disable=no-value-for-parameter
except KeyboardInterrupt:
pass
except exceptions.TutorError as e:
fmt.echo_error("Error: {}".format(e.args[0]))
sys.exit(1)

View File

@ -120,9 +120,9 @@ def restart(context, services):
else:
for service in services:
if service == "openedx":
if config["ACTIVATE_LMS"]:
if config["RUN_LMS"]:
command += ["lms", "lms-worker"]
if config["ACTIVATE_CMS"]:
if config["RUN_CMS"]:
command += ["cms", "cms-worker"]
else:
command.append(service)

View File

@ -8,6 +8,15 @@ from .. import utils
BASE_IMAGE_NAMES = ["openedx", "forum", "android"]
DEV_IMAGE_NAMES = ["openedx-dev"]
VENDOR_IMAGES = [
"caddy",
"elasticsearch",
"mongodb",
"mysql",
"nginx",
"redis",
"smtp",
]
@click.group(name="images", short_help="Manage docker images")
@ -131,17 +140,9 @@ def all_image_names(config):
def vendor_image_names(config):
vendor_images = [
"elasticsearch",
"memcached",
"mongodb",
"mysql",
"nginx",
"rabbitmq",
"smtp",
]
for image in vendor_images[:]:
if not config.get("ACTIVATE_" + image.upper(), True):
vendor_images = VENDOR_IMAGES[:]
for image in VENDOR_IMAGES:
if not config.get("RUN_" + image.upper(), True):
vendor_images.remove(image)
return vendor_images

View File

@ -24,12 +24,15 @@ def k8s():
def quickstart(context, non_interactive):
click.echo(fmt.title("Interactive platform configuration"))
config = interactive_config.update(context.root, interactive=(not non_interactive))
if config["ACTIVATE_HTTPS"] and not config["WEB_PROXY"]:
if not config["RUN_CADDY"]:
fmt.echo_alert(
"Potentially invalid configuration: ACTIVATE_HTTPS=true WEB_PROXY=false\n"
"You should either disable HTTPS support or configure your platform to use"
" a web proxy. See the Kubernetes section in the Tutor documentation for"
" more information."
"Potentially invalid configuration: RUN_CADDY=false\n"
"This setting might have been defined because you previously set WEB_PROXY=true. This is no longer"
" necessary in order to get Tutor to work on Kubernetes. In Tutor v11+ a Caddy-based load balancer is"
" provided out of the box to handle SSL/TLS certificate generation at runtime. If you disable this"
" service, you will have to configure an Ingress resource and a certificate manager yourself to redirect"
" traffic to the nginx service. See the Kubernetes section in the Tutor documentation for more"
" information."
)
click.echo(fmt.title("Updating the current environment"))
tutor_env.save(context.root, config)
@ -37,6 +40,17 @@ def quickstart(context, non_interactive):
start.callback()
click.echo(fmt.title("Database creation and migrations"))
init.callback(limit=None)
fmt.echo_info(
"""Your Open edX platform is ready and can be accessed at the following urls:
{http}://{lms_host}
{http}://{cms_host}
""".format(
http="https" if config["ENABLE_HTTPS"] else "http",
lms_host=config["LMS_HOST"],
cms_host=config["CMS_HOST"],
)
)
@click.command(help="Run all configured Open edX services")
@ -60,13 +74,14 @@ def start(context):
"--selector",
"app.kubernetes.io/component=volume",
)
# Create everything else except jobs, ingress and issuer
# Create everything else except jobs
utils.kubectl(
"apply",
"--kustomize",
tutor_env.pathjoin(context.root),
"--selector",
"app.kubernetes.io/component notin (job, ingress, issuer)",
# Here use `notin (job, xxx)` when there are other components to ignore
"app.kubernetes.io/component!=job",
)
@ -77,7 +92,7 @@ def stop(context):
utils.kubectl(
"delete",
*resource_selector(config),
"deployments,services,ingress,configmaps,jobs",
"deployments,services,configmaps,jobs",
)
@ -201,6 +216,14 @@ def logs(context, container, follow, tail, service):
utils.kubectl(*command)
@click.command(help="Wait for a pod to become ready")
@click.argument("name")
@click.pass_obj
def wait(context, name):
config = tutor_config.load(context.root)
wait_for_pod_ready(config, name)
@click.command(help="Upgrade from a previous Open edX named release")
@click.option(
"--from", "from_version", default="ironwood", type=click.Choice(["ironwood"])
@ -209,15 +232,26 @@ def logs(context, container, follow, tail, service):
def upgrade(context, from_version):
config = tutor_config.load(context.root)
if from_version == "ironwood":
if not config["ACTIVATE_MONGODB"]:
fmt.echo_info(
"You are not running MongDB (ACTIVATE_MONGODB=false). It is your "
"responsibility to upgrade your MongoDb instance to v3.6. There is "
"nothing left to do."
)
return
message = """Automatic release upgrade is unsupported in Kubernetes. To upgrade from Ironwood, you should upgrade your MongoDb cluster from v3.2 to v3.6. You should run something similar to:
running_version = from_version
if running_version == "ironwood":
upgrade_from_ironwood(config)
running_version = "juniper"
if running_version == "juniper":
running_version = "koa"
def upgrade_from_ironwood(config):
if not config["RUN_MONGODB"]:
fmt.echo_info(
"You are not running MongDB (RUN_MONGODB=false). It is your "
"responsibility to upgrade your MongoDb instance to v3.6. There is "
"nothing left to do to upgrade from Ironwood."
)
return
message = """Automatic release upgrade is unsupported in Kubernetes. To upgrade from Ironwood, you should upgrade
your MongoDb cluster from v3.2 to v3.6. You should run something similar to:
# Upgrade from v3.2 to v3.4
tutor k8s stop
@ -232,7 +266,27 @@ def upgrade(context, from_version):
tutor k8s exec mongodb mongo --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "3.6" })'
tutor config save --unset DOCKER_IMAGE_MONGODB"""
fmt.echo_info(message)
fmt.echo_info(message)
def upgrade_from_juniper(config):
if not config["RUN_MYSQL"]:
fmt.echo_info(
"You are not running MySQL (RUN_MYSQL=false). It is your "
"responsibility to upgrade your MySQL instance to v5.7. There is "
"nothing left to do to upgrade from Juniper."
)
return
message = """Automatic release upgrade is unsupported in Kubernetes. To upgrade from Juniper, you should upgrade
your MySQL database from v5.6 to v5.7. You should run something similar to:
tutor k8s start
tutor k8s exec mysql bash -e -c "mysql_upgrade \
-u $(tutor config printvalue MYSQL_ROOT_USERNAME) \
--password='$(tutor config printvalue MYSQL_ROOT_PASSWORD)'
"""
fmt.echo_info(message)
class K8sClients:
@ -329,12 +383,16 @@ class K8sScriptRunner(scripts.BaseRunner):
job["metadata"]["name"] = job_name
job["metadata"].setdefault("labels", {})
job["metadata"]["labels"]["app.kubernetes.io/name"] = job_name
job["spec"]["template"]["spec"]["containers"][0]["args"] = [
"sh",
"-e",
"-c",
command,
]
# Define k8s entrypoint/args
shell_command = ["sh", "-e", "-c"]
if job["spec"]["template"]["spec"]["containers"][0].get("command") == []:
# Empty "command" (aka: entrypoint) might not be taken into account by jobs, so we need to manually
# override the entrypoint. We do not do this for every job, because some entrypoints are actually useful.
job["spec"]["template"]["spec"]["containers"][0]["command"] = shell_command
container_args = [command]
else:
container_args = shell_command + [command]
job["spec"]["template"]["spec"]["containers"][0]["args"] = container_args
job["spec"]["backoffLimit"] = 1
job["spec"]["ttlSecondsAfterFinished"] = 3600
# Save patched job to "jobs.yml" file
@ -429,4 +487,5 @@ k8s.add_command(importdemocourse)
k8s.add_command(settheme)
k8s.add_command(exec_command)
k8s.add_command(logs)
k8s.add_command(wait)
k8s.add_command(upgrade)

View File

@ -1,5 +1,4 @@
import os
from textwrap import indent
import click
@ -57,8 +56,6 @@ def quickstart(context, non_interactive, pullimages_):
)
click.echo(fmt.title("Stopping any existing platform"))
compose.stop.callback([])
click.echo(fmt.title("HTTPS certificates generation"))
https_create.callback()
if pullimages_:
click.echo(fmt.title("Docker image updates"))
compose.dc_command.callback(["pull"])
@ -75,95 +72,19 @@ Your Open edX platform is ready and can be accessed at the following urls:
{http}://{lms_host}
{http}://{cms_host}
""".format(
http="https" if config["ACTIVATE_HTTPS"] else "http",
http="https" if config["ENABLE_HTTPS"] else "http",
lms_host=config["LMS_HOST"],
cms_host=config["CMS_HOST"],
)
)
@click.group(help="Manage https certificates")
def https():
pass
@click.command(help="Create https certificates", name="create")
@click.pass_obj
def https_create(context):
"""
Note: there are a couple issues with https certificate generation.
1. Certificates are generated and renewed by using port 80, which is not necessarily open.
a. It may be occupied by the nginx container
b. It may be occupied by an external web server
2. On certificate renewal, nginx is not reloaded
"""
config = tutor_config.load(context.root)
runner = compose.ScriptRunner(context.root, config, context.docker_compose)
if not config["ACTIVATE_HTTPS"]:
fmt.echo_info("HTTPS is not activated: certificate generation skipped")
return
script = runner.render("hooks", "certbot", "create")
if config["WEB_PROXY"]:
fmt.echo_info(
"""You are running Tutor behind a web proxy (WEB_PROXY=true): SSL/TLS
certificates must be generated on the host. For instance, to generate
certificates with Let's Encrypt, run:
{}
See the official certbot documentation for your platform: https://certbot.eff.org/""".format(
indent(script, " ")
)
)
return
utils.docker_run(
"--volume",
"{}:/etc/letsencrypt/".format(tutor_env.data_path(context.root, "letsencrypt")),
"-p",
"80:80",
"--entrypoint=sh",
"docker.io/certbot/certbot:latest",
"-e",
"-c",
script,
)
@click.command(help="Renew https certificates", name="renew")
@click.pass_obj
def https_renew(context):
config = tutor_config.load(context.root)
if not config["ACTIVATE_HTTPS"]:
fmt.echo_info("HTTPS is not activated: certificate renewal skipped")
return
if config["WEB_PROXY"]:
fmt.echo_info(
"""You are running Tutor behind a web proxy (WEB_PROXY=true): SSL/TLS
certificates must be renewed on the host. For instance, to renew Let's Encrypt
certificates, run:
certbot renew
See the official certbot documentation for your platform: https://certbot.eff.org/"""
)
return
docker_run = [
"--volume",
"{}:/etc/letsencrypt/".format(tutor_env.data_path(context.root, "letsencrypt")),
"-p",
"80:80",
"certbot/certbot:latest",
"renew",
]
utils.docker_run(*docker_run)
@click.command(help="Upgrade from a previous Open edX named release")
@click.option(
"--from", "from_version", default="ironwood", type=click.Choice(["ironwood"])
"--from",
"from_version",
default="juniper",
type=click.Choice(["ironwood", "juniper"]),
)
@click.option("-I", "--non-interactive", is_flag=True, help="Run non-interactively")
@click.pass_obj
@ -183,8 +104,14 @@ Are you sure you want to continue?"""
fmt.question(question), default=True, abort=True, prompt_suffix=" "
)
if from_version == "ironwood":
running_version = from_version
if running_version == "ironwood":
upgrade_from_ironwood(context, config)
running_version = "juniper"
if running_version == "juniper":
upgrade_from_juniper(context, config)
running_version = "koa"
def upgrade_from_ironwood(context, config):
@ -194,11 +121,11 @@ def upgrade_from_ironwood(context, config):
click.echo(fmt.title("Stopping any existing platform"))
compose.stop.callback([])
if not config["ACTIVATE_MONGODB"]:
if not config["RUN_MONGODB"]:
fmt.echo_info(
"You are not running MongDB (ACTIVATE_MONGODB=false). It is your "
"You are not running MongDB (RUN_MONGODB=false). It is your "
"responsibility to upgrade your MongoDb instance to v3.6. There is "
"nothing left to do."
"nothing left to do to upgrade from Ironwood."
)
return
@ -233,9 +160,37 @@ def upgrade_from_ironwood(context, config):
compose.stop.callback([])
https.add_command(https_create)
https.add_command(https_renew)
local.add_command(https)
def upgrade_from_juniper(context, config):
click.echo(fmt.title("Upgrading from Juniper"))
tutor_env.save(context.root, config)
click.echo(fmt.title("Stopping any existing platform"))
compose.stop.callback([])
if not config["RUN_MYSQL"]:
fmt.echo_info(
"You are not running MySQL (RUN_MYSQL=false). It is your "
"responsibility to upgrade your MySQL instance to v5.7. There is "
"nothing left to do to upgrade from Juniper."
)
return
click.echo(fmt.title("Upgrading MySQL from v5.6 to v5.7"))
compose.start.callback(detach=True, services=["mysql"])
compose.execute.callback(
[
"mysql",
"bash",
"-e",
"-c",
"mysql_upgrade -u {} --password='{}'".format(
config["MYSQL_ROOT_USERNAME"], config["MYSQL_ROOT_PASSWORD"]
),
]
)
compose.stop.callback([])
local.add_command(quickstart)
local.add_command(upgrade)
compose.add_commands(local)

View File

@ -50,11 +50,16 @@ def enable(context, plugin_names):
)
@click.command(help="Disable a plugin")
@click.command(
short_help="Disable a plugin",
help="Disable one or more plugins. Specify 'all' to disable all enabled plugins at once.",
)
@click.argument("plugin_names", metavar="plugin", nargs=-1)
@click.pass_obj
def disable(context, plugin_names):
config = tutor_config.load_user(context.root)
if "all" in plugin_names:
plugin_names = [plugin.name for plugin in plugins.iter_enabled(config)]
for plugin_name in plugin_names:
plugins.disable(config, plugin_name)
delete_plugin(context.root, plugin_name)

View File

@ -134,7 +134,7 @@ def load_plugins(config, defaults):
def is_service_activated(config, service):
return config["ACTIVATE_" + service.upper()]
return config["RUN_" + service.upper()]
def upgrade_obsolete(config):
@ -147,16 +147,35 @@ def upgrade_obsolete(config):
config["OPENEDX_MYSQL_DATABASE"] = config.pop("MYSQL_DATABASE")
if "MYSQL_USERNAME" in config:
config["OPENEDX_MYSQL_USERNAME"] = config.pop("MYSQL_USERNAME")
if "ACTIVATE_NOTES" in config:
if config["ACTIVATE_NOTES"]:
if "RUN_NOTES" in config:
if config["RUN_NOTES"]:
plugins.enable(config, "notes")
config.pop("ACTIVATE_NOTES")
if "ACTIVATE_XQUEUE" in config:
if config["ACTIVATE_XQUEUE"]:
config.pop("RUN_NOTES")
if "RUN_XQUEUE" in config:
if config["RUN_XQUEUE"]:
plugins.enable(config, "xqueue")
config.pop("ACTIVATE_XQUEUE")
config.pop("RUN_XQUEUE")
if "SECRET_KEY" in config:
config["OPENEDX_SECRET_KEY"] = config.pop("SECRET_KEY")
# Replace WEB_PROXY by RUN_CADDY
if "WEB_PROXY" in config:
config["RUN_CADDY"] = not config.pop("WEB_PROXY")
# Rename ACTIVATE_HTTPS to ENABLE_HTTPS
if "ACTIVATE_HTTPS" in config:
config["ENABLE_HTTPS"] = config.pop("ACTIVATE_HTTPS")
# Replace RUN_* variables by RUN_*
for name in [
"ACTIVATE_LMS",
"ACTIVATE_CMS",
"ACTIVATE_FORUM",
"ACTIVATE_ELASTICSEARCH",
"ACTIVATE_MONGODB",
"ACTIVATE_MYSQL",
"ACTIVATE_REDIS",
"ACTIVATE_SMTP",
]:
if name in config:
config[name.replace("ACTIVATE_", "RUN_")] = config.pop(name)
def convert_json2yml(root):

View File

@ -318,7 +318,7 @@ def current_release(root):
"""
Return the name of the current Open edX release.
"""
return {"0": "ironwood", "3": "ironwood", "10": "juniper"}[
return {"0": "ironwood", "3": "ironwood", "10": "juniper", "11": "koa"}[
current_version(root).split(".")[0]
]

View File

@ -40,7 +40,7 @@ def ask_questions(config, defaults):
dev_values = {
"LMS_HOST": "local.overhang.io",
"CMS_HOST": "studio.local.overhang.io",
"ACTIVATE_HTTPS": False,
"ENABLE_HTTPS": False,
}
fmt.echo_info(
"""As you are not running this platform in production, we automatically set the following configuration values:"""
@ -150,7 +150,7 @@ def ask_questions(config, defaults):
"Activate SSL/TLS certificates for HTTPS access? Important note:"
" this will NOT work in a development environment."
),
"ACTIVATE_HTTPS",
"ENABLE_HTTPS",
config,
defaults,
)

View File

@ -1,5 +1,5 @@
# See docs: https://openedx.atlassian.net/wiki/spaces/LEARNER/pages/48792067/App+Configuration+Flags
API_HOST_URL: "{{ "https" if ACTIVATE_HTTPS else "http" }}://{{ LMS_HOST }}"
API_HOST_URL: "{{ "https" if ENABLE_HTTPS else "http" }}://{{ LMS_HOST }}"
ENVIRONMENT_DISPLAY_NAME: "tutor"
PLATFORM_NAME: "{{ PLATFORM_NAME }}"
PLATFORM_DESTINATION_NAME: "{{ LMS_HOST }}"

View File

@ -0,0 +1,11 @@
{{ LMS_HOST }}{% if not ENABLE_HTTPS %}:80{% endif %} {
reverse_proxy nginx:80
}
preview.{{ LMS_HOST }}{% if not ENABLE_HTTPS %}:80{% endif %} {
reverse_proxy nginx:80
}
{{ CMS_HOST }}{% if not ENABLE_HTTPS %}:80{% endif %} {
reverse_proxy nginx:80
}
{{ patch("caddyfile") }}

View File

@ -1,25 +1,12 @@
{% if ACTIVATE_CMS %}
{% if RUN_CMS %}
upstream cms-backend {
server cms:8000 fail_timeout=0;
}
{% if ACTIVATE_HTTPS %}
server {
server_name {{ CMS_HOST }};
listen 80;
return 301 https://$server_name$request_uri;
}
{% endif %}
server {
{% if ACTIVATE_HTTPS %}listen 443 {{ "" if WEB_PROXY else "ssl" }};{% else %}listen 80;{% endif %}
listen 80;
server_name {{ CMS_HOST }};
{% if ACTIVATE_HTTPS and not WEB_PROXY %}
ssl_certificate /etc/letsencrypt/live/{{ LMS_HOST }}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{ LMS_HOST }}/privkey.pem;
{% endif %}
access_log /var/log/nginx/access.log tutor;
client_max_body_size 250M;
server_tokens off;
@ -27,14 +14,8 @@ server {
rewrite ^(.*)/favicon.ico$ /static/images/favicon.ico last;
location @proxy_to_cms_app {
{% if not WEB_PROXY %}
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $remote_addr;
{% endif %}
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_pass http://cms-backend;
}
@ -42,11 +23,6 @@ server {
try_files $uri @proxy_to_cms_app;
}
location ~ ^/media/(?P<file>.*) {
root /var/www/openedx-media;
try_files /$file =404;
expires 31536000s;
}
{{ patch("nginx-cms") }}
{{ patch("nginx-cms")|indent(2) }}
}
{% endif %}

View File

@ -1,26 +1,12 @@
{% if ACTIVATE_LMS %}
{% if RUN_LMS %}
upstream lms-backend {
server lms:8000 fail_timeout=0;
}
{% if ACTIVATE_HTTPS %}
server {
server_name {{ LMS_HOST }} preview.{{ LMS_HOST }};
listen 80;
access_log /var/log/nginx/access.log tutor;
return 301 https://$server_name$request_uri;
}
{% endif %}
server {
{% if ACTIVATE_HTTPS %}listen 443 {{ "" if WEB_PROXY else "ssl" }};{% else %}listen 80;{% endif %}
listen 80;
server_name {{ LMS_HOST }} preview.{{ LMS_HOST }};
{% if ACTIVATE_HTTPS and not WEB_PROXY %}
ssl_certificate /etc/letsencrypt/live/{{ LMS_HOST }}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{ LMS_HOST }}/privkey.pem;
{% endif %}
access_log /var/log/nginx/access.log tutor;
client_max_body_size 4M;
server_tokens off;
@ -28,14 +14,8 @@ server {
rewrite ^(.*)/favicon.ico$ /static/images/favicon.ico last;
location @proxy_to_lms_app {
{% if not WEB_PROXY %}
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $remote_addr;
{% endif %}
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_pass http://lms-backend;
}
@ -59,11 +39,6 @@ server {
client_max_body_size 1049576;
}
location ~ ^/media/(?P<file>.*) {
root /var/www/openedx-media;
try_files /$file =404;
expires 31536000s;
}
{{ patch("nginx-lms") }}
{{ patch("nginx-lms")|indent(2) }}
}
{% endif %}

View File

@ -15,15 +15,15 @@
"ENABLE_LEARNER_RECORDS": false,
"ENABLE_LIBRARY_INDEX": true
},
"LMS_ROOT_URL": "{{ "https" if ACTIVATE_HTTPS else "http" }}://{{ LMS_HOST }}",
"CMS_ROOT_URL": "{{ "https" if ACTIVATE_HTTPS else "http" }}://{{ CMS_HOST }}",
"LMS_ROOT_URL": "{{ "https" if ENABLE_HTTPS else "http" }}://{{ LMS_HOST }}",
"CMS_ROOT_URL": "{{ "https" if ENABLE_HTTPS else "http" }}://{{ CMS_HOST }}",
"CMS_BASE": "{{ CMS_HOST }}",
"LMS_BASE": "{{ LMS_HOST }}",
"CONTACT_EMAIL": "{{ CONTACT_EMAIL }}",
"CELERY_BROKER_TRANSPORT": "amqp",
"CELERY_BROKER_HOSTNAME": "{{ RABBITMQ_HOST }}",
"CELERY_BROKER_USER": "{{ RABBITMQ_USERNAME }}",
"CELERY_BROKER_PASSWORD": "{{ RABBITMQ_PASSWORD }}",
"CELERY_BROKER_TRANSPORT": "redis",
"CELERY_BROKER_HOSTNAME": "{{ REDIS_HOST }}:{{ REDIS_PORT }}",
"CELERY_BROKER_USER": "{{ REDIS_USERNAME }}",
"CELERY_BROKER_PASSWORD": "{{ REDIS_PASSWORD }}",
"ALTERNATE_WORKER_QUEUES": "lms",
"ENABLE_COMPREHENSIVE_THEMING": true,
"COMPREHENSIVE_THEME_DIRS": ["/openedx/themes"],
@ -37,7 +37,7 @@
"EMAIL_HOST": "{{ SMTP_HOST }}",
"EMAIL_PORT": {{ SMTP_PORT }},
"EMAIL_USE_TLS": {{ "true" if SMTP_USE_TLS else "false" }},
"HTTPS": "{{ "on" if ACTIVATE_HTTPS else "off" }}",
"HTTPS": "{{ "on" if ENABLE_HTTPS else "off" }}",
"LANGUAGE_CODE": "{{ LANGUAGE_CODE }}",
"SESSION_COOKIE_DOMAIN": ".{{ LMS_HOST|common_domain(CMS_HOST) }}",
{{ patch("cms-env", separator=",\n", suffix=",")|indent(2) }}
@ -45,48 +45,41 @@
"default": {
"KEY_PREFIX": "default",
"VERSION": "1",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"general": {
"KEY_PREFIX": "general",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"mongo_metadata_inheritance": {
"KEY_PREFIX": "mongo_metadata_inheritance",
"TIMEOUT": 300,
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"staticfiles": {
"KEY_PREFIX": "staticfiles_cms",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "staticfiles_cms"
},
"configuration": {
"KEY_PREFIX": "configuration",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"celery": {
"KEY_PREFIX": "celery",
"TIMEOUT": "7200",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"course_structure_cache": {
"KEY_PREFIX": "course_structure",
"TIMEOUT": "7200",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
}
},
{% include "apps/openedx/config/partials/auth.json" %}

View File

@ -22,15 +22,15 @@
"ENABLE_OAUTH2_PROVIDER": true,
"ENABLE_THIRD_PARTY_AUTH": true
},
"LMS_ROOT_URL": "{{ "https" if ACTIVATE_HTTPS else "http" }}://{{ LMS_HOST }}",
"CMS_ROOT_URL": "{{ "https" if ACTIVATE_HTTPS else "http" }}://{{ CMS_HOST }}",
"LMS_ROOT_URL": "{{ "https" if ENABLE_HTTPS else "http" }}://{{ LMS_HOST }}",
"CMS_ROOT_URL": "{{ "https" if ENABLE_HTTPS else "http" }}://{{ CMS_HOST }}",
"CMS_BASE": "{{ CMS_HOST }}",
"LMS_BASE": "{{ LMS_HOST }}",
"CONTACT_EMAIL": "{{ CONTACT_EMAIL }}",
"CELERY_BROKER_TRANSPORT": "amqp",
"CELERY_BROKER_HOSTNAME": "{{ RABBITMQ_HOST }}",
"CELERY_BROKER_USER": "{{ RABBITMQ_USERNAME }}",
"CELERY_BROKER_PASSWORD": "{{ RABBITMQ_PASSWORD }}",
"CELERY_BROKER_TRANSPORT": "redis",
"CELERY_BROKER_HOSTNAME": "{{ REDIS_HOST }}:{{ REDIS_PORT }}",
"CELERY_BROKER_USER": "{{ REDIS_USERNAME }}",
"CELERY_BROKER_PASSWORD": "{{ REDIS_PASSWORD }}",
"ALTERNATE_WORKER_QUEUES": "cms",
"COMMENTS_SERVICE_URL": "http://{{ FORUM_HOST }}:4567",
"COMMENTS_SERVICE_KEY": "forumapikey",
@ -46,7 +46,7 @@
"EMAIL_HOST": "{{ SMTP_HOST }}",
"EMAIL_PORT": {{ SMTP_PORT }},
"EMAIL_USE_TLS": {{ "true" if SMTP_USE_TLS else "false" }},
"HTTPS": "{{ "on" if ACTIVATE_HTTPS else "off" }}",
"HTTPS": "{{ "on" if ENABLE_HTTPS else "off" }}",
"LANGUAGE_CODE": "{{ LANGUAGE_CODE }}",
"SESSION_COOKIE_DOMAIN": ".{{ LMS_HOST|common_domain(CMS_HOST) }}",
{{ patch("lms-env", separator=",\n", suffix=",")|indent(2) }}
@ -54,54 +54,46 @@
"default": {
"KEY_PREFIX": "default",
"VERSION": "1",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"general": {
"KEY_PREFIX": "general",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"mongo_metadata_inheritance": {
"KEY_PREFIX": "mongo_metadata_inheritance",
"TIMEOUT": 300,
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"staticfiles": {
"KEY_PREFIX": "staticfiles_lms",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "staticfiles_lms"
},
"configuration": {
"KEY_PREFIX": "configuration",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"celery": {
"KEY_PREFIX": "celery",
"TIMEOUT": "7200",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"course_structure_cache": {
"KEY_PREFIX": "course_structure",
"TIMEOUT": "7200",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
},
"ora2-storage": {
"KEY_PREFIX": "ora2-storage",
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"KEY_FUNCTION": "util.memcache.safe_key",
"LOCATION": "{{ MEMCACHED_HOST }}:{{ MEMCACHED_PORT }}"
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/1"
}
},
{% include "apps/openedx/config/partials/auth.json" %}

View File

@ -0,0 +1 @@
{% include "apps/openedx/settings/partials/pre_common_all.py" %}

View File

@ -9,6 +9,4 @@ ALLOWED_HOSTS = [
"cms",
]
MIDDLEWARE.insert(0, "whitenoise.middleware.WhiteNoiseMiddleware")
{{ patch("openedx-cms-production-settings") }}

View File

@ -0,0 +1 @@
{% include "apps/openedx/settings/partials/pre_common_all.py" %}

View File

@ -10,9 +10,7 @@ ALLOWED_HOSTS = [
"lms",
]
MIDDLEWARE.insert(0, "whitenoise.middleware.WhiteNoiseMiddleware")
{% if ACTIVATE_HTTPS %}
{% if ENABLE_HTTPS %}
# Properly set the "secure" attribute on session/csrf cookies. This is required in
# Chrome to support samesite=none cookies.
SESSION_COOKIE_SECURE = True

View File

@ -2,6 +2,8 @@
import json
import os
from xmodule.modulestore.modulestore_settings import update_module_store_settings
# Mongodb connection parameters: simply modify `mongodb_parameters` to affect all connections to MongoDb.
mongodb_parameters = {
"host": "{{ MONGODB_HOST }}",
@ -27,6 +29,8 @@ DATA_DIR = "/openedx/data/"
for store in MODULESTORE["default"]["OPTIONS"]["stores"]:
store["OPTIONS"]["fs_root"] = DATA_DIR
# Behave like memcache when it comes to connection errors
DJANGO_REDIS_IGNORE_EXCEPTIONS = True
DEFAULT_FROM_EMAIL = ENV_TOKENS.get("DEFAULT_FROM_EMAIL", ENV_TOKENS["CONTACT_EMAIL"])
DEFAULT_FEEDBACK_EMAIL = ENV_TOKENS.get("DEFAULT_FEEDBACK_EMAIL", ENV_TOKENS["CONTACT_EMAIL"])
@ -42,7 +46,7 @@ API_ACCESS_MANAGER_EMAIL = ENV_TOKENS.get("API_ACCESS_MANAGER_EMAIL", ENV_TOKENS
API_ACCESS_FROM_EMAIL = ENV_TOKENS.get("API_ACCESS_FROM_EMAIL", ENV_TOKENS["CONTACT_EMAIL"])
# Get rid completely of coursewarehistoryextended, as we do not use the CSMH database
INSTALLED_APPS.remove("coursewarehistoryextended")
INSTALLED_APPS.remove("lms.djangoapps.coursewarehistoryextended")
DATABASE_ROUTERS.remove(
"openedx.core.lib.django_courseware_routers.StudentModuleHistoryExtendedRouter"
)
@ -82,15 +86,6 @@ LOGGING["handlers"]["tracking"] = {
"formatter": "standard",
}
LOGGING["loggers"]["tracking"]["handlers"] = ["console", "local", "tracking"]
# Disable django/drf deprecation warnings
import logging
import warnings
from django.utils.deprecation import RemovedInDjango30Warning, RemovedInDjango31Warning
from rest_framework import RemovedInDRF310Warning, RemovedInDRF311Warning
warnings.simplefilter('ignore', RemovedInDjango30Warning)
warnings.simplefilter('ignore', RemovedInDjango31Warning)
warnings.simplefilter('ignore', RemovedInDRF310Warning)
warnings.simplefilter('ignore', RemovedInDRF311Warning)
# Email
EMAIL_USE_SSL = {{ SMTP_USE_SSL }}

View File

@ -3,6 +3,10 @@
######## Common LMS settings
LOGIN_REDIRECT_WHITELIST = ["{{ CMS_HOST }}"]
# Better layout of honor code/tos links during registration
REGISTRATION_EXTRA_FIELDS["terms_of_service"] = "required"
REGISTRATION_EXTRA_FIELDS["honor_code"] = "hidden"
# This url must not be None and should not be used anywhere
LEARNING_MICROFRONTEND_URL = "http://learn.openedx.org"

View File

@ -0,0 +1,10 @@
# Silence overly verbose warnings
import logging
import warnings
from django.utils.deprecation import RemovedInDjango30Warning, RemovedInDjango31Warning
from rest_framework import RemovedInDRF310Warning, RemovedInDRF311Warning
warnings.simplefilter('ignore', RemovedInDjango30Warning)
warnings.simplefilter('ignore', RemovedInDjango31Warning)
warnings.simplefilter('ignore', RemovedInDRF310Warning)
warnings.simplefilter('ignore', RemovedInDRF311Warning)
warnings.simplefilter('ignore', DeprecationWarning)

View File

@ -0,0 +1,41 @@
# https://raw.githubusercontent.com/redis/redis/6.0/redis.conf
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile ""
databases 16
################################ SNAPSHOTTING ################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dir /openedx/redis/data/
dbfilename dump.rdb
rdb-del-sync-files no
############################## APPEND ONLY MODE ###############################
# http://redis.io/topics/persistence
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes

View File

@ -1,6 +1,7 @@
FROM docker.io/ubuntu:18.04
FROM docker.io/ubuntu:20.04
MAINTAINER Overhang.io <contact@overhang.io>
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && \
apt upgrade -y && \
apt install -y wget unzip git openjdk-8-jre openjdk-8-jdk
@ -25,7 +26,7 @@ RUN yes | /openedx/android-sdk/tools/bin/sdkmanager --sdk_root=${ANDROID_HOME} -
# Install android app repo
ARG ANDROID_APP_REPOSITORY=https://github.com/edx/edx-app-android
ARG ANDROID_APP_VERSION=release/2.21.1
ARG ANDROID_APP_VERSION=release/2.23.2
RUN git clone $ANDROID_APP_REPOSITORY --branch $ANDROID_APP_VERSION /openedx/edx-app-android
WORKDIR /openedx/edx-app-android
@ -37,5 +38,3 @@ RUN ./gradlew tasks
COPY ./edx.properties ./OpenEdXMobile/edx.properties
RUN mkdir /openedx/config
RUN ln -s /openedx/config/gradle.properties ./OpenEdXMobile/gradle.properties
ENTRYPOINT ["sh", "-e", "-c"]

View File

@ -1,8 +1,7 @@
FROM docker.io/ubuntu:18.04
FROM docker.io/ubuntu:20.04
MAINTAINER Overhang.io <contact@overhang.io>
RUN mkdir /openedx
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && \
apt upgrade -y && \
apt install -y git wget autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm-dev
@ -13,6 +12,8 @@ RUN wget -O /tmp/dockerize.tar.gz https://github.com/jwilder/dockerize/releases/
&& tar -C /usr/local/bin -xzvf /tmp/dockerize.tar.gz \
&& rm /tmp/dockerize.tar.gz
RUN mkdir /openedx
# Install ruby-build for building specific version of ruby
# The ruby-build version should be periodically updated to reflect the latest release
ARG RUBY_BUILD_VERSION=v20200401
@ -39,7 +40,7 @@ RUN chmod a+x /openedx/bin/*
ENV PATH /openedx/bin:${PATH}
ENTRYPOINT ["docker-entrypoint.sh"]
ENV RACK_ENV staging
ENV SINATRA_ENV staging
ENV NEW_RELIC_ENABLE false
ENV API_KEY forumapikey
ENV SEARCH_SERVER "http://elasticsearch:9200"

View File

@ -8,8 +8,10 @@ RUN apt update && \
# Install dev python requirements
RUN pip install -r requirements/edx/development.txt
# Pinning the jedi library is required in python 3.5 to avoid deprecation warnings from ipython
RUN pip install ipdb==0.13.2 ipython==7.9.0 jedi==0.15.2
# We install ipython from source to avoid too many deprecation warnings
# https://github.com/ipython/ipython/issues/12206
# We might be able to avoid this once they make a release later than 7.19.0.
RUN pip install ipdb==0.13.4 git+https://github.com/ipython/ipython.git@d0649a54a8936a8019d54549779dc92bcbde4e68#egg=ipython
# Recompile static assets: in development mode all static assets are stored in edx-platform,
# and the location of these files is stored in webpack-stats.json. If we don't recompile

View File

@ -1,7 +1,8 @@
###### Minimal image with base system requirements for most stages
FROM docker.io/ubuntu:16.04 as minimal
FROM docker.io/ubuntu:20.04 as minimal
MAINTAINER Overhang.io <contact@overhang.io>
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && \
apt install -y build-essential curl git language-pack-en
ENV LC_ALL en_US.UTF-8
@ -13,11 +14,11 @@ RUN apt update && \
apt install -y libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
ARG PYTHON_VERSION=3.5.9
ARG PYTHON_VERSION=3.8.6
ENV PYENV_ROOT /opt/pyenv
RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v1.2.18 --depth 1
RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v1.2.21 --depth 1
RUN $PYENV_ROOT/bin/pyenv install $PYTHON_VERSION
RUN $PYENV_ROOT/versions/$PYTHON_VERSION/bin/pyvenv /openedx/venv
RUN $PYENV_ROOT/versions/$PYTHON_VERSION/bin/python -m venv /openedx/venv
###### Install Dockerize to wait for mysql DB availability
FROM minimal as dockerize
@ -35,31 +36,19 @@ RUN mkdir -p /openedx/edx-platform && \
WORKDIR /openedx/edx-platform
# Patch edx-platform
# Get rid of lepl-related warnings
# https://github.com/edx/edx-platform/pull/24059
# https://github.com/overhangio/edx-platform/tree/overhangio/lepl-rfc6266-warning
RUN curl https://github.com/overhangio/edx-platform/commit/5f21bbe77056d71ca61b97b6badcff3c1a31b858.patch | git apply -
# Fix creation of LTI provider objects
# https://github.com/edx/edx-platform/pull/24055
# https://github.com/overhangio/edx-platform/tree/regisb/fix-lti-provider-admin
RUN curl https://github.com/overhangio/edx-platform/commit/089b26eed0302ed1f9a5b24c5f3e563dd44abb04.patch | git apply -
# Fix problem viewing when CSMH is disabled
# https://github.com/edx/edx-platform/pull/24237
# https://github.com/overhangio/edx-platform/tree/overhangio/fix-no-csmh
RUN curl https://github.com/overhangio/edx-platform/commit/6dbf2eddf7a4563c04c3b51edf5e131106d69e19.patch | git apply -
# Fix upload of video transcripts to s3
# https://github.com/edx/edx-platform/pull/24800
RUN curl https://github.com/edx/edx-platform/commit/80fa2cae128e2a1fd8ab298351b7b36c9d139e6c.patch | git apply -
# Make it possible to disable learner records globally
# https://github.com/edx/edx-platform/pull/25182
# https://github.com/overhangio/edx-platform/tree/overhangio/disable-learner-records-from-settings
RUN curl https://github.com/overhangio/edx-platform/commit/58f20a0547355080eeee346104a1719ad806902e.patch | git apply -
# Fix security issues
# https://github.com/edx/edx-platform/pull/25782/commits
RUN curl https://github.com/edx/edx-platform/commit/c03857b78d6204ed3b9a3093367348ebfaaf7d04.patch | git apply -
RUN curl https://github.com/overhangio/edx-platform/commit/63e8fcb403f18ff24ef9ef6dbd5d38a7fa82fed6.patch | git apply -
# Fix inconvenient pavelib warning
# https://github.com/edx/edx-platform/pull/25771
# https://github.com/overhangio/edx-platform/tree/overhangio/fix-paver-warning
RUN curl https://github.com/overhangio/edx-platform/commit/bc0ab09f9945bd14aa6be1dbbf928cce58f079d2.patch | git apply -
###### Download extra locales to /openedx/locale/contrib/locale
FROM minimal as locales
# TODO: openedx-i18n is not yet tagged for koa.1: replace versions below by OPENEDX_COMMON_VERSION
ARG OPENEDX_I18N_VERSION=open-release/koa.test02
RUN cd /tmp \
&& curl -L -o openedx-i18n.tar.gz https://github.com/openedx/openedx-i18n/archive/{{ OPENEDX_COMMON_VERSION }}.tar.gz \
&& tar xzf /tmp/openedx-i18n.tar.gz \
@ -86,14 +75,14 @@ RUN pip install setuptools==44.1.0 pip==20.0.2 wheel==0.34.2
# Install base requirements
RUN pip install -r ./requirements/edx/base.txt
# Install patched version of ora2
RUN pip install https://github.com/overhangio/edx-ora2/archive/overhangio/boto2to3.zip
# Install scorm xblock
RUN pip install "openedx-scorm-xblock<11.0.0,>=10.0.1"
RUN pip install "openedx-scorm-xblock<12.0.0,>=11.0.0"
# Install whitenoise, for serving static assets
RUN pip install "whitenoise==5.1.0"
# Install django-redis for using redis as a django cache
RUN pip install django-redis==4.12.1
# Install uwsgi
RUN pip install uwsgi==2.0.19.1
# Install private requirements: this is useful for installing custom xblocks.
COPY ./requirements/ /openedx/requirements
@ -120,7 +109,7 @@ FROM minimal as production
# Install system requirements
RUN apt update && \
apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng12-dev libsqlite3-dev libxmlsec1-dev lynx ntp pkg-config rdfind && \
apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng-dev libsqlite3-dev libxmlsec1-dev lynx ntp pkg-config rdfind && \
rm -rf /var/lib/apt/lists/*
COPY --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize
@ -203,6 +192,13 @@ ENV SETTINGS tutor.production
ENTRYPOINT ["docker-entrypoint.sh"]
# Run server
COPY gunicorn_conf.py /openedx/gunicorn_conf.py
EXPOSE 8000
CMD gunicorn -c /openedx/gunicorn_conf.py --name ${SERVICE_VARIANT} --bind=0.0.0.0:8000 --max-requests=1000 --max-requests-jitter=100 --timeout=120 --access-logfile - ${SERVICE_VARIANT}.wsgi:application
CMD uwsgi \
--static-map /static=/openedx/staticfiles/ \
--static-map /media=/openedx/media/ \
--http 0.0.0.0:8000 \
--thunder-lock \
--single-interpreter \
--enable-threads \
--processes=${UWSGI_WORKERS:-2} \
--wsgi-file ${SERVICE_VARIANT}/wsgi.py

View File

@ -1,3 +0,0 @@
#! /bin/bash
echo "Reloading gunicorn process..."
kill -HUP $(pidof /openedx/venv/bin/python3 /openedx/venv/bin/gunicorn)

View File

@ -0,0 +1,3 @@
#! /bin/bash
echo "Reloading uwsgi process..."
kill -HUP $(pidof /openedx/venv/bin/python3 /openedx/venv/bin/uwsgi)

View File

@ -1,4 +0,0 @@
import os
# Set the number of gunicorn workers
workers = int(os.environ.get("GUNICORN_WORKERS", "2"))

View File

@ -1 +1 @@
EDX_PLATFORM_REVISION: juniper
EDX_PLATFORM_REVISION: koa

View File

@ -10,16 +10,16 @@ ID: "{{ 24|random_string }}"
LMS_HOST: "www.myopenedx.com"
# The following are default values
ACTIVATE_LMS: true
ACTIVATE_CMS: true
ACTIVATE_FORUM: true
ACTIVATE_ELASTICSEARCH: true
ACTIVATE_HTTPS: false
ACTIVATE_MEMCACHED: true
ACTIVATE_MONGODB: true
ACTIVATE_MYSQL: true
ACTIVATE_RABBITMQ: true
ACTIVATE_SMTP: true
RUN_CADDY: true
RUN_LMS: true
RUN_CMS: true
RUN_FORUM: true
RUN_ELASTICSEARCH: true
ENABLE_HTTPS: false
RUN_MONGODB: true
RUN_MYSQL: true
RUN_REDIS: true
RUN_SMTP: true
CMS_HOST: "studio.{{ LMS_HOST }}"
CONTACT_EMAIL: "contact@{{ LMS_HOST }}"
OPENEDX_AWS_ACCESS_KEY: ""
@ -32,13 +32,13 @@ DOCKER_REGISTRY: "docker.io/"
DOCKER_IMAGE_OPENEDX: "{{ DOCKER_REGISTRY }}overhangio/openedx:{{ TUTOR_VERSION }}"
DOCKER_IMAGE_OPENEDX_DEV: "{{ DOCKER_REGISTRY }}overhangio/openedx-dev:{{ TUTOR_VERSION }}"
DOCKER_IMAGE_ANDROID: "{{ DOCKER_REGISTRY }}overhangio/openedx-android:{{ TUTOR_VERSION }}"
DOCKER_IMAGE_CADDY: "{{ DOCKER_REGISTRY }}caddy:2.2.1"
DOCKER_IMAGE_FORUM: "{{ DOCKER_REGISTRY }}overhangio/openedx-forum:{{ TUTOR_VERSION }}"
DOCKER_IMAGE_MEMCACHED: "{{ DOCKER_REGISTRY }}memcached:1.4.38"
DOCKER_IMAGE_MONGODB: "{{ DOCKER_REGISTRY }}mongo:3.6.18"
DOCKER_IMAGE_MYSQL: "{{ DOCKER_REGISTRY }}mysql:5.6.49"
DOCKER_IMAGE_MYSQL: "{{ DOCKER_REGISTRY }}mysql:5.7.32"
DOCKER_IMAGE_ELASTICSEARCH: "{{ DOCKER_REGISTRY }}elasticsearch:1.5.2"
DOCKER_IMAGE_NGINX: "{{ DOCKER_REGISTRY }}nginx:1.13"
DOCKER_IMAGE_RABBITMQ: "{{ DOCKER_REGISTRY }}rabbitmq:3.6.10-management-alpine"
DOCKER_IMAGE_REDIS: "{{ DOCKER_REGISTRY }}redis:6.0.9"
DOCKER_IMAGE_SMTP: "{{ DOCKER_REGISTRY }}namshi/smtp:latest"
LOCAL_PROJECT_NAME: "tutor_local"
ELASTICSEARCH_HOST: "elasticsearch"
@ -47,38 +47,35 @@ ELASTICSEARCH_SCHEME: "http"
ELASTICSEARCH_HEAP_SIZE: 1g
FORUM_HOST: "forum"
JWT_COMMON_AUDIENCE: "openedx"
JWT_COMMON_ISSUER: "{% if ACTIVATE_HTTPS %}https{% else %}http{% endif %}://{{ LMS_HOST }}/oauth2"
JWT_COMMON_ISSUER: "{% if ENABLE_HTTPS %}https{% else %}http{% endif %}://{{ LMS_HOST }}/oauth2"
JWT_COMMON_SECRET_KEY: "{{ OPENEDX_SECRET_KEY }}"
JWT_RSA_PRIVATE_KEY: "{{ 2048|rsa_private_key }}"
K8S_NAMESPACE: "openedx"
LANGUAGE_CODE: "en"
MEMCACHED_HOST: "memcached"
MEMCACHED_PORT: 11211
MONGODB_HOST: "mongodb"
MONGODB_DATABASE: "openedx"
MONGODB_PORT: 27017
MONGODB_USERNAME: ""
MONGODB_PASSWORD: ""
OPENEDX_CMS_GUNICORN_WORKERS: 2
OPENEDX_LMS_GUNICORN_WORKERS: 2
OPENEDX_CMS_UWSGI_WORKERS: 2
OPENEDX_LMS_UWSGI_WORKERS: 2
OPENEDX_MYSQL_DATABASE: "openedx"
OPENEDX_CSMH_MYSQL_DATABASE: "{{ OPENEDX_MYSQL_DATABASE }}_csmh"
OPENEDX_MYSQL_USERNAME: "openedx"
OPENEDX_COMMON_VERSION: "open-release/juniper.3"
OPENEDX_COMMON_VERSION: "open-release/koa.1"
MYSQL_HOST: "mysql"
MYSQL_PORT: 3306
MYSQL_ROOT_USERNAME: "root"
NGINX_HTTP_PORT: 80
NGINX_HTTPS_PORT: 443
PLATFORM_NAME: "My Open edX"
PLUGINS: []
RABBITMQ_HOST: "rabbitmq"
RABBITMQ_USERNAME: ""
RABBITMQ_PASSWORD: ""
REDIS_HOST: "redis"
REDIS_PORT: 6379
REDIS_USERNAME: ""
REDIS_PASSWORD: ""
SMTP_HOST: "smtp"
SMTP_PORT: 25
SMTP_USERNAME: ""
SMTP_PASSWORD: ""
SMTP_USE_TLS: false
SMTP_USE_SSL: false
WEB_PROXY: false

View File

@ -1,2 +0,0 @@
certbot certonly --standalone -n --agree-tos -m admin@{{ LMS_HOST }} -d {{ LMS_HOST }} -d {{ CMS_HOST }} -d preview.{{ LMS_HOST }}
{{ patch("https-create") }}

View File

@ -1,4 +1,2 @@
export MONGOHQ_URL="mongodb://$MONGODB_AUTH$MONGODB_HOST:$MONGODB_PORT/cs_comments_service"
bundle exec rake search:initialize
bundle exec rake search:rebuild_index

View File

@ -1,3 +1,40 @@
{% if RUN_CADDY %}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: caddy
labels:
app.kubernetes.io/name: caddy
spec:
selector:
matchLabels:
app.kubernetes.io/name: caddy
template:
metadata:
labels:
app.kubernetes.io/name: caddy
spec:
containers:
- name: caddy
image: {{ DOCKER_IMAGE_CADDY }}
volumeMounts:
- mountPath: /etc/caddy/
name: config
- mountPath: /data/
name: data
ports:
- containerPort: 80
- containerPort: 443
volumes:
- name: config
configMap:
name: caddy-config
- name: data
persistentVolumeClaim:
claimName: caddy
{% endif %}
{% if RUN_CMS %}
---
apiVersion: apps/v1
kind: Deployment
@ -61,7 +98,7 @@ spec:
containers:
- name: cms-worker
image: {{ DOCKER_IMAGE_OPENEDX }}
args: ["./manage.py", "cms", "celery", "worker", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--maxtasksperchild", "100", "--exclude-queues=edx.lms.core.default"]
args: ["celery", "worker", "--app=cms.celery", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--maxtasksperchild", "100", "--exclude-queues=edx.lms.core.default"]
env:
- name: SERVICE_VARIANT
value: cms
@ -84,7 +121,8 @@ spec:
- name: config
configMap:
name: openedx-config
{% if ACTIVATE_FORUM %}
{% endif %}
{% if RUN_FORUM %}
---
apiVersion: apps/v1
kind: Deployment
@ -116,6 +154,7 @@ spec:
- name: MONGODB_PORT
value: "{{ MONGODB_PORT }}"
{% endif %}
{% if RUN_LMS %}
---
apiVersion: apps/v1
kind: Deployment
@ -176,7 +215,7 @@ spec:
containers:
- name: lms-worker
image: {{ DOCKER_IMAGE_OPENEDX }}
args: ["./manage.py", "lms", "celery", "worker", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--maxtasksperchild", "100", "--exclude-queues=edx.cms.core.default"]
args: ["celery", "worker", "--app=cms.celery", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--maxtasksperchild", "100", "--exclude-queues=edx.cms.core.default"]
env:
- name: SERVICE_VARIANT
value: lms
@ -199,7 +238,8 @@ spec:
- name: config
configMap:
name: openedx-config
{% if ACTIVATE_ELASTICSEARCH %}
{% endif %}
{% if RUN_ELASTICSEARCH %}
---
apiVersion: apps/v1
kind: Deployment
@ -238,30 +278,7 @@ spec:
persistentVolumeClaim:
claimName: elasticsearch
{% endif %}
{% if ACTIVATE_MEMCACHED %}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: memcached
labels:
app.kubernetes.io/name: memcached
spec:
selector:
matchLabels:
app.kubernetes.io/name: memcached
template:
metadata:
labels:
app.kubernetes.io/name: memcached
spec:
containers:
- name: memcached
image: {{ DOCKER_IMAGE_MEMCACHED }}
ports:
- containerPort: 11211
{% endif %}
{% if ACTIVATE_MONGODB %}
{% if RUN_MONGODB %}
---
apiVersion: apps/v1
kind: Deployment
@ -295,7 +312,7 @@ spec:
persistentVolumeClaim:
claimName: mongodb
{% endif %}
{% if ACTIVATE_MYSQL %}
{% if RUN_MYSQL %}
---
apiVersion: apps/v1
kind: Deployment
@ -317,7 +334,9 @@ spec:
containers:
- name: mysql
image: {{ DOCKER_IMAGE_MYSQL }}
args: ["mysqld", "--character-set-server=utf8", "--collation-server=utf8_general_ci"]
# Note the ignore-db-dir: this is because ext4 volumes are created with a lost+found directory in them, which causes mysql
# initialization to fail
args: ["mysqld", "--character-set-server=utf8", "--collation-server=utf8_general_ci", "--ignore-db-dir=lost+found"]
env:
- name: MYSQL_ROOT_PASSWORD
value: "{{ MYSQL_ROOT_PASSWORD }}"
@ -331,7 +350,7 @@ spec:
persistentVolumeClaim:
claimName: mysql
{% endif %}
{% if ACTIVATE_SMTP %}
{% if RUN_SMTP %}
---
apiVersion: apps/v1
kind: Deployment
@ -379,42 +398,46 @@ spec:
{{ patch("k8s-deployments-nginx-volume-mounts")|indent(12) }}
ports:
- containerPort: 80
- containerPort: 443
volumes:
- name: config
configMap:
name: nginx-config
{{ patch("k8s-deployments-nginx-volumes")|indent(8) }}
{% if ACTIVATE_RABBITMQ %}
{% if RUN_REDIS %}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq
name: redis
labels:
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/name: redis
spec:
selector:
matchLabels:
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/name: redis
strategy:
type: Recreate
template:
metadata:
labels:
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/name: redis
spec:
containers:
- name: rabbitmq
image: {{ DOCKER_IMAGE_RABBITMQ }}
- name: redis
image: {{ DOCKER_IMAGE_REDIS }}
ports:
- containerPort: 5672
- containerPort: {{ REDIS_PORT }}
volumeMounts:
- mountPath: /var/lib/rabbitmq
- mountPath: /openedx/redis/config/
name: config
- mountPath: /openedx/redis/data
name: data
volumes:
- name: config
configMap:
name: redis-config
- name: data
persistentVolumeClaim:
claimName: rabbitmq
claimName: redis
{% endif %}
{{ patch("k8s-deployments") }}

View File

@ -1,57 +0,0 @@
---{% set hosts = [LMS_HOST, "preview." + LMS_HOST, CMS_HOST] %}
# This is an nginx-based Ingress object that relies on a letsencrypt Issuer for SSL
# termination. By default, this ingress and issuer are *not* deployed to the Kubernetes
# cluster when running "quickstart". This is because there exist many different
# ingress/issuer combinations and it should not be Tutor's job to choose which one you
# should use.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: web
labels:
app.kubernetes.io/name: web
app.kubernetes.io/component: ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-body-size: 1000m
{% if ACTIVATE_HTTPS%}kubernetes.io/tls-acme: "true"
cert-manager.io/issuer: letsencrypt{% endif %}
spec:
rules:
{% for host in hosts %}
- host: {{ host }}
http:
paths:
- backend:
serviceName: nginx
servicePort: {% if ACTIVATE_HTTPS %}443{% else %}80{% endif %}{% endfor %}
{{ patch("k8s-ingress-rules")|indent(2) }}
{% if ACTIVATE_HTTPS %}
tls:
- hosts:
{% for host in hosts %}
- {{ host }}{% endfor %}
{{ patch("k8s-ingress-tls-hosts")|indent(6) }}
secretName: letsencrypt
{%endif%}
{% if ACTIVATE_HTTPS %}
---
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: letsencrypt
labels:
app.kubernetes.io/name: letsencrypt
app.kubernetes.io/component: issuer
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: {{ CONTACT_EMAIL }}
privateKeySecretRef:
name: letsencrypt-privatekey
solvers:
- selector: {}
http01:
ingress:
class: nginx
{% endif %}

View File

@ -1,3 +1,20 @@
{% if RUN_CADDY %}
---
apiVersion: v1
kind: Service
metadata:
name: caddy
spec:
type: LoadBalancer
ports:
- port: 80
name: http
- port: 443
name: https
selector:
app.kubernetes.io/name: caddy
{% endif %}
{% if RUN_CMS %}
---
apiVersion: v1
kind: Service
@ -10,7 +27,8 @@ spec:
protocol: TCP
selector:
app.kubernetes.io/name: cms
{% if ACTIVATE_FORUM %}
{% endif %}
{% if RUN_FORUM %}
---
apiVersion: v1
kind: Service
@ -24,6 +42,7 @@ spec:
selector:
app.kubernetes.io/name: forum
{% endif %}
{% if RUN_LMS %}
---
apiVersion: v1
kind: Service
@ -36,7 +55,8 @@ spec:
protocol: TCP
selector:
app.kubernetes.io/name: lms
{% if ACTIVATE_ELASTICSEARCH %}
{% endif %}
{% if RUN_ELASTICSEARCH %}
---
apiVersion: v1
kind: Service
@ -50,21 +70,7 @@ spec:
selector:
app.kubernetes.io/name: elasticsearch
{% endif %}
{% if ACTIVATE_MEMCACHED %}
---
apiVersion: v1
kind: Service
metadata:
name: memcached
spec:
type: NodePort
ports:
- port: 11211
protocol: TCP
selector:
app.kubernetes.io/name: memcached
{% endif %}
{% if ACTIVATE_MONGODB %}
{% if RUN_MONGODB %}
---
apiVersion: v1
kind: Service
@ -78,7 +84,7 @@ spec:
selector:
app.kubernetes.io/name: mongodb
{% endif %}
{% if ACTIVATE_MYSQL %}
{% if RUN_MYSQL %}
---
apiVersion: v1
kind: Service
@ -102,25 +108,23 @@ spec:
ports:
- port: 80
name: http
- port: 443
name: https
selector:
app.kubernetes.io/name: nginx
{% if ACTIVATE_RABBITMQ %}
{% if RUN_REDIS %}
---
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
name: redis
spec:
type: NodePort
ports:
- port: 5672
- port: {{ REDIS_PORT }}
protocol: TCP
selector:
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/name: redis
{% endif %}
{% if ACTIVATE_SMTP %}
{% if RUN_SMTP %}
---
apiVersion: v1
kind: Service

View File

@ -1,4 +1,20 @@
{% if ACTIVATE_ELASTICSEARCH %}
{% if RUN_CADDY %}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: caddy
labels:
app.kubernetes.io/component: volume
app.kubernetes.io/name: caddy
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
{% endif %}
{% if RUN_ELASTICSEARCH %}
---
apiVersion: v1
kind: PersistentVolumeClaim
@ -14,7 +30,7 @@ spec:
requests:
storage: 2Gi
{% endif %}
{% if ACTIVATE_MONGODB %}
{% if RUN_MONGODB %}
---
apiVersion: v1
kind: PersistentVolumeClaim
@ -30,7 +46,7 @@ spec:
requests:
storage: 5Gi
{% endif %}
{% if ACTIVATE_MYSQL %}
{% if RUN_MYSQL %}
---
apiVersion: v1
kind: PersistentVolumeClaim
@ -46,15 +62,15 @@ spec:
requests:
storage: 5Gi
{% endif %}
{% if ACTIVATE_RABBITMQ %}
{% if RUN_REDIS %}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: rabbitmq
name: redis
labels:
app.kubernetes.io/component: volume
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/name: redis
spec:
accessModes:
- ReadWriteOnce

View File

@ -4,7 +4,6 @@ kind: Kustomization
resources:
- k8s/namespace.yml
- k8s/deployments.yml
- k8s/ingress.yml
- k8s/jobs.yml
- k8s/services.yml
- k8s/volumes.yml
@ -23,6 +22,9 @@ commonLabels:
{{ patch("kustomization-commonlabels")|indent(2) }}
configMapGenerator:
- name: caddy-config
files:
- apps/caddy/Caddyfile
- name: openedx-settings-lms
files:{% for file in "apps/openedx/settings/lms"|walk_templates %}
- {{ file }}{% endfor %}
@ -35,6 +37,9 @@ configMapGenerator:
- name: nginx-config
files:{% for file in "apps/nginx"|walk_templates %}
- {{ file }}{% endfor %}
- name: redis-config
files:
- apps/redis/redis.conf
{{ patch("kustomization-configmapgenerator") }}
{{ patch("kustomization") }}

View File

@ -5,7 +5,7 @@ services:
image: {{ DOCKER_IMAGE_MYSQL }}
entrypoint: []
command: ["echo", "done"]
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
depends_on: {{ [("mysql", RUN_MYSQL)]|list_if }}
lms-job:
image: {{ DOCKER_IMAGE_OPENEDX }}
@ -16,7 +16,7 @@ services:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro
- ../apps/openedx/settings/cms/:/openedx/edx-platform/cms/envs/tutor/:ro
- ../apps/openedx/config/:/openedx/config/:ro
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
depends_on: {{ [("mysql", RUN_MYSQL)]|list_if }}
cms-job:
image: {{ DOCKER_IMAGE_OPENEDX }}
@ -27,7 +27,7 @@ services:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro
- ../apps/openedx/settings/cms/:/openedx/edx-platform/cms/envs/tutor/:ro
- ../apps/openedx/config/:/openedx/config/:ro
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
depends_on: {{ [("mysql", RUN_MYSQL)]|list_if }}
forum-job:
image: {{ DOCKER_IMAGE_FORUM }}
@ -36,6 +36,12 @@ services:
MONGODB_AUTH: "{% if MONGODB_USERNAME and MONGODB_PASSWORD %}{{ MONGODB_USERNAME}}:{{ MONGODB_PASSWORD }}@{% endif %}"
MONGODB_HOST: "{{ MONGODB_HOST }}"
MONGODB_PORT: "{{ MONGODB_PORT }}"
depends_on: {{ [("elasticsearch", ACTIVATE_ELASTICSEARCH), ("mongodb", ACTIVATE_MONGODB)]|list_if }}
depends_on: {{ [("elasticsearch", RUN_ELASTICSEARCH), ("mongodb", RUN_MONGODB)]|list_if }}
android-job:
image: {{ DOCKER_IMAGE_ANDROID }}
volumes:
- "../android/:/openedx/config/"
- "../data/android/:/openedx/data/"
{{ patch("local-docker-compose-jobs-services")|indent(4) }}

View File

@ -1,23 +1,36 @@
version: "3.7"
services:
{% if RUN_CADDY %}
# Web proxy for SSL termination
caddy:
image: {{ DOCKER_IMAGE_CADDY }}
restart: unless-stopped
ports:
- "80:80"
{% if ENABLE_HTTPS %}- "443:443"{% endif %}
volumes:
- ../apps/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
{% if ENABLE_HTTPS %}- ../../data/caddy:/data{% endif %}
{% endif %}
# Web server
nginx:
image: {{ DOCKER_IMAGE_NGINX }}
restart: unless-stopped
{% if not RUN_CADDY %}
ports:
- "{{ NGINX_HTTP_PORT }}:80"
- "{{ NGINX_HTTPS_PORT }}:443"
{% if not WEB_PROXY %}
{% endif %}
{% if RUN_CADDY and not ENABLE_HTTPS %}
networks:
default:
# These aliases are for internal communication between containers when running locally with *.local.overhang.io hostnames.
aliases:
- "{{ LMS_HOST }}"
{{ patch("local-docker-compose-nginx-aliases")|indent(10) }}
{% endif %}
volumes:
- ../apps/nginx:/etc/nginx/conf.d/:ro
- ../../data/openedx-media:/var/www/openedx-media:ro
{% if ACTIVATE_HTTPS %}- ../../data/letsencrypt:/etc/letsencrypt/:ro{% endif %}
{{ patch("local-docker-compose-nginx-volumes")|indent(6) }}
depends_on: {{ [("lms", ACTIVATE_LMS), ("cms", ACTIVATE_CMS)]|list_if }}
depends_on: {{ [("lms", RUN_LMS), ("cms", RUN_CMS)]|list_if }}
{{ patch("local-docker-compose-prod-services")|indent(2) }}

View File

@ -3,13 +3,7 @@ services:
############# External services
{% if ACTIVATE_MEMCACHED %}
memcached:
image: {{ DOCKER_IMAGE_MEMCACHED }}
restart: unless-stopped
{% endif %}
{% if ACTIVATE_MONGODB %}
{% if RUN_MONGODB %}
mongodb:
image: {{ DOCKER_IMAGE_MONGODB }}
# Use WiredTiger in all environments, just like at edx.org
@ -19,7 +13,7 @@ services:
- ../../data/mongodb:/data/db
{% endif %}
{% if ACTIVATE_MYSQL %}
{% if RUN_MYSQL %}
mysql:
image: {{ DOCKER_IMAGE_MYSQL }}
command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci
@ -30,7 +24,7 @@ services:
MYSQL_ROOT_PASSWORD: "{{ MYSQL_ROOT_PASSWORD }}"
{% endif %}
{% if ACTIVATE_ELASTICSEARCH %}
{% if RUN_ELASTICSEARCH %}
elasticsearch:
image: {{ DOCKER_IMAGE_ELASTICSEARCH }}
command: ["elasticsearch", "-Xms{{ ELASTICSEARCH_HEAP_SIZE }}", "-Xmx{{ ELASTICSEARCH_HEAP_SIZE }}", "--cluster.name=openedx", "--bootstrap.mlockall=true"]
@ -43,15 +37,17 @@ services:
- ../../data/elasticsearch:/usr/share/elasticsearch/data
{% endif %}
{% if ACTIVATE_RABBITMQ %}
rabbitmq:
image: {{ DOCKER_IMAGE_RABBITMQ }}
{% if RUN_REDIS %}
redis:
image: {{ DOCKER_IMAGE_REDIS }}
volumes:
- ../../data/rabbitmq:/var/lib/rabbitmq
- ../../env/redis/redis.conf:/openedx/redis/config/redis.conf:ro
- ../../data/redis:/openedx/redis/data
command: redis-server /openedx/redis/config/redis.conf
restart: unless-stopped
{% endif %}
{% if ACTIVATE_SMTP %}
{% if RUN_SMTP %}
smtp:
image: {{ DOCKER_IMAGE_SMTP }}
restart: unless-stopped
@ -59,7 +55,7 @@ services:
############# Forum
{% if ACTIVATE_FORUM %}
{% if RUN_FORUM %}
forum:
image: {{ DOCKER_IMAGE_FORUM }}
environment:
@ -68,17 +64,17 @@ services:
MONGODB_HOST: "{{ MONGODB_HOST }}"
MONGODB_PORT: "{{ MONGODB_PORT }}"
restart: unless-stopped
depends_on: {{ [("elasticsearch", ACTIVATE_ELASTICSEARCH), ("mongodb", ACTIVATE_MONGODB)]|list_if }}
depends_on: {{ [("elasticsearch", RUN_ELASTICSEARCH), ("mongodb", RUN_MONGODB)]|list_if }}
{% endif %}
############# LMS and CMS
{% if ACTIVATE_LMS %}
{% if RUN_LMS %}
lms:
image: {{ DOCKER_IMAGE_OPENEDX }}
environment:
SERVICE_VARIANT: lms
GUNICORN_WORKERS: {{ OPENEDX_LMS_GUNICORN_WORKERS }}
UWSGI_WORKERS: {{ OPENEDX_LMS_UWSGI_WORKERS }}
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
restart: unless-stopped
volumes:
@ -88,22 +84,21 @@ services:
- ../../data/lms:/openedx/data
- ../../data/openedx-media:/openedx/media
depends_on:
{% if ACTIVATE_MYSQL %}- mysql{% endif %}
{% if ACTIVATE_ELASTICSEARCH %}- elasticsearch{% endif %}
{% if ACTIVATE_FORUM %}- forum{% endif %}
{% if ACTIVATE_MEMCACHED %}- memcached{% endif %}
{% if ACTIVATE_MONGODB %}- mongodb{% endif %}
{% if ACTIVATE_RABBITMQ %}- rabbitmq{% endif %}
{% if ACTIVATE_SMTP %}- smtp{% endif %}
{% if RUN_MYSQL %}- mysql{% endif %}
{% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %}
{% if RUN_FORUM %}- forum{% endif %}
{% if RUN_MONGODB %}- mongodb{% endif %}
{% if RUN_REDIS %}- redis{% endif %}
{% if RUN_SMTP %}- smtp{% endif %}
{{ patch("local-docker-compose-lms-dependencies")|indent(6) }}
{% endif %}
{% if ACTIVATE_CMS %}
{% if RUN_CMS %}
cms:
image: {{ DOCKER_IMAGE_OPENEDX }}
environment:
SERVICE_VARIANT: cms
GUNICORN_WORKERS: {{ OPENEDX_CMS_GUNICORN_WORKERS }}
UWSGI_WORKERS: {{ OPENEDX_CMS_UWSGI_WORKERS }}
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
restart: unless-stopped
volumes:
@ -113,26 +108,25 @@ services:
- ../../data/cms:/openedx/data
- ../../data/openedx-media:/openedx/media
depends_on:
{% if ACTIVATE_MYSQL %}- mysql{% endif %}
{% if ACTIVATE_ELASTICSEARCH %}- elasticsearch{% endif %}
{% if ACTIVATE_MEMCACHED %}- memcached{% endif %}
{% if ACTIVATE_MONGODB %}- mongodb{% endif %}
{% if ACTIVATE_RABBITMQ %}- rabbitmq{% endif %}
{% if ACTIVATE_SMTP %}- smtp{% endif %}
{% if ACTIVATE_LMS %}- lms{% endif %}
{% if RUN_MYSQL %}- mysql{% endif %}
{% if RUN_ELASTICSEARCH %}- elasticsearch{% endif %}
{% if RUN_MONGODB %}- mongodb{% endif %}
{% if RUN_REDIS %}- redis{% endif %}
{% if RUN_SMTP %}- smtp{% endif %}
{% if RUN_LMS %}- lms{% endif %}
{{ patch("local-docker-compose-cms-dependencies")|indent(6) }}
{% endif %}
############# LMS and CMS workers
{% if ACTIVATE_LMS %}
{% if RUN_LMS %}
lms-worker:
image: {{ DOCKER_IMAGE_OPENEDX }}
environment:
SERVICE_VARIANT: lms
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
C_FORCE_ROOT: "1" # run celery tasks as root #nofear
command: ./manage.py lms celery worker --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.cms.core.default
command: celery worker --app=cms.celery --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.cms.core.default
restart: unless-stopped
volumes:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro
@ -144,14 +138,14 @@ services:
- lms
{% endif %}
{% if ACTIVATE_CMS %}
{% if RUN_CMS %}
cms-worker:
image: {{ DOCKER_IMAGE_OPENEDX }}
environment:
SERVICE_VARIANT: cms
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
C_FORCE_ROOT: "1" # run celery tasks as root #nofear
command: ./manage.py cms celery worker --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.lms.core.default
command: celery worker --app=cms.celery --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.lms.core.default
restart: unless-stopped
volumes:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro

View File

@ -1,39 +0,0 @@
{% if ACTIVATE_HTTPS %}
<VirtualHost *:80>
ServerName {{ LMS_HOST }}
Redirect / https://{{ LMS_HOST }}/
</VirtualHost>
<VirtualHost *:80>
ServerName preview.{{ LMS_HOST }}
Redirect / https://preview.{{ LMS_HOST }}/
</VirtualHost>
<VirtualHost *:80>
ServerName {{ CMS_HOST }}
Redirect / https://{{ CMS_HOST }}/
</VirtualHost>
<VirtualHost *:443>
ServerName {{ LMS_HOST }}
ServerAlias preview.{{ LMS_HOST }} {{ CMS_HOST }}
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/{{ LMS_HOST }}/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/{{ LMS_HOST }}/privkey.pem
ProxyPreserveHost On
ProxyRequests On
ProxyPass / http://localhost:{{ NGINX_HTTPS_PORT }}/
ProxyPassReverse / http://localhost:{{ NGINX_HTTPS_PORT }}/
</VirtualHost>
{% else %}
<VirtualHost *:80>
ServerName {{ LMS_HOST }}
ServerAlias preview.{{ LMS_HOST }} {{ CMS_HOST }}
ProxyPreserveHost On
ProxyRequests On
ProxyPass / http://localhost:{{ NGINX_HTTP_PORT }}/
ProxyPassReverse / http://localhost:{{ NGINX_HTTP_PORT }}/
</VirtualHost>
{% endif %}
{{ patch("proxy-apache") }}

View File

@ -1,36 +0,0 @@
server {
listen 80;
server_name {{ LMS_HOST }} preview.{{ LMS_HOST }} {{ CMS_HOST }};
server_tokens off;
location / {
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://localhost:{{ NGINX_HTTP_PORT }};
}
}
{% if ACTIVATE_HTTPS %}
server {
listen 443 ssl;
server_name {{ LMS_HOST }} preview.{{ LMS_HOST }} {{ CMS_HOST }};
ssl_certificate /etc/letsencrypt/live/{{ LMS_HOST }}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{ LMS_HOST }}/privkey.pem;
server_tokens off;
location / {
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://localhost:{{ NGINX_HTTPS_PORT }};
}
}
{% endif %}
{{ patch("proxy-nginx") }}