docs: reorganize local guides in tutorials

There is too much information in each of the local/k8s/dev docs pages. The
"guides" that are listed in each one of those pages are moved either to "common
tasks" or to a dedicated "tutorials" section. This paves the way for more
comprehensive tutorials, where we describe how to run the latest master
branches of Open edX.

I am well aware that, as they stand, the tutorials are of poor quality and
should be rewritten. This is a task for another day/commit. For now, we only
move the contents to a separate part of the docs.

Also, we should add a "reference" section to the docs, where we add the result
of `tutor <subcommand> --help`.
This commit is contained in:
Régis Behmo 2021-09-16 10:53:52 +02:00 committed by Régis Behmo
parent df6a1c3b4e
commit e14f660cb1
16 changed files with 216 additions and 194 deletions

View File

@ -230,28 +230,7 @@ These arguments can be specified from the command line, `very much like Docker <
Adding custom themes
~~~~~~~~~~~~~~~~~~~~
Comprehensive theming is enabled by default, but only the default theme is compiled. `Indigo <https://github.com/overhangio/indigo>`__ is a better, ready-to-run theme which you can start using today.
To compile your own theme, add it to the ``env/build/openedx/themes/`` folder::
git clone https://github.com/me/myopenedxtheme.git "$(tutor config printroot)/env/build/openedx/themes/myopenedxtheme"
The ``themes`` folder should have the following structure::
openedx/themes/
mycustomtheme1/
cms/
...
lms/
...
mycustomtheme2/
...
Then you must rebuild the openedx Docker image::
tutor images build openedx
Finally, you should enable your theme with the :ref:`settheme command <settheme>`.
See :ref:`the corresponding tutorial <theming>`.
.. _custom_extra_xblocks:

View File

@ -58,8 +58,8 @@ To collect assets, you can use the ``openedx-assets`` command that ships with Tu
.. _bind_mounts:
Bind-mount container directories
--------------------------------
Sharing directories with containers
-----------------------------------
It may sometimes be convenient to mount container directories on the host, for instance: for editing and debugging. Tutor provides different solutions to this problem.
@ -117,8 +117,11 @@ This override file will be loaded when running any ``tutor dev ..`` command. The
.. note::
The ``tutor local`` commands loads the ``docker-compose.override.yml`` file from the ``$(tutor config printroot)/env/local/docker-compose.override.yml`` directory.
Point to a local edx-platform
-----------------------------
Common tasks
------------
Setting up a development environment for edx-platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Following the instructions :ref:`above <bind_mounts>` on how to bind-mount directories from the host above, you may mount your own `edx-platform <https://github.com/edx/edx-platform/>`__ fork in your containers by running either::
@ -132,9 +135,6 @@ Following the instructions :ref:`above <bind_mounts>` on how to bind-mount direc
# Add your own volumes to $(tutor config printroot)/env/dev/docker-compose.override.yml
tutor dev runserver lms
Prepare the edx-platform repo
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you choose any but the first solution above, you will have to make sure that your fork works with Tutor.
First of all, you should make sure that you are working off the ``open-release/lilac.2`` tag. See the :ref:`fork edx-platform section <edx_platform_fork>` for more information.
@ -153,16 +153,12 @@ Then, you should run the following commands::
# Rebuild static assets
openedx-assets build --env=dev
Debug edx-platform
~~~~~~~~~~~~~~~~~~
To debug a local edx-platform repository, add a ``import ipdb; ipdb.set_trace()`` breakpoint anywhere in your code and run::
tutor dev runserver [--volume=...] lms
XBlock and edx-platform plugin development
------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In some cases you will have to develop features for packages that are pip-installed next to edx-platform. This is quite easy with Tutor. Just add your packages to the ``$(tutor config printroot)/env/build/openedx/requirements/private.txt`` file. To avoid re-building the openedx Docker image at every change, you should add your package in editable mode. For instance::
@ -182,38 +178,8 @@ You will have to re-build the openedx Docker image once::
You should then run the development server as usual, with ``runserver``. Every change made to the ``mypackage`` folder will be picked up and the development server will be automatically reloaded.
.. _theming:
Customised themes
-----------------
With Tutor, it's pretty easy to develop your own themes. Start by placing your files inside the ``env/build/openedx/themes`` directory. For instance, you could start from the ``edx.org`` theme present inside the ``edx-platform`` repository::
cp -r /path/to/edx-platform/themes/edx.org "$(tutor config printroot)/env/build/openedx/themes/"
.. warning::
You should not create a soft link here. If you do, it will trigger a ``Theme not found in any of the themes dirs`` error. This is because soft links are not properly resolved from inside docker containers.
Then, run a local webserver::
tutor dev runserver lms
The LMS can then be accessed at http://local.overhang.io:8000. You will then have to :ref:`enable that theme <settheme>` for the development domain names::
tutor dev settheme mythemename local.overhang.io:8000 studio.local.overhang.io:8001
Re-build development docker image (and compile assets)::
tutor images build openedx-dev
Watch the themes folders for changes (in a different terminal)::
tutor dev run watchthemes
Make changes to some of the files inside the theme directory: the theme assets should be automatically recompiled and visible at http://local.overhang.io:8000.
Custom edx-platform settings
----------------------------
Loading custom edx-platform settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By default, tutor settings files are mounted inside the docker images at ``/openedx/edx-platform/lms/envs/tutor/`` and ``/openedx/edx-platform/cms/envs/tutor/``. In the various ``dev`` commands, the default ``edx-platform`` settings module is set to ``tutor.development`` and you don't have to do anything to set up these settings.
@ -238,7 +204,7 @@ From then on, all ``dev`` commands will use the ``mysettings`` module. For insta
tutor dev runserver lms
Running edx-platform unit tests
-------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to run the full set of unit tests that ship with `edx-platform <https://github.com/edx/edx-platform/>`__. To do so, run a shell in the LMS development container::

BIN
docs/img/portainer.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 215 KiB

View File

@ -23,6 +23,7 @@
run
configuration
plugins
tutorials
troubleshooting
tutor
faq

View File

@ -17,7 +17,7 @@ Requirements
.. warning::
Do not attempt to simply run ``apt-get install docker docker-compose`` on older Ubuntu platforms, such as 16.04 (Xenial), as you will get older versions of these utilities.
* Ports 80 and 443 should be open. If other web services run on these ports, check the section on :ref:`how to setup a web proxy <web_proxy>`.
* Ports 80 and 443 should be open. If other web services run on these ports, check the tutorial on :ref:`how to setup a web proxy <web_proxy>`.
* Hardware:
- Minimum configuration: 4 GB RAM, 2 CPU, 8 GB disk space

View File

@ -97,8 +97,8 @@ In particular, the `tutor k8s start` command restarts and reconfigures all servi
All non-persisting data will be deleted, and then re-created.
Guides
------
Common tasks
------------
Running a custom "openedx" Docker image
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -97,8 +97,8 @@ Finally, tracking logs that store `user events <https://edx.readthedocs.io/proje
$(tutor config printroot)/data/cms/logs/tracking.log
Extra commands
--------------
Common tasks
------------
.. _createuser:
@ -150,127 +150,6 @@ After modifying Open edX settings, for instance when running ``tutor config save
tutor local exec lms reload-uwsgi
.. _portainer:
Docker container web UI with `Portainer <https://portainer.io/>`__
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Portainer is a web UI for managing docker containers. It lets you view your entire Open edX platform at a glace. Try it! It's really cool::
docker run --rm \
--volume=/var/run/docker.sock:/var/run/docker.sock \
--volume=/tmp/portainer:/data \
-p 9000:9000 \
portainer/portainer:latest --bind=:9000
.. .. image:: https://portainer.io/images/screenshots/portainer.gif
..:alt: Portainer demo
You can then view the portainer UI at `http://localhost:9000 <http://localhost:9000>`_. You will be asked to define a password for the admin user. Then, select a "Local environment" to work on; hit "Connect" and select the "local" group to view all running containers.
Among many other things, you'll be able to view the logs for each container, which is really useful.
Guides
------
.. _web_proxy:
Running Open edX behind a web proxy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The containerized web server (`Caddy <https://caddyserver.com/>`__) needs to listen to ports 80 and 443 on the host in order to serve requests. If there is already a webserver running on the host, such as Apache or Nginx, the caddy container will not be able to start. Tutor supports running behind a web proxy. To do so, add the following configuration::
tutor config save --set RUN_CADDY=false --set NGINX_HTTP_PORT=81
In this example, the nginx container port would be mapped to 81 instead of 80. You must then configure the web proxy on the host. As of v11.0.0, configuration files are no longer provided for automatic configuration of your web proxy. Basically, you should setup a reverse proxy to `localhost:NGINX_HTTP_PORT` from the following hosts: LMS_HOST, PREVIEW_LMS_HOST, CMS_HOST, as well as any additional host exposed by your plugins.
.. warning::
In this setup, the Nginx HTTP port will be exposed to the world. Make sure to configure your server firewall to block unwanted connections to your server's `NGINX_HTTP_PORT`. Alternatively, you can configure the Nginx container to accept only local connections::
tutor config save --set NGINX_HTTP_PORT=127.0.0.1:81
The same solution applies if you would like to enable https in Tutor, but with your own custom certificates instead of Let's Encrypt's. In that case, you should keep ``ENABLE_HTTPS=true``, disable Caddy (``RUN_CADDY=false``) and configure your own web proxy on the host (or elsewhere) to serve requests using your own certificates.
Running multiple Open edX platforms on a single server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With Tutor, it is easy to run multiple Open edX instances on a single server. To do so, the following configuration parameters must be different for all platforms:
- ``TUTOR_ROOT``: so that configuration, environment and data are not mixed up between platforms.
- ``LOCAL_PROJECT_NAME``: the various docker-compose projects cannot share the same name.
- ``NGINX_HTTP_PORT``: ports cannot be shared by two different containers.
- ``LMS_HOST``, ``CMS_HOST``: the different platforms must be accessible from different domain (or subdomain) names.
In addition, a web proxy must be setup on the host, as described :ref:`above <web_proxy>`.
Loading different production settings for ``edx-platform``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The default settings module loaded by ``edx-platform`` is ``tutor.production``. The folders ``$(tutor config printroot)/env/apps/openedx/settings/lms`` and ``$(tutor config printroot)/env/apps/openedx/settings/cms`` are mounted as ``edx-platform/lms/envs/tutor`` and ``edx-platform/cms/envs/tutor`` inside the docker containers. Thus, to use your own settings, you must do two things:
1. Copy your settings files for the lms and the cms to ``$(tutor config printroot)/env/apps/openedx/settings/lms/mysettings.py`` and ``$(tutor config printroot)/env/apps/openedx/settings/cms/mysettings.py``.
2. Load your settings by adding ``TUTOR_EDX_PLATFORM_SETTINGS=tutor.mysettings`` to ``$(tutor config printroot)/env/local/.env``.
Of course, your settings should be compatible with the docker installation. You can get some inspiration from the ``production.py`` settings modules generated by Tutor, or just import it as a base by adding ``from .production import *`` at the top of ``mysettings.py``.
Upgrading from earlier versions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Upgrading from v3+
******************
Just upgrade Tutor using your :ref:`favorite installation method <install>` and run quickstart again::
tutor local quickstart
Upgrading from v1 or v2
***********************
Versions 1 and 2 of Tutor were organized differently: they relied on many different ``Makefile`` and ``make`` commands instead of a single ``tutor`` executable. To migrate from an earlier version, you should first stop your platform::
make stop
Then, install Tutor using one of the :ref:`installation methods <install>`. Then, create the Tutor project root and move your data::
mkdir -p "$(tutor config printroot)"
mv config.json data/ "$(tutor config printroot)"
Finally, launch your platform with::
tutor local quickstart
Backups/Migrating to a different server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With Tutor, all data are stored in a single folder. This means that it's extremely easy to migrate an existing platform to a different server. For instance, it's possible to configure a platform locally on a laptop, and then move this platform to a production server.
1. Make sure `tutor` is installed on both servers with the same version.
2. Stop any running platform on server 1::
tutor local stop
3. Transfer the configuration, environment and platform data from server 1 to server 2::
rsync -avr "$(tutor config printroot)/" username@server2:/tmp/tutor/
4. On server 2, move the data to the right location::
mv /tmp/tutor "$(tutor config printroot)"
5. Start the instance with::
tutor local start -d
Making database dumps
~~~~~~~~~~~~~~~~~~~~~
To dump all data from the MySQL and Mongodb databases used on the platform, run the following commands::
tutor local exec -e MYSQL_ROOT_PASSWORD="$(tutor config printvalue MYSQL_ROOT_PASSWORD)" mysql \
sh -c 'mysqldump --all-databases --password=$MYSQL_ROOT_PASSWORD > /var/lib/mysql/dump.sql'
tutor local exec mongodb mongodump --out=/data/db/dump.mongodb
The ``dump.sql`` and ``dump.mongodb`` files will be located in ``$(tutor config printroot)/data/mysql`` and ``$(tutor config printroot)/data/mongodb``.
Customizing the deployed services
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

25
docs/tutorials.rst Normal file
View File

@ -0,0 +1,25 @@
Tutorials
=========
Open edX customization
----------------------
.. toctree::
:maxdepth: 2
tutorials/theming
tutorials/edx-platform-settings
System administration
---------------------
.. toctree::
:maxdepth: 2
tutorials/portainer
tutorials/proxy
tutorials/datamigration
tutorials/multiplatforms
tutorials/oldreleases
Other tutorials can be found in the official Tutor forums, `in the "Tutorials" category <https://discuss.overhang.io/c/tutor/tutorials/13>`__.

View File

@ -0,0 +1,32 @@
Making backups and migrating data
---------------------------------
With Tutor, all data are stored in a single folder. This means that it's extremely easy to migrate an existing platform to a different server. For instance, it's possible to configure a platform locally on a laptop, and then move this platform to a production server.
1. Make sure `tutor` is installed on both servers with the same version.
2. Stop any running platform on server 1::
tutor local stop
3. Transfer the configuration, environment and platform data from server 1 to server 2::
rsync -avr "$(tutor config printroot)/" username@server2:/tmp/tutor/
4. On server 2, move the data to the right location::
mv /tmp/tutor "$(tutor config printroot)"
5. Start the instance with::
tutor local start -d
Making database dumps
---------------------
To dump all data from the MySQL and Mongodb databases used on the platform, run the following commands::
tutor local exec -e MYSQL_ROOT_PASSWORD="$(tutor config printvalue MYSQL_ROOT_PASSWORD)" mysql \
sh -c 'mysqldump --all-databases --password=$MYSQL_ROOT_PASSWORD > /var/lib/mysql/dump.sql'
tutor local exec mongodb mongodump --out=/data/db/dump.mongodb
The ``dump.sql`` and ``dump.mongodb`` files will be located in ``$(tutor config printroot)/data/mysql`` and ``$(tutor config printroot)/data/mongodb``.

View File

@ -0,0 +1,9 @@
Modifying ``edx-platform`` settings
-----------------------------------
The default settings module loaded by ``edx-platform`` is ``tutor.production`` in production and ``tutor.development`` in development. The folders ``$(tutor config printroot)/env/apps/openedx/settings/lms`` and ``$(tutor config printroot)/env/apps/openedx/settings/cms`` are mounted as ``edx-platform/lms/envs/tutor`` and ``edx-platform/cms/envs/tutor`` inside the docker containers. Thus, to use your own settings, you must do two things:
1. Copy your settings files for the lms and the cms to ``$(tutor config printroot)/env/apps/openedx/settings/lms/mysettings.py`` and ``$(tutor config printroot)/env/apps/openedx/settings/cms/mysettings.py``.
2. Load your settings by adding ``TUTOR_EDX_PLATFORM_SETTINGS=tutor.mysettings`` to ``$(tutor config printroot)/env/local/.env``.
Of course, your settings should be compatible with the docker installation. You can get some inspiration from the ``production.py`` settings modules generated by Tutor, or just import it as a base by adding ``from .production import *`` at the top of ``mysettings.py``.

View File

@ -0,0 +1,11 @@
Running multiple Open edX platforms on a single server
======================================================
With Tutor, it is easy to run multiple Open edX instances on a single server. To do so, the following configuration parameters must be different for all platforms:
- ``TUTOR_ROOT``: so that configuration, environment and data are not mixed up between platforms.
- ``LOCAL_PROJECT_NAME``: the various docker-compose projects cannot share the same name.
- ``NGINX_HTTP_PORT``: ports cannot be shared by two different containers.
- ``LMS_HOST``, ``CMS_HOST``: the different platforms must be accessible from different domain (or subdomain) names.
In addition, a web proxy must be setup on the host, as described :ref:`in the corresponding tutorial <web_proxy>`.

View File

@ -0,0 +1,25 @@
Upgrading from older releases
-----------------------------
Upgrading from v3+
~~~~~~~~~~~~~~~~~~
Just upgrade Tutor using your :ref:`favorite installation method <install>` and run quickstart again::
tutor local quickstart
Upgrading from v1 or v2
~~~~~~~~~~~~~~~~~~~~~~~
Versions 1 and 2 of Tutor were organized differently: they relied on many different ``Makefile`` and ``make`` commands instead of a single ``tutor`` executable. To migrate from an earlier version, you should first stop your platform::
make stop
Then, install Tutor using one of the :ref:`installation methods <install>`. Then, create the Tutor project root and move your data::
mkdir -p "$(tutor config printroot)"
mv config.json data/ "$(tutor config printroot)"
Finally, launch your platform with::
tutor local quickstart

View File

@ -0,0 +1,19 @@
.. _portainer:
Running a Docker container web UI with `Portainer <https://portainer.io/>`__
----------------------------------------------------------------------------
Portainer is a web UI for managing docker containers. It lets you view your entire Open edX platform at a glace. Try it! It's really cool::
docker run --rm \
--volume=/var/run/docker.sock:/var/run/docker.sock \
--volume=/tmp/portainer:/data \
-p 9000:9000 \
portainer/portainer:latest --bind=:9000
You can then view the portainer UI at `http://localhost:9000 <http://localhost:9000>`_. You will be asked to define a password for the admin user. Then, select a "Local environment" to work on; hit "Connect" and select the "local" group to view all running containers.
.. image:: ../img/portainer.png
:alt: Portainer demo
Among many other things, you'll be able to view the logs for each container, which is really useful.

15
docs/tutorials/proxy.rst Normal file
View File

@ -0,0 +1,15 @@
.. _web_proxy:
Running Open edX behind a web proxy
===================================
The containerized web server (`Caddy <https://caddyserver.com/>`__) needs to listen to ports 80 and 443 on the host. If there is already a webserver running on the host, such as Apache or Nginx, the caddy container will not be able to start. Tutor supports running behind a web proxy. To do so, add the following configuration::
tutor config save --set RUN_CADDY=false --set NGINX_HTTP_PORT=81
In this example, the nginx container port would be mapped to 81 instead of 80. You must then configure the web proxy on the host. As of v11.0.0, configuration files are no longer provided for automatic configuration of your web proxy. Basically, you should setup a reverse proxy to `localhost:NGINX_HTTP_PORT` from the following hosts: LMS_HOST, PREVIEW_LMS_HOST, CMS_HOST, as well as any additional host exposed by your plugins.
.. warning::
In this setup, the Nginx HTTP port will be exposed to the world. Make sure to configure your server firewall to block unwanted connections to your server's ``NGINX_HTTP_PORT``. Alternatively, you can configure the Nginx container to accept only local connections::
tutor config save --set NGINX_HTTP_PORT=127.0.0.1:81

View File

@ -0,0 +1,61 @@
.. _theming:
Changing the appearance of Open edX
===================================
Installing a custom theme
-------------------------
Comprehensive theming is enabled by default, but only the default theme is compiled. `Indigo <https://github.com/overhangio/indigo>`__ is a better, ready-to-run theme which you can start using today.
To compile your own theme, add it to the ``env/build/openedx/themes/`` folder::
git clone https://github.com/me/myopenedxtheme.git \
"$(tutor config printroot)/env/build/openedx/themes/myopenedxtheme"
The ``themes`` folder should have the following structure::
openedx/themes/
mycustomtheme1/
cms/
...
lms/
...
mycustomtheme2/
...
Then you must rebuild the openedx Docker image::
tutor images build openedx
Finally, you should enable your theme with the :ref:`settheme command <settheme>`.
.. _theme_development:
Developing a new theme
----------------------
With Tutor, it's pretty easy to develop your own themes. Start by placing your files inside the ``env/build/openedx/themes`` directory. For instance, you could start from the ``edx.org`` theme present inside the ``edx-platform`` repository::
cp -r /path/to/edx-platform/themes/edx.org "$(tutor config printroot)/env/build/openedx/themes/"
.. warning::
You should not create a soft link here. If you do, it will trigger a ``Theme not found in any of the themes dirs`` error. This is because soft links are not properly resolved from inside docker containers.
Then, run a local webserver::
tutor dev runserver lms
The LMS can then be accessed at http://local.overhang.io:8000. You will then have to :ref:`enable that theme <settheme>`::
tutor dev settheme mythemename
Re-build development docker image (and compile assets)::
tutor images build openedx-dev
Watch the themes folders for changes (in a different terminal)::
tutor dev run watchthemes
Make changes to some of the files inside the theme directory: the theme assets should be automatically recompiled and visible at http://local.overhang.io:8000.

View File

@ -18,7 +18,7 @@ To get a glimpse of the possibilities of Open edX, we recommend you import the `
Making Open edX look better
---------------------------
Tutor makes it easy to :ref:`develop <theming>` and :ref:`install <settheme>` your own themes. We also provide `Indigo <https://github.com/overhangio/indigo>`__: a free, customizable theme that you can install today.
Tutor makes it easy to :ref:`install <theming>` and :ref:`develop <theme_development>` your own themes. We also provide `Indigo <https://github.com/overhangio/indigo>`__: a free, customizable theme that you can install today.
Adding features
---------------