mirror of
https://github.com/ChristianLight/tutor.git
synced 2024-12-12 14:17:46 +00:00
Merge remote-tracking branch 'origin/master' into nightly
This commit is contained in:
Overhang.IO
commit
ea1ed85694
1
changelog.d/20230313_110919_kdmc_mounted_init.md
Normal file
1
changelog.d/20230313_110919_kdmc_mounted_init.md
Normal file
@ -0,0 +1 @@
|
||||
- [Improvement] Running ``tutor dev launch --mount=edx-platform`` now performs all necessary setup for a local edx-platform development. This includes running setup.py, installing node modules, and building assets; previously, those steps had to be run explicitly after bind-mounting a local copy of edx-platform (by @kdmccormick).
|
||||
106
docs/dev.rst
106
docs/dev.rst
@ -5,23 +5,29 @@ Open edX development
|
||||
|
||||
In addition to running Open edX in production, Tutor can be used for local development of Open edX. This means that it is possible to hack on Open edX without setting up a Virtual Machine. Essentially, this replaces the devstack provided by edX.
|
||||
|
||||
.. _edx_platform_dev_env:
|
||||
|
||||
First-time setup
|
||||
----------------
|
||||
|
||||
First, ensure you have already :ref:`installed Tutor <install>` (for development against the named releases of Open edX) or :ref:`Tutor Nightly <nightly>` (for development against Open edX's master branches).
|
||||
Firstly, either :ref:`install Tutor <install>` (for development against the named releases of Open edX) or :ref:`install Tutor Nightly <nightly>` (for development against Open edX's master branches).
|
||||
|
||||
Then, launch the developer platform setup process::
|
||||
Then, run one of the following in order to launch the developer platform setup process::
|
||||
|
||||
# To use the edx-platform repository that is built into the image, run:
|
||||
tutor dev launch
|
||||
|
||||
This will perform several tasks for you. It will:
|
||||
# To bind-mount and run a local clone of edx-platform, replace
|
||||
# './edx-platform' with the path to the local clone and run:
|
||||
tutor dev launch --mount=./edx-platform
|
||||
|
||||
This will perform several tasks. It will:
|
||||
|
||||
* stop any existing locally-running Tutor containers,
|
||||
|
||||
* disable HTTPS,
|
||||
|
||||
* set your ``LMS_HOST`` to `local.overhang.io <http://local.overhang.io>`_ (a convenience domain that simply `points at 127.0.0.1 <https://dnschecker.org/#A/local.overhang.io>`_),
|
||||
* set ``LMS_HOST`` to `local.overhang.io <http://local.overhang.io>`_ (a convenience domain that simply `points at 127.0.0.1 <https://dnschecker.org/#A/local.overhang.io>`_),
|
||||
|
||||
* prompt for a platform details (with suitable defaults),
|
||||
|
||||
@ -33,52 +39,82 @@ This will perform several tasks for you. It will:
|
||||
|
||||
* run service initialization scripts, such as service user creation and Waffle configuration.
|
||||
|
||||
Additionally, when a local clone of edx-platform is bind-mounted, it will:
|
||||
|
||||
* re-run setup.py,
|
||||
* clean-reinstall Node modules, and
|
||||
* regenerate static assets.
|
||||
|
||||
Once setup is complete, the platform will be running in the background:
|
||||
|
||||
* LMS will be accessible at `http://local.overhang.io:8000 <http://local.overhang.io:8000>`_.
|
||||
* CMS will be accessible at `http://studio.local.overhang.io:8001 <http://studio.local.overhang.io:8001>`_.
|
||||
* Plugged-in services should be accessible at their documented URLs.
|
||||
|
||||
Now, use the ``tutor dev ...`` command-line interface to manage the development environment. Some common commands are described below.
|
||||
|
||||
.. note::
|
||||
|
||||
Wherever the ``[--mount=./edx-platform]`` option is present, either:
|
||||
|
||||
* omit it when running of the edx-platform repository built into the image, or
|
||||
* substitute it with ``--mount=<path/to/edx-platform>``.
|
||||
|
||||
Read more about bind-mounts :ref:`below <bind_mounts>`.
|
||||
|
||||
Stopping the platform
|
||||
---------------------
|
||||
|
||||
To bring down your platform's containers, simply run::
|
||||
To bring down the platform's containers, simply run::
|
||||
|
||||
tutor dev stop
|
||||
|
||||
|
||||
Starting the platform back up
|
||||
-----------------------------
|
||||
|
||||
Once you have used ``launch`` once, you can start the platform in the future with the lighter-weight ``start`` command, which brings up containers but does not perform any initialization tasks::
|
||||
Once first-time setup has been performed with ``launch``, the platform can be started going forward with the lighter-weight ``start -d`` command, which brings up containers *detached* (that is: in the background), but does not perform any initialization tasks::
|
||||
|
||||
tutor dev start # Run platform in the same terminal ("attached")
|
||||
tutor dev start -d # Or, run platform the in the background ("detached")
|
||||
tutor dev start -d [--mount=./edx-platform]
|
||||
|
||||
Nonetheless, ``launch`` is idempotent, so it is always safe to run it again in the future without risk to your data. In fact, you may find it useful to use this command as a one-stop-shop for pulling images, running migrations, initializing new plugins you have enabled, and/or executing any new initialization steps that may have been introduced since you set up Tutor::
|
||||
Or, to start with platform with containers *attached* (that is: in the foreground, the current terminal), omit the ``-d`` flag::
|
||||
|
||||
tutor dev launch --pullimages
|
||||
tutor dev start [--mount=./edx-platform]
|
||||
|
||||
When running containers attached, stop the platform with ``Ctrl+c``, or switch to detached mode using ``Ctrl+z``.
|
||||
|
||||
Finally, the platform can also be started back up with ``launch``. It will take longer than ``start``, but it will ensure that config is applied, databases are provisioned & migrated, plugins are fully initialized, and (if applicable) the bind-mounted edx-platform is set up. Notably, ``launch`` is idempotent, so it is always safe to run it again without risk to data. Including the ``--pullimages`` flag will also ensure that container images are up-to-date::
|
||||
|
||||
tutor dev launch [--mount=./edx-platform] --pullimages
|
||||
|
||||
Debugging with breakpoints
|
||||
--------------------------
|
||||
|
||||
To debug a local edx-platform repository, add a `python breakpoint <https://docs.python.org/3/library/functions.html#breakpoint>`__ with ``breakpoint()`` anywhere in the code. Then, attach to the applicable service's container by running ``start`` (without ``-d``) followed by the service's name::
|
||||
|
||||
# Debugging LMS:
|
||||
tutor dev start [--mount=./edx-platform] lms
|
||||
|
||||
# Or, debugging CMS:
|
||||
tutor dev start [--mount=./edx-platform] cms
|
||||
|
||||
Running arbitrary commands
|
||||
--------------------------
|
||||
|
||||
To run any command inside one of the containers, run ``tutor dev run [OPTIONS] SERVICE [COMMAND] [ARGS]...``. For instance, to open a bash shell in the LMS or CMS containers::
|
||||
|
||||
tutor dev run lms bash
|
||||
tutor dev run cms bash
|
||||
tutor dev run [--mount=./edx-platform] lms bash
|
||||
tutor dev run [--mount=./edx-platform] cms bash
|
||||
|
||||
To open a python shell in the LMS or CMS, run::
|
||||
|
||||
tutor dev run lms ./manage.py lms shell
|
||||
tutor dev run cms ./manage.py cms shell
|
||||
tutor dev run [--mount=./edx-platform] lms ./manage.py lms shell
|
||||
tutor dev run [--mount=./edx-platform] cms ./manage.py cms shell
|
||||
|
||||
You can then import edx-platform and django modules and execute python code.
|
||||
|
||||
To collect assets, you can use the ``openedx-assets`` command that ships with Tutor::
|
||||
To rebuild assets, you can use the ``openedx-assets`` command that ships with Tutor::
|
||||
|
||||
tutor dev run lms openedx-assets build --env=dev
|
||||
tutor dev run [--mount=./edx-platform] lms openedx-assets build --env=dev
|
||||
|
||||
|
||||
.. _specialized for developer usage:
|
||||
@ -193,42 +229,6 @@ This override file will be loaded when running any ``tutor dev ..`` command. The
|
||||
Common tasks
|
||||
------------
|
||||
|
||||
.. _edx_platform_dev_env:
|
||||
|
||||
Setting up a development environment for edx-platform
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Following the instructions :ref:`above <bind_mounts>` on how to bind-mount directories from the host above, you may mount your own `edx-platform <https://github.com/openedx/edx-platform/>`__ fork in your containers by running::
|
||||
|
||||
tutor dev start -d --mount=/path/to/edx-platform lms
|
||||
|
||||
But to achieve that, you will have to make sure that your fork works with Tutor.
|
||||
|
||||
First of all, you should make sure that you are working off the latest release tag (unless you are running the Tutor :ref:`nightly <nightly>` branch). See the :ref:`fork edx-platform section <edx_platform_fork>` for more information.
|
||||
|
||||
Then, you should run the following commands::
|
||||
|
||||
# Run bash in the lms container
|
||||
tutor dev run --mount=/path/to/edx-platform lms bash
|
||||
|
||||
# Compile local python requirements
|
||||
pip install --requirement requirements/edx/development.txt
|
||||
|
||||
# Install nodejs packages in node_modules/
|
||||
npm clean-install
|
||||
|
||||
# Rebuild static assets
|
||||
openedx-assets build --env=dev
|
||||
|
||||
After running all these commands, your edx-platform repository will be ready for local development. To debug a local edx-platform repository, you can then add a `python breakpoint <https://docs.python.org/3/library/functions.html#breakpoint>`__ with ``breakpoint()`` anywhere in your code and run::
|
||||
|
||||
tutor dev start --mount=/path/to/edx-platform lms
|
||||
|
||||
The default debugger is ``ipdb.set_trace``. ``PYTHONBREAKPOINT`` can be modified by setting an environment variable in the Docker imamge.
|
||||
|
||||
If LMS isn't running, this will start it in your terminal. If an LMS container is already running background, this command will stop it, recreate it, and attach your terminal to it. Later, to detach your terminal without stopping the container, just hit ``Ctrl+z``.
|
||||
|
||||
|
||||
XBlock and edx-platform plugin development
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
@ -11,6 +11,7 @@ from typing_extensions import ParamSpec
|
||||
|
||||
from tutor import config as tutor_config
|
||||
from tutor import env, fmt, hooks
|
||||
from tutor.hooks import priorities
|
||||
|
||||
|
||||
class DoGroup(click.Group):
|
||||
@ -40,6 +41,15 @@ def _add_core_init_tasks() -> None:
|
||||
("mysql", env.read_core_template_file("jobs", "init", "mysql.sh"))
|
||||
)
|
||||
with hooks.Contexts.APP("lms").enter():
|
||||
hooks.Filters.CLI_DO_INIT_TASKS.add_item(
|
||||
(
|
||||
"lms",
|
||||
env.read_core_template_file("jobs", "init", "mounted-edx-platform.sh"),
|
||||
),
|
||||
# If edx-platform is mounted, then we may need to perform some setup
|
||||
# before other initialization scripts can be run.
|
||||
priority=priorities.HIGH,
|
||||
)
|
||||
hooks.Filters.CLI_DO_INIT_TASKS.add_item(
|
||||
("lms", env.read_core_template_file("jobs", "init", "lms.sh"))
|
||||
)
|
||||
|
||||
@ -200,6 +200,13 @@ RUN openedx-assets themes \
|
||||
# Create a data directory, which might be used (or not)
|
||||
RUN mkdir /openedx/data
|
||||
|
||||
# If this "canary" file is missing from a container, then that indicates that a
|
||||
# local edx-platform was bind-mounted into that container, thus overwriting the
|
||||
# canary. This information is useful during edx-platform initialisation.
|
||||
RUN echo \
|
||||
"This copy of edx-platform was built into a Docker image." \
|
||||
> bindmount-canary
|
||||
|
||||
# service variant is "lms" or "cms"
|
||||
ENV SERVICE_VARIANT lms
|
||||
ENV DJANGO_SETTINGS_MODULE lms.envs.tutor.production
|
||||
|
||||
@ -9,6 +9,15 @@ x-openedx-job-service:
|
||||
args:
|
||||
# Note that we never build the openedx-dev image with root user ID, as it would simply fail.
|
||||
APP_USER_ID: "{{ HOST_USER_ID or 1000 }}"
|
||||
volumes:
|
||||
# Settings & config
|
||||
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
|
||||
- ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
|
||||
- ../apps/openedx/config:/openedx/config:ro
|
||||
# theme files
|
||||
- ../build/openedx/themes:/openedx/themes
|
||||
# editable requirements
|
||||
- ../build/openedx/requirements:/openedx/requirements
|
||||
|
||||
services:
|
||||
|
||||
|
||||
26
tutor/templates/jobs/init/mounted-edx-platform.sh
Normal file
26
tutor/templates/jobs/init/mounted-edx-platform.sh
Normal file
@ -0,0 +1,26 @@
|
||||
# When a new local copy of edx-platform is bind-mounted, certain build
|
||||
# artifacts from the openedx image's edx-platform directory are lost.
|
||||
# We regenerate them here.
|
||||
|
||||
if [ -f /openedx/edx-platform/bindmount-canary ] ; then
|
||||
# If this file exists, then edx-platform has not been bind-mounted,
|
||||
# so no build artifacts need to be regenerated.
|
||||
echo "Using edx-platform from image (not bind-mount)."
|
||||
echo "No extra setup is required."
|
||||
exit
|
||||
fi
|
||||
|
||||
echo "Performing additional setup for bind-mounted edx-platform."
|
||||
set -x # Echo out executed lines
|
||||
|
||||
# Regenerate Open_edX.egg-info
|
||||
pip install -e .
|
||||
|
||||
# Regenerate node_modules
|
||||
npm clean-install
|
||||
|
||||
# Regenerate static assets.
|
||||
openedx-assets build --env=dev
|
||||
|
||||
set -x
|
||||
echo "Done setting up bind-mounted edx-platform."
|
||||
Loading…
Reference in New Issue
Block a user