Templated hooks we almost completely useless, so we get rid of them.
This allows us to get rid entirely of hook names and hook indexes, which
makes the whole implementation much simpler. Hook removal (with
`clear_all`) is achieved thanks to weak references.
This is an important change, where we get remove the previous `--mount`
option, and instead opt for persistent bind-mounts.
Persistent bind mounts have several advantages:
- They make it easier to remember which folders need to be bind-mounted.
- Code is *much* less clunky, as we no longer need to generate temporary
docker-compose files.
- They allow us to bind-mount host directories *at build time* using the
buildx `--build-context` option.
- The transition from development to production becomes much easier, as
images will automatically be built using the host repo.
The only drawback is that persistent bind-mounts are slightly less
portable: when a config.yml file is moved to a different folder, many
things will break if the repo is not checked out in the same path.
For instance, this is how to start working on a local fork of
edx-platform:
tutor config save --append MOUNTS=/path/to/edx-platform
And that's all there is to it. No, this fork will be used whenever we
run:
tutor images build openedx
tutor local start
tutor dev start
This change is made possible by huge improvements in the build time
performance. These improvements make it convenient to re-build Docker
images often.
Related issues:
https://github.com/openedx/wg-developer-experience/issues/71https://github.com/openedx/wg-developer-experience/issues/66https://github.com/openedx/wg-developer-experience/issues/166
It was useless to create a *-permissions job for every application.
Instead, we create a single "permissions" service. It can be extended
via the "docker-compose-permissions-command" patch.
Among other changes: ORA2 file uploads were stored in a folder named
"SET-ME-PLEASE (ex. bucket-name)" (sigh). With this change, the folder
should be automatically renamed to "openedxuploads". This issue has been
occuring since June 2019... (sigh²)
Close #707
Before this commit, setting up an edx-platform development environment
took multiple steps:
tutor dev launch
tutor dev run --mount=/path/to/edx-platform lms bash
>> pip install -e .
>> npm clean-install
>> openedx-assets build --env=dev
This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:
tutor dev launch --mount=edx-platform
These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.
Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.
These bind-mounts:
* ../build/openedx/themes:/openedx/themes
* ../build/openedx/requirements:/openedx/requirements
existed in the dev lms and cms containers, but they did
not exist in the lms-job and cms-job containers.
This means that themes and requirements that were *built into the
image* would exist in the job containers, but live updates to the
themes and requirements would not apply.
To resolve this, we set ``volumes:`` on the lms-job and cms-job
services so that they match the volumes for the normal lms and
cms services.
Part of: https://github.com/openedx/wg-developer-experience/issues/146
Closes: https://github.com/openedx/wg-developer-experience/issues/152
This works around (but does not close) these related issues:
* https://github.com/openedx/wg-developer-experience/issues/150
* https://github.com/openedx/wg-developer-experience/issues/151
On some OS, high ulimit causes some container to use a lot of memory. We
do not attempt to resolve this issue in Tutor because this is a
mysql/uwsgi issue. Instead, we explain how to resolve it in the
troubleshooting docs.
Close #671.
We implement this TEP: https://discuss.openedx.org/t/tutor-enhancement-proposal-tep-plugin-indices/8182
With plugin indexes, tutor users can install and upgrade plugins directly from indexes:
tutor plugins install ecommerce
tutor plugins index add contrib
tutor plugins install codejail
tutor plugins upgrade all
This change has been long in the coming \o/
Users want to be able to override the request `max_size` to upload
larger files. But they will not be able to if the patch is placed after
the `request` directive. So we move the patch statement before the
directive. Also, we wrap the `request_body` directives within `handle`
statements. If not, then different sizes are not managed properly.
To override the max upload size in the cms, add the following to the
"caddyfile-cms" patch:
handle_path /import/* {
request_body {
max_size 500MB
}
}
See discussion:
https://discuss.openedx.org/t/how-to-update-caddyfile-using-tutor-plugin/8944
The hooks API had several issues which are summarized in this comment:
https://github.com/openedx/wg-developer-experience/issues/125#issuecomment-1313553526
1. "consts" was a bad name
2. "hooks.filters" and "hooks.Filters" could easily be confused
3. docs made it difficult to understand that plugin developers should use the catalog
To address these issues, we:
1. move "consts.py" to "catalog.py"
2. Remove "hooks.actions", "hooks.filters", "hooks.contexts" from the API.
3. re-organize the docs and give better usage examples in the catalog.
This change is a partial fix for https://github.com/openedx/wg-developer-experience/issues/125
Adds `from __future__ import annotations` to the top of every module,
right below the module's docstring. Replaces any usages of t.List,
t.Dict, t.Set, t.Tuple, and t.Type with their built-in equivalents:
list, dict, set, tuple, and type. Ensures that make test still passes
under Python 3.7, 3.8 and 3.9.
When running:
tutor dev run -m /path/to/edx-platform lms
pip install -r requirements/edx/development.txt
I realised that I was re-installing packages that should already have been
present in the image. The reason for that was that I was running an outdated
version of the dev version of the openedx Docker image. This happens because
`tutor dev run` does not trigger an image re-build.
We solve this issue by pinning the openedx dev Docker image tag to the current
tutor version.
This change builds upon a previously proposed PR:
https://github.com/overhangio/tutor/pull/437
There was another long conversation about this topic here:
https://github.com/overhangio/tutor-forum/pull/10#issuecomment-1314799915
We could have supported the MongoDB auth/replica set/ssl parameters as part of
the MongoDB host URI, but then this URI is not supported in the forum plugin,
which uses an old version of the mongoid client. We were hoping that the client
would have been upgraded by now, but it's not been upgraded for a long time.
The changes introduced here are 100% backward-compatible. The forum plugin will
have to be updated to take into account the new parameters.
Changelog management was starting to be a hassle:
- there were conflicts every time a PR was merged
- there were conflicts every time we merged the nightly branch in the new
release branch, or vice versa.
Now, all changelog entries are stored as separate files in changelog.d,
including nightly. Nightly entries will be collected for every major release.
Now that the mypy bugs have been resolved, we are able to define more precisely
and cleanly the types of Actions and Filters.
Moreover, can now strongly type named actions and hooks (in consts.py). With
such a strong typing, we get early alerts of hooks called with incorrect
arguments, which is nothing short of awesome :)
This change breaks the hooks API by removing the `context=...` argument. The
reason for that is that we cannot insert arbitrary arguments between `P.args,
P.kwargs`: https://peps.python.org/pep-0612/#the-components-of-a-paramspec
> A function declared as def inner(a: A, b: B, *args: P.args, **kwargs:
> P.kwargs) -> R has type Callable[Concatenate[A, B, P], R]. Placing
> keyword-only parameters between the *args and **kwargs is forbidden.
Getting the documentation to build in nitpicky mode is quite difficult... We
need to add `nitpick_ignore` to the docs conf.py, otherwise sphinx complains
about many missing class references. This, despite upgrading almost all doc
requirements (except docutils).
We introduce a new filter to implement custom commands in arbitrary containers.
It becomes easy to write convenient ad-hoc commands that users will
then be able to run either on Kubernetes or locally using a documented CLI.
Pluggable jobs are declared as Click commands and are responsible for
parsing their own arguments. See the new CLI_DO_COMMANDS filter.
Close https://github.com/overhangio/2u-tutor-adoption/issues/75
