This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.
Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.
This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.
The plugin v0 API remains supported for backward compatibility of
existing plugins.
Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority
Close #499.
MySQL 8 drop the support for creating users by executing `GRANT ALL`. This commit splits the user creation and permission granting, therefore the newer MySQL versions are supported too.
MySQL 8 is supported by edx-platform: 1cdb0347c5/playbooks/roles/mysql/tasks/mysql.yml (L93-L98)
PR #619 set the EDX_PLATFORM_VERSION build arg's default to
OPENEDX_COMMON_VERSION. While this works fine for setting a
non-default branch to run edx code from (say, "master"), it may break
if the user sets OPENEDX_COMMON_VERSION to a branch or tag name that
does not exist upstream in repositories *other than*
EDX_PLATFORM_REPOSITORY.
Thus, introduce a separate configuration parameter,
EDX_PLATFORM_VERSION, to match the build arg of the same name. Set its
default to OPENEDX_COMMON_VERSION.
This way, the user can deploy an arbitrarily-named fork of
edx-platform, while retaining the default OPENEDX_COMMON_VERSION
(like, for example "open-release/maple.3") for everything else.
Previously, the `k8s exec` command did not support unknown "--options". This
made it impossible to launch, say, a django shell in the lms container.
While implementing this feature we saw an opportunity to simplify the way jobs
are handled in the k8s commands.
Close #636.
Another related issue is: https://github.com/overhangio/2u-tutor-adoption/issues/52
The docs recommend commands like:
pip install tutor[full]
pip install -e ./tutor[full]
for installing Tutor. These work fine in bash. For zsh,
though, which is now the default on macOS, quotes are
needed, otherwise zsh will interpret the brackets as
special syntax:
pip install "tutor[full]"
pip install -e "./tutor[full]"
Caveat: I have not tested this myself since I don't
own a Mac, but I've read several issue reports to this
effect, such as:
https://github.com/pypa/pipenv/issues/2830#issuecomment-419593199
The full installation will include all the plugins that
come bundled with Tutor stable. This is made possible by
a recent change to Tutor Nightly
(https://github.com/overhangio/tutor/pull/626).
- [Security] Apply SAML security fix.
- [Improvement] In addition to the Docker build arguments
`EDX_PLATFORM_REPOSITORY` and `NPM_REGISTRY`, also support two corresponding
and identically-named `config.yml` values serving the same purpose.
Previously, the only way for Tutor users to use a fork of edx-platform
or a custom NPM registry was to use build args during the image build.
This is suboptimal in the case of automatically building images from
CI pipelines, which may want to auto-detect when an image needs to be
rebuilt based on config.yml changes.
In addition, the EDX_PLATFORM_VERSION build argument can already be
set via a corresponding config.yml parameter (OPENEDX_COMMON_VERSION),
so it's reasonable to follow that precedent and also introduce
config.yml parameters to correspond with the EDX_PLATFORM_REPOSITORY
and NPM_REGISTRY build arguments.
Thus, introduce two new configuration parameters:
- EDX_PLATFORM_REPOSITORY
- NPM_REGISTRY
These parameters can now optionally be used instead of the
aforementioned build args.
The version of the nightly python package should not include the "-nightly"
suffix. That's because when we `pip install -e` tutor plugins, pip also
installs the latest tutor release, as part of the requirements. This overrides
the local (nightly) installation of tutor.
See: https://app.slack.com/client/T02SNA1T6/C02V3GHE3UP
- [Bugfix] Fix dockerize on arm64 by switching to the [powerman/dockerize](https://github.com/powerman/dockerize) fork (#591).
- [Bugfix] Fix "Unexpected args" error during service initialization on Kubernetes (#611).
The version of dockerize that shipped with the "openedx" image was not
compatible with arm64. The original project is unmaintained, but there
is a fork that provides a version that is compatible with arm64.
This was tested on arm64 with buildx:
docker buildx build --tag=openedx --platform=linux/arm64 ~/.local/share/tutor/env/build/openedx
Close #591
- [Bugfix] Fix `local/k8s quickstart` commands when upgrading from an older release (#595).
- [Bugfix] Fix running the default exim-relay SMTP server on arm64 (#600).
- [Feature] Add `tutor k8s apply` comand, which is a direct interface with `kubectl apply`.
- [Feature] Add `openedx-dockerfile-minimal` patch, which you can use to install custom packages and run commands as root in the Docker image.
I found the existing docs a bit light on the particulars
of how the YAML and Python plugin APIs relate. I was
able to figure it out (there's a nice congruence
between them) but I think these tweaks should it make
it more immediately obvious to readers how the Python
API is a essentially a superset of the YAML API that
allows for dynamic behavior.
because it only contains CLI reference information currently.
The folder structure implies that eventually there will be
more reference material, so the name of 'reference.rst'
was *not* changed to 'cli-reference.rst'.
- [Security] Fix vulnerability in call to invalid enrollment API (see [commit](e9369cffde)).
- [Bugfix] Fix "Internal Server Error / AttributeError / object has no attribute 'get_metadata'" in learning MFE.
- [Improvement] Replace all links to github.com/edx by github.com/openedx, following the migration of all repositories.
- [Bugfix] Fix `k8s start caddy` command.
Régis Behmo
Gábor Boros
Kyle McCormick
Florian Haas
Rebecca Graber
Emad Rad