6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-12 22:27:47 +00:00
Commit Graph

29 Commits

Author SHA1 Message Date
Régis Behmo
15b219e235 feat: migrate to plugins.v1 with filters & actions
This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.

Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.

This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.

The plugin v0 API remains supported for backward compatibility of
existing plugins.

Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
  the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority

Close #499.
2022-04-15 15:30:54 +02:00
Régis Behmo
89520c016f chore: upgrade mypy
As a consequence, we need to resolve a typing issue.
2022-02-21 10:59:53 +01:00
Régis Behmo
78117d16f2 chore: get rid of outdated pycryptodome ugly patch
This patch is no longer required now that the fix has been merged upstream, in
3.10.3: https://github.com/Legrandin/pycryptodome/issues/506
2021-10-25 20:19:27 +02:00
Régis Behmo
07ae8d472f docs: generate reference docs automatically
This is performed with the help of sphinx-click:
https://sphinx-click.readthedocs.io
2021-10-14 12:59:57 +02:00
Régis Behmo
d4dc02b0fd chore: upgrade requirements
I just applied `make upgrade-requirements`. This chore needs to be performed
once in a while.
2021-10-05 00:02:36 +02:00
Régis Behmo
14d8276529 chore: upgrade pinned requirements
The dev requirements had to be updated to take into account the fact that some
type annotations have been moved to dedicated packages.

See the related packages:
https://pypi.org/project/types-setuptools/
https://pypi.org/project/types-PyYAML/

And the corresponding parent project:
https://github.com/python/typeshed
2021-07-03 11:08:27 +02:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
e990291d16 feat: upgrade pinned requirements to click 8+
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.

We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
2021-06-06 14:38:59 +02:00
Régis Behmo
1a5e2e005b fix: requirements of pallets project
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/

We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.

The following error was being raised:

    $ tutor plugins list
    Traceback (most recent call last):
      File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
	from tutor.commands.cli import main
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
	import click_repl
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
	import click._bashcomplete
    ModuleNotFoundError: No module named 'click._bashcomplete'

Close #430.
2021-05-12 10:12:31 +02:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Régis Behmo
e3a00c1bb6 Upgrade tutor requirements
This addresses a cryptography vulnerability. See:
https://github.com/overhangio/tutor/pull/400
2021-02-10 13:34:51 +01:00
Régis Behmo
25e838856d Upgrade requirements
In particular, this upgrades the bleach package.
See: https://github.com/overhangio/tutor/pull/399
2021-02-04 17:02:40 +01:00
Régis Behmo
ad858cd699 Fix urllib3 dependency error on installation of tutor
requests depends on urllib3<1.26.0, while urllib3==1.26.0 was just
released. We need to introduce a constraint on urllib3 to avoid the
following error when running `pip install -e`:

    pkg_resources.ContextualVersionConflict: (urllib3 1.26.0
    (./venv/lib/python3.7/site-packages),
    Requirement.parse('urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1'),
    {'requests'})

 We will be able to remove this constraint once requests becomes
 compatible with urllib3==1.26.0. To test this change just run `pip
 install -e tutor` and then run a dummy command, such as `tutor local
 stop`.
2020-11-11 10:36:42 +01:00
Régis Behmo
8d4bb1fc9e Upgrade all requirements after cryptography security release
See https://github.com/overhangio/tutor/pull/378
2020-10-27 23:53:51 +01:00
Régis Behmo
310dbe968a Upgrade pinned requirements 2020-10-15 17:20:27 +02:00
Régis Behmo
6bc3e23dc1 Upgrade tutor requirements 2020-06-17 12:11:54 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
bce6432d85 Improve job running in local and k8s
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:

- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.

Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.

This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.

In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
2020-04-25 23:12:42 +02:00
Régis Behmo
d7f13ee60e Fix yet another bleach vulnerability by requirement upgrade 2020-04-01 20:15:54 +02:00
Régis Behmo
a991784d8f Upgrade requirements
This allows us to fix a medium security issue in bleach: see v3.1.3 in
https://github.com/mozilla/bleach/blob/master/CHANGES
2020-03-24 22:01:44 +01:00
Régis Behmo
e91495ecc6 Upgrade requirements
This follows a security upgrade announcement for bleach (3.1.0 ->
3.1.1).
2020-02-25 11:26:50 +01:00
Régis Behmo
6d46d59be7 Upgrade all requirements 2020-01-13 22:40:10 +01:00
Régis Behmo
2e3ebaf084 Upgrade all requirements
Also, add a `upgrade-requirements` target.
2020-01-10 15:38:17 +01:00
Régis Behmo
f7a45ebc7d Upgrade pyyaml 2019-10-09 13:22:00 +02:00
Régis Behmo
c43e041e23 Get rid of kubernetes pip package dependency
This is not required anymore, now that we fetch the pod name from the
command line.
2019-06-07 22:49:45 +02:00
Régis Behmo
5e36febce4 Upgrade urllib3 following vulnerability alert
https://nvd.nist.gov/vuln/detail/CVE-2019-11324
2019-04-20 00:45:48 +02:00
Régis Behmo
d2f6783306 Fix security issue with jinja2 requirement
https://nvd.nist.gov/vuln/detail/CVE-2019-10906

To fix this issue, all requirements were simply upgraded to the latest
version.
2019-04-17 08:18:45 +02:00
Régis Behmo
ddbcedab5a Fix pyyaml deprecated warning
We were getting the following warning with the latest version of pyyaml:

    YAMLLoadWarning: calling yaml.load() without Loader=... is
    deprecated, as the default Loader is unsafe. Please read
    https://msg.pyyaml.org/load for full details.
2019-03-24 14:43:17 -07:00
Régis Behmo
4331bc5712 Tutor v3 complete rewrite
Replace all make commands by a single "tutor" binary. Environment and
data are all moved to ~/.tutor/local/share/tutor. We take the
opportunity to add a web UI and revamp the documentation.

This is a complete rewrite.

Close #121.
Close #147.
2019-02-09 20:30:01 +01:00