6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-16 10:05:18 +00:00
Commit Graph

33 Commits

Author SHA1 Message Date
Régis Behmo
15b219e235 feat: migrate to plugins.v1 with filters & actions
This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.

Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.

This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.

The plugin v0 API remains supported for backward compatibility of
existing plugins.

Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
  the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority

Close #499.
2022-04-15 15:30:54 +02:00
Régis Behmo
4dc772d1e4 fix: attempt to make upgrade much clearer
`upgrade` had several issues, which are summarized here:
https://discuss.overhang.io/t/confusing-instructions-during-upgrade/2281/7

- The docs say that you should run quickstart, but what most people will see is
the big command tutor local upgrade --from=lilac verbatim paragraph.
- The local upgrade command should be very explicit about the fact that users
need to run quickstart.
- Maybe the name of the local upgrade command should be improved.
- When upgrading tutor from one major release to the next, there should be a
more explicit warning to inform users of what they are doing (see this other
conversation 1)
- We should tell people that they almost certainly need to enable the tutor and
the mfe plugins, if they are not enabled during upgrade.
- A link to all of the breaking changes from the changelog should be
prominently displayed during upgrade.
- The docs should emphasize that upgrading from one major release to the next
is potentially a risky endeavor and that downgrading is not possible. The docs
should also link to the changelog.

This commit has grown slightly beyond the intended scope, but the changes should be mostly positive.
2022-01-08 19:07:26 +01:00
Régis Behmo
1daba42f1e code: refactor version checking code 2022-01-08 19:07:26 +01:00
alex.soh
72843c06f9 refactor: add code coverage, cover CLI commands with tests 2022-01-04 13:40:33 +01:00
Régis Behmo
c40e682f5d refactor: clarify configuration management
Previously, configuration management was very confusing because we kept mixing
"base" and "defaults" configuration:

- It was difficult to make the difference between core settings that were
  necessary (e.g: passwords) as opposed to others that could simply be
  defaulted to.
- The order of settings in config.yml mattered: config entries that depended on
  other needed to be defined later. As a consequence, Tutor was not compatible
  with Python 3.5, where dict entries are not sorted.
2021-12-20 21:19:10 +01:00
Régis Behmo
0c1d4ebf51 fix: https test 2021-12-20 21:19:10 +01:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
e990291d16 feat: upgrade pinned requirements to click 8+
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.

We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
2021-06-06 14:38:59 +02:00
Régis Behmo
336cb79fa8 refactor: better config type checking
I stumbled upon a bug that should have been detected by the type
checking. Turns out, considering that config is of type Dict[str, Any]
means that we can use just any method on all config values -- which is
terrible. I discovered this after I set `config["PLUGINS"] = None`:
this triggered a crash when I enabled a plugin.
We resolve this by making the Config type more explicit. We also take
the opportunity to remove a few cast statements.
2021-04-18 18:02:02 +02:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Régis Behmo
728ef966dc v11.0.0 (2020-12-09)
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
    - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
    - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
    - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
    - The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
    - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
	- Nginx no longer listens to port 443 for https traffic
	- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
	- Configuration files for web proxies are no longer provided.
	- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
    - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
    - Replace memcached and rabbitmq by redis.
- Additional features:
    - Make it possible to disable all plugins at once with ``plugins disable all``.
    - Add ``tutor k8s wait`` command to wait for a pod to become ready
    - Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
    - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
    - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
2020-12-10 01:05:02 +01:00
Régis Behmo
998dec7149 Fix template name separator for Windows users
See: https://github.com/overhangio/tutor/issues/381
2020-11-07 16:37:43 +01:00
Régis Behmo
7ed49e4716 Fix formatting according to newest black release 2020-10-15 17:33:28 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
2159226b4b Fix missing partials folder in config render
"partials" folders were not rendered at all when using `config render`.
2020-01-16 23:58:20 +01:00
Régis Behmo
72e23f3f96 Add config render command
This is going to be useful for using custom themes with user-defined
variables.
2020-01-16 17:15:55 +01:00
Régis Behmo
c4c12b0ab8 env.py refactoring
Clarify a few variable names, make code more modular. Also, the Renderer
class now makes more sense as a singleton. We took the opportunity to
delete quite a lot of code.
2020-01-16 15:07:35 +01:00
Régis Behmo
c1e47cec82 Minor internal function renaming 2020-01-14 12:01:06 +01:00
Régis Behmo
0c0db78310 Get rid of mysql-client container
This has an impact on plugin hooks. Plugin hooks that needed to run
inside mysql-client now need to run inside mysql container. This
simplifies the deployment, as we no longer have an empty mysql-client
container sitting around.

When mysql is not enabled (ACTIVATE_MYSQL=False) the mysql container is
simply a mysql client.
2019-09-03 10:36:10 +02:00
Régis Behmo
2c01c8cc01 Fox test_env formatting 2019-08-21 18:41:55 +02:00
Régis Behmo
7790028cf7 Move tutor-minio to dedicated plugin repo 2019-08-20 17:46:53 +02:00
Régis Behmo
1e995aaf1a More robust plugin/env config checking 2019-07-10 16:20:43 +08:00
Régis Behmo
07a0323d8e Move Xqueue to a dedicated plugin
This gives us the opportunity to develop new hooks: build-image and
remote-image.
2019-07-04 09:31:12 +08:00
Régis Behmo
84f2060d33 Working Kubernetes quickstart
The k8s quickstart command is now functional, with suppport for https,
xqueue, notes and minio. There are still a few bugs to get rid of,
though.
2019-06-17 22:38:55 +02:00
Régis Behmo
6a68c4cc20 Progress on the plugins/k8s front
This commit introduces many changes:
- a fully functional minio plugin for local installation
- an almost-functional native k8s deployment
- a new way to process configuration, better suited to plugins

There are still many things to do:
- get rid of all the TODOs
- get a fully functional minio plugin for k8s
- add documentation for pluginso
- ...
2019-06-07 22:49:45 +02:00
Régis Behmo
f5c225231f Move config logic to dedicated non-command module 2019-06-07 22:49:45 +02:00
Régis Behmo
3b108d21de 🔌 Introduce a plugin system for tutor
This adds the basic feratures that we need for a working plugin system,
but there are still many TODOs in the codebase.
2019-06-07 22:49:45 +02:00
Régis Behmo
58d2af30a0 Minor formatting 2019-05-29 11:20:37 +02:00
Régis Behmo
ebcb08b5eb Fix save/render for single files 2019-05-29 10:31:53 +02:00
Régis Behmo
65dce18908 Add TUTOR_VERSION to template variables 2019-05-29 09:53:54 +02:00
Régis Behmo
0199a1e916 Add convenient fmt.echo_* functions 2019-05-29 09:53:54 +02:00
Régis Behmo
407659ff06 Major refactoring of config module
Configuration loading was overly complex. Here, we simplify it
drastically with reasonable defaults. Hacky additional variables are
unncessary now that we use custom jinja2 filters.
2019-05-29 09:53:54 +02:00
Régis Behmo
68d23c5dd5 Env module refactoring
Better naming and tests.
2019-05-29 09:53:54 +02:00