6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-14 06:58:21 +00:00
Commit Graph

42 Commits

Author SHA1 Message Date
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
e990291d16 feat: upgrade pinned requirements to click 8+
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.

We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
2021-06-06 14:38:59 +02:00
Régis Behmo
1a5e2e005b fix: requirements of pallets project
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/

We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.

The following error was being raised:

    $ tutor plugins list
    Traceback (most recent call last):
      File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
	from tutor.commands.cli import main
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
	import click_repl
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
	import click._bashcomplete
    ModuleNotFoundError: No module named 'click._bashcomplete'

Close #430.
2021-05-12 10:12:31 +02:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Régis Behmo
e3a00c1bb6 Upgrade tutor requirements
This addresses a cryptography vulnerability. See:
https://github.com/overhangio/tutor/pull/400
2021-02-10 13:34:51 +01:00
Régis Behmo
25e838856d Upgrade requirements
In particular, this upgrades the bleach package.
See: https://github.com/overhangio/tutor/pull/399
2021-02-04 17:02:40 +01:00
Régis Behmo
728ef966dc v11.0.0 (2020-12-09)
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
    - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
    - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
    - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
    - The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
    - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
	- Nginx no longer listens to port 443 for https traffic
	- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
	- Configuration files for web proxies are no longer provided.
	- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
    - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
    - Replace memcached and rabbitmq by redis.
- Additional features:
    - Make it possible to disable all plugins at once with ``plugins disable all``.
    - Add ``tutor k8s wait`` command to wait for a pod to become ready
    - Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
    - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
    - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
2020-12-10 01:05:02 +01:00
Régis Behmo
ad858cd699 Fix urllib3 dependency error on installation of tutor
requests depends on urllib3<1.26.0, while urllib3==1.26.0 was just
released. We need to introduce a constraint on urllib3 to avoid the
following error when running `pip install -e`:

    pkg_resources.ContextualVersionConflict: (urllib3 1.26.0
    (./venv/lib/python3.7/site-packages),
    Requirement.parse('urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1'),
    {'requests'})

 We will be able to remove this constraint once requests becomes
 compatible with urllib3==1.26.0. To test this change just run `pip
 install -e tutor` and then run a dummy command, such as `tutor local
 stop`.
2020-11-11 10:36:42 +01:00
Régis Behmo
8d4bb1fc9e Upgrade all requirements after cryptography security release
See https://github.com/overhangio/tutor/pull/378
2020-10-27 23:53:51 +01:00
Régis Behmo
310dbe968a Upgrade pinned requirements 2020-10-15 17:20:27 +02:00
Régis Behmo
6bc3e23dc1 Upgrade tutor requirements 2020-06-17 12:11:54 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
bce6432d85 Improve job running in local and k8s
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:

- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.

Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.

This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.

In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
2020-04-25 23:12:42 +02:00
Régis Behmo
d7f13ee60e Fix yet another bleach vulnerability by requirement upgrade 2020-04-01 20:15:54 +02:00
Régis Behmo
a991784d8f Upgrade requirements
This allows us to fix a medium security issue in bleach: see v3.1.3 in
https://github.com/mozilla/bleach/blob/master/CHANGES
2020-03-24 22:01:44 +01:00
Régis Behmo
e91495ecc6 Upgrade requirements
This follows a security upgrade announcement for bleach (3.1.0 ->
3.1.1).
2020-02-25 11:26:50 +01:00
Régis Behmo
6d46d59be7 Upgrade all requirements 2020-01-13 22:40:10 +01:00
Régis Behmo
2e3ebaf084 Upgrade all requirements
Also, add a `upgrade-requirements` target.
2020-01-10 15:38:17 +01:00
Régis Behmo
9620447a4d Add the lts plugin to the tutor binary 2019-12-25 01:09:53 +01:00
Régis Behmo
5675af4d42 Fix jinja2 dependency
jinja2 should include the `tojson` filter, which is only available in
2.9 (Released 2017-01-07, codename Derivation).
2019-10-14 06:26:25 +02:00
Régis Behmo
f7a45ebc7d Upgrade pyyaml 2019-10-09 13:22:00 +02:00
Régis Behmo
1224313e7c Make sure twine & pip are up-to-date in CI
We were affected by the following bug when installing pyinstaller in CI:

https://github.com/pyinstaller/pyinstaller/issues/4006
https://github.com/pypa/pip/issues/6163

As a consequence, pip is upgraded in CI. Also, twine is now part of dev
requirements.
2019-10-09 13:22:00 +02:00
Régis Behmo
a94cbc2914 Move plugins to dedicated repositories
minio, notes and xqueue plugins are moved to tutor-* github repos.
2019-08-21 18:32:39 +02:00
Régis Behmo
7790028cf7 Move tutor-minio to dedicated plugin repo 2019-08-20 17:46:53 +02:00
Régis Behmo
be1ff08917 Add all plugins (with data) to binary bundle
All existing plugins are added to the binary bundle, in their latest
version, so that users don't need to pip install tutor.

Also, the tutor MANIFEST.in file was removed to simplify the management
of package data.

Close #242.
2019-08-20 17:03:46 +02:00
Régis Behmo
11e735f4e5 Migrate notes to a dedicated plugin 2019-07-04 09:31:12 +08:00
Régis Behmo
07a0323d8e Move Xqueue to a dedicated plugin
This gives us the opportunity to develop new hooks: build-image and
remote-image.
2019-07-04 09:31:12 +08:00
Régis Behmo
4c96e83e3c Get rid of kubernetes requirement from pypi package
v3.4.0 package from pypi was still installing the kubernetes
requirement, although is wasn't required anymore.
2019-06-23 13:22:59 +02:00
Régis Behmo
c43e041e23 Get rid of kubernetes pip package dependency
This is not required anymore, now that we fetch the pod name from the
command line.
2019-06-07 22:49:45 +02:00
Régis Behmo
334f3e720e Towards a stable Kubernetes integration
Missing features:
- https certificates
- xqueue
- lms/cms workers

Moreover, we scalability issues due to the uploaded file storage in the
lms/cms. To address this issue we need to develop the MinIO plugin so
that it becomes compatible with Open edX.

Close #126 #179 #187
2019-06-07 22:49:45 +02:00
Régis Behmo
3b108d21de 🔌 Introduce a plugin system for tutor
This adds the basic feratures that we need for a working plugin system,
but there are still many TODOs in the codebase.
2019-06-07 22:49:45 +02:00
Régis Behmo
4ac7dff06a Switch from nose to unittest for running tests
This harmonizes test running between CI and local development.
2019-05-05 12:30:20 +02:00
Régis Behmo
9dd4c26706 Introduce automatic code formatting/linting
Code formatting makes sure that the python code looks decent, but it
does not check for coding errors.
https://black.readthedocs.io/en/stable/

Code linting runs static error detection on the python code, but does
not bother about formatting: https://pylint.readthedocs.io/en/latest/
2019-05-05 12:30:20 +02:00
Régis Behmo
5f625f7baa Simplify docs requirement management
It is assumed that users who compile the documentation also need base
requirements.
2019-04-20 00:47:20 +02:00
Régis Behmo
5e36febce4 Upgrade urllib3 following vulnerability alert
https://nvd.nist.gov/vuln/detail/CVE-2019-11324
2019-04-20 00:45:48 +02:00
Régis Behmo
d2f6783306 Fix security issue with jinja2 requirement
https://nvd.nist.gov/vuln/detail/CVE-2019-10906

To fix this issue, all requirements were simply upgraded to the latest
version.
2019-04-17 08:18:45 +02:00
Régis Behmo
ddbcedab5a Fix pyyaml deprecated warning
We were getting the following warning with the latest version of pyyaml:

    YAMLLoadWarning: calling yaml.load() without Loader=... is
    deprecated, as the default Loader is unsafe. Please read
    https://msg.pyyaml.org/load for full details.
2019-03-24 14:43:17 -07:00
Régis Behmo
7324d71d2c Add missing nose requirements
This should fix ci build.
2019-03-24 09:40:33 -07:00
Régis Behmo
6889f6fa90 Add unit tests! yay!
Now, that was seriously missing.
2019-03-23 16:16:44 -07:00
Régis Behmo
786778e5fb Add requirements for docs generation 2019-02-20 12:52:57 +01:00
Régis Behmo
5b4beed1d1 Minimum required click version is 7.0
Close #171
2019-02-13 17:14:10 +01:00
Régis Behmo
4331bc5712 Tutor v3 complete rewrite
Replace all make commands by a single "tutor" binary. Environment and
data are all moved to ~/.tutor/local/share/tutor. We take the
opportunity to add a web UI and revamp the documentation.

This is a complete rewrite.

Close #121.
Close #147.
2019-02-09 20:30:01 +01:00