pkg_resources is available in python 3.12 only if setuptools is
explicitely installed, which is not always the case. We fix that by
replacing all usage of pkg_resources with importlib_resources and
importlib_metadata.
Close #966
Running `tutor config save` with an outdated version of pycryptodome was
failing with the following error:
Error: Missing configuration value: 'Crypto.PublicKey.RSA.RsaKey object' has no attribute 'dq'
This is because the "dq" attribute was only introduced in pycryptodome
3.17.0: https://www.pycryptodome.org/src/changelog#january-2023
To resolve this issue we bump the minimum requirements.
Close #962
People running typing-extensions==3.10 faced this error for just any tutor command:
$ tutor version
...
Traceback (most recent call last):
File "/usr/local/bin/tutor", line 5, in <module>
from tutor.commands.cli import main
File "/usr/local/lib/python3.8/dist-packages/tutor/commands/cli.py", line 7, in <module>
from tutor import exceptions, fmt, hooks, utils
File "/usr/local/lib/python3.8/dist-packages/tutor/hooks/__init__.py", line 7, in <module>
from . import actions, contexts, filters, priorities
File "/usr/local/lib/python3.8/dist-packages/tutor/hooks/actions.py", line 18, in <module>
class ActionCallback(Contextualized, t.Generic[P]):
File "/usr/lib/python3.8/typing.py", line 261, in inner
return func(*args, **kwds)
File "/usr/lib/python3.8/typing.py", line 890, in __class_getitem__
raise TypeError(
TypeError: Parameters to Generic[...] must all be type variables
We fix this error by requiring a more recent version of typing-extensions.
See: https://discuss.openedx.org/t/tutor-v15-python-error-when-running-on-quickstart/8910/2
The following syntax is only supported in jinja2>=2.10:
{% set jwt_rsa_key | rsa_import_key %}{{ JWT_RSA_PRIVATE_KEY }}{% endset %}
Thus, we bump the minimal working version of jinja2 in the base requirements.
See discussion: https://discuss.openedx.org/t/error-while-tutor-local-quickstart/8796
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.
We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/
We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.
The following error was being raised:
$ tutor plugins list
Traceback (most recent call last):
File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
from tutor.commands.cli import main
File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
import click_repl
File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
import click._bashcomplete
ModuleNotFoundError: No module named 'click._bashcomplete'
Close #430.
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate
We are running in strict mode, which is awesome!
This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
requests depends on urllib3<1.26.0, while urllib3==1.26.0 was just
released. We need to introduce a constraint on urllib3 to avoid the
following error when running `pip install -e`:
pkg_resources.ContextualVersionConflict: (urllib3 1.26.0
(./venv/lib/python3.7/site-packages),
Requirement.parse('urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1'),
{'requests'})
We will be able to remove this constraint once requests becomes
compatible with urllib3==1.26.0. To test this change just run `pip
install -e tutor` and then run a dummy command, such as `tutor local
stop`.
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:
- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7
We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.
To upgrade, users should simply run:
tutor local quickstart
For Kubernetes platform, the upgrade process is outlined when running:
tutor k8s upgrade --from=ironwood
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:
- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.
Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.
This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.
In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
Missing features:
- https certificates
- xqueue
- lms/cms workers
Moreover, we scalability issues due to the uploaded file storage in the
lms/cms. To address this issue we need to develop the MinIO plugin so
that it becomes compatible with Open edX.
Close #126#179#187
Replace all make commands by a single "tutor" binary. Environment and
data are all moved to ~/.tutor/local/share/tutor. We take the
opportunity to add a web UI and revamp the documentation.
This is a complete rewrite.
Close #121.
Close #147.
