We have a side effect whenever we add a plugin manually to the tutor
configuration: the renderer does not update itself. We fix this by
checking the config at load time.
We achieve this thanks to a patch to edx-oauth2-provider:
https://github.com/edx/edx-oauth2-provider/pull/59
The network alias was required to circumvent url validation by the
oauth2 client creation tool. But the network alias was not compatible
with k8s deployment.
The port number is not included in the hostname when checking the list
of allowed values.
We also add the "lms"/"cms" values for interaction with the API from
within the internal docker network.
Video transcripts uploaded in the CMS were not visible in the LMS. This
was a symptom caused by the fact that the LMS and the CMS do not share
the same MEDIA_ROOT. We initially thought that data uploaded in the CMS
(such as transcripts) was stored in a shared data service, such as
mongodb. It is, in fact, not. This makes it even more important to run
an object storage service like minio for distributed services.
Close #229
The 0003 migration from the certificates app of the LMS requires that
the S3-like platform is correctly setup during initialisation. To solve
this issue, we introduce a pre-init hook that is run prior to the LMS
migrations.
P3P is obsolete https://en.wikipedia.org/wiki/P3P
Also, these P3P headers leak information about the application that is
running, making it sensitive to attacks.
- [Feature] Creation of a plugin system
- [Feature] Kubernetes support out of beta
- [Improvement] Switch to pinned image tags for easier upgrades
- 💥[Improvement] Remove the `-y/--yes` option: `tutor config save` is
now non-interactive by default. Use `-i/--interactive` to force
interactive mode.
- 💥[Improvement] Replace the `databases` command by `init`.
- [Improvement] Upgrade to ironwood.2
- [Improvement] Add `-y/--yes` option to `local quickstart` for
non-interactive quickstart
- [Improvement] Persist LMS/CMS logs to disk by default (with
collaboration from @silviot 💪)
- [Bugfix] Fix installing a locally cloned requirement repository
- [Improvement] Add `--no-cache` option to `images build`
- [Improvement] Make it possible to configure the notes service hostname
- [Improvement] Better, more robust MySQL initialisation
Having an identical "ironwood" tag for all releases is not practical, in
particular for breaking changes. Thus, docker images are now pinned to
the tutor version that they were build with.
Thus, we remove the -y/--yes options, which were kind of unintuitive,
and we add instead `-i/--interactive`. The quickstart commands remain
interactive by default, but can be silenced with `-I/--non-interactive`.
This commit introduces many changes:
- a fully functional minio plugin for local installation
- an almost-functional native k8s deployment
- a new way to process configuration, better suited to plugins
There are still many things to do:
- get rid of all the TODOs
- get a fully functional minio plugin for k8s
- add documentation for pluginso
- ...
Missing features:
- https certificates
- xqueue
- lms/cms workers
Moreover, we scalability issues due to the uploaded file storage in the
lms/cms. To address this issue we need to develop the MinIO plugin so
that it becomes compatible with Open edX.
Close #126#179#187
- [Improvement] Upgrade to ironwood.2
- [Improvement] Add `-y/--yes` option to `local quickstart` for non-interactive quickstart
- [Improvement] Persist LMS/CMS logs to disk by default (with collaboration from @silviot 💪)
- [Bugfix] Fix installing a locally cloned requirement repository
- [Improvement] Add `--no-cache` option to `images build`
- [Improvement] Make it possible to configure the notes service hostname
- [Improvement] Better, more robust MySQL initialization
This is especially useful on Kubernetes. With this change, the forum
container no longer crashes whenever mongodb or elasticsearch are
unavailable. Instead, it just waits for thoses services to be up.
Previously, we could not run forum migrations in Kubernetes because they
relied on exec-ing the migration command in the running container -- and
there was no such container, because the services where not already up.
Configuration loading was overly complex. Here, we simplify it
drastically with reasonable defaults. Hacky additional variables are
unncessary now that we use custom jinja2 filters.
For initializing the mysql database, we now run the mysql client from
the mysql container. That means that the mysql client is no longer
required in the openedx container, as previously suggested in PR #209.
When running `pip install -r /path/to/x.txt`, relative paths listed in
x.txt are supposed to be relative to the current folder, and not the
x.txt folder. This results in an error during `pip install`:
./xblock-poll/ should either be a path to a local project or a VCS
url beginning with svn+, git+, hg+, or bzr+
This error was first reported here:
https://discuss.overhang.io/t/installing-custom-xblock/61
Tutor is not using any bash-specific feature.
This commit changes (almost) all invocations to /bin/bash replacing them with
/bin/sh. It allows building a docker image that does not bundle bash in it.
The file k8s.py still has a bash invocation, to make sure this change does not
impact functionality for any user.
Given that each gunicorn worker uses ~200-250Mb virtual memory, we should
really avoid running too many of them. Here, we reduce the default
number of workers to 2. For the LMS and the CMS, this number can be
adjusted by setting an environment variable GUNICORN_WORKERS.
This is related to #207.
Settings the number of workers for the LMS and the CMS to 2*|CPU|+1
results in 4*|CPU]+2 gunicorn workers, which is too much for a laptop to
cope with.
Close #207
Migrations were being done on openedx-assets, which does not depend on
mysql. As a consequence, the mysql container was not being created on
"migrate" and migrations were failing with "Timeout after 20s".
Installing from pip was broken because, for some reason, the
templates/apps/xqueue/settings/ folder contained a __pycache__ folder
with compiled *.pyc files. Rendering the binary files was failing
miserably.
Login from localhost/studio.localhost was broken on Ironwood because the
session cookie was configured to be stored under the production domain
name. We fix this by setting the SESSION_COOKIE_DOMAIN to None: in the
edx-platform code, this corresponds to using the current request domain.
Now that the correct webpack settings are loaded by the `update_assets`
command in Ironwood, we can stop relying on the `openedx-assets` script.
Actually, we could probably remove it.
We were getting the following warning with the latest version of pyyaml:
YAMLLoadWarning: calling yaml.load() without Loader=... is
deprecated, as the default Loader is unsafe. Please read
https://msg.pyyaml.org/load for full details.
With the upgrade to Ironwood, studio login redirects to the LMS by
default. This breaks login from studio.localhost. So we activate SSO
login only when HTTPS is activated, which should happen only when the
domains are correctly configured.
- [Feature] Multiple platforms on a single server \o/
- [Feature] Easily configure web proxy on the host
- [Bugfix] Fix `images pull all` command which failed on "all" image
- [Improvement] Add configurable mongodb, SMTP and rabbitmq
authentication
- [Improvement] Harmonize mysql username/password configuration
parameters
- [Feature] Configurable and pluggable data storage backends (#114)
In the future, we want to allow users to rely on third-party services
for data storage, such as hosted MySQL and such. To do so, we need to be
able to configure the host/port of these services, which we do here.
This is to address part of #114.
It was tricky to define correct settings: the "localfs" default storage
backend was causing incorrect urls of the form "/media/<id>/<name>.csv".
Close #143
Configuration values can be loaded from the system environment by adding
a "TUTOR_" prefix.
Environment values supersede values from the user configuration file, so
that we can set values from the command line with "KEY=VAL tutor config
save --silent" even when KEY is already present in the user
configuration file.
Environment is no longer generated separately for each target, but only
once the configuration is saved.
Note that the environment is automatically updated during
re-configuration, based on a "version" file stored in the environment.
- [Improvement] Install python requirements in virtual env in docker
image
- [Bugfix] Add missing volume for theme development
- [Improvement] Rename "config [non]interactive" command to "config save
[--silent]"
- [Improvement] More explicit logging during environment generation
- [Improvement] Configurable docker images (#122)
- [Bugfix] Fix "android pullimage" command
- [Improvement] Do not upgrade images as part of quickstart
- [Bugfix] Fix USERID setup in development mode and various dev-related
docs (#177)
Pulling images is slow, often unnecessary, and sometimes suprising for
the user, as the newer images may break the older tutor version.
Instead, we add a "--pullimages" option to "quickstart" to explicitely
update docker images.
USERID environment variable was no longer passed to docker image in
development mode.
We take the opportunity to improve the documentation regarding the dev
environment.
Close #177.
We had to backtrack from the latest release of the
android app, which is not compatible with the mobile api v0.5 available
in Hawthorn. This should change in ironwood.
Also, we included the correct oauth client ID in the app, which
prevented communication with the LMS.
The android app is now out of beta \o/
Close #89
- [Minor] Minimum required `click` version is 7.0 (#171)
- [Bugfix] Fix `runserver` dev command (#172)
- [Minor] Fix non-https link to documentation in pypi
- [Minor] Fix `createuser` documentation
Mysql triggers "Host 'xxx' is not allowed to connect to this MySQL
server" error because of race condition.
This was an issue that had been (painfully) solved in v2 and has reared
its ugly head again in v3.
Close #159
Close #160
`./dist/tutor --version` raises an error:
Traceback (most recent call last):
File "main", line 4, in <module>
File "tutor/cli.py", line 21, in main
File "click/core.py", line 764, in __call__
File "click/core.py", line 716, in main
File "click/core.py", line 641, in make_context
File "click/core.py", line 1089, in parse_args
File "click/core.py", line 940, in parse_args
File "click/core.py", line 1477, in handle_parse_result
File "click/core.py", line 96, in invoke_param_callback
File "click/decorators.py", line 270, in callback
RuntimeError: Could not determine version
[1659] Failed to execute script main
To address this, we pass the version value directly to click.
This is for issue #156.
Replace all make commands by a single "tutor" binary. Environment and
data are all moved to ~/.tutor/local/share/tutor. We take the
opportunity to add a web UI and revamp the documentation.
This is a complete rewrite.
Close #121.
Close #147.
