Redis data was not actually persisted, because the redis configuration file was
not mounted from the right location. In order to mount redis data in a
host-mounted directory, the working directory has to be properly set.
The problem was occurring both with docker-compose and Kubernetes.
Close #404.
When tutor is run as root, the `tutor config printroot` and `tutor
config printvalue` commands used to write to stdout, which was causing
issues when combined with `tutor config render` and `tutor local
settheme` commands.
See discussion:
https://discuss.overhang.io/t/error-while-install-indigo-theme/1056
First, allow using custom Django settings on a development
environment (as documented but not implemented), setting it to the
correct value of `tutor.development`. Prior to this, `tutor dev
runserver lms` would default to `tutor.production` when on a custom edX
branch.
Second, fix the documentation so the correct environment variable is
described, at the same time removing an option that doesn't seem to work.
See discussion: https://discuss.overhang.io/t/koa-dev-lms-doesnt-find-static-content/1250
We manage to get unit tests to run in a dedicated openedx-test container. Only
35 tests are failing (out of 17k). I suspect these tests are also failing in
the devstack.
`local exec` was failing with the following stacktrace:
Traceback (most recent call last):
File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 38, in main
cli() # pylint: disable=no-value-for-parameter
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params) File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs) File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 237, in ex
ecute dc_command.callback("exec", args)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 271, in dc_command
volumes, non_volume_args = bindmounts.parse_volumes(args) File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/bindmounts.py", line 73, in parse_vol
umes
context = custom_docker_compose.make_context("custom", args)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 700, in make_context self.parse_args(ctx, args)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1045, in parse_args
opts, args, param_order = parser.parse_args(args=args)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 269, in parse_args
self._process_args_for_options(state)
File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 289, in _process_a
rgs_for_options
arg = state.rargs.pop(0)
AttributeError: 'tuple' object has no attribute 'pop'
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
runserver` commands on Mac OS. (again, see #392)
- [Feature] Automatically bind-mount volumes from the `volumes/` directory with
the `--volume=/...` syntax.
Looks like the `-perm /u=w` argument incorrectly includes files from the
read-only filesystem. This should be fixed by the `-writable` option.
Close #392
This introduces a new dev/local command:
tutor dev bindmount CONTAINER PATH
And a new volume syntax:
tutor dev run --volume=PATH CONTAINER
This syntax automatically bind-mounts folders from the tutorroot/volumes
directory, which is pretty nifty.
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
- The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
- Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
- The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
- The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
- Use Caddy as a web proxy for automated SSL/TLS certificate generation:
- Nginx no longer listens to port 443 for https traffic
- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
- Configuration files for web proxies are no longer provided.
- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
- Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
- Replace memcached and rabbitmq by redis.
- Additional features:
- Make it possible to disable all plugins at once with ``plugins disable all``.
- Add ``tutor k8s wait`` command to wait for a pod to become ready
- Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
- "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
- "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
As discussed in
https://discuss.overhang.io/t/lms-worker-timeout-issue/1057/8, the
timeout of the gunicorn worker is a bit low, causing the worker to
crashloop on slower systems. By increasing the timeout to 120 seconds,
the worker should have time to come up before it gets restarted.
- 💥[Improvement] Remove `dev/local pullimages`. Instead, run `dev/local
dc pull`.
- 💥[Improvement] Add `dev dc` and `local dc` commands as thin wrappers
of the `docker-compose` CLI.
- 💥[Improvement] Remove the undocumented `local run_hook` command.
Instead, run `local init --limit=...`.
- 💥[Improvement] Remove `tutor android pullimage` command. Instead, run
`tutor images pull android`.
- [Bugfix] Fix `config save` on Windows.
- [Bugfix] Fix platform checking in user ID function
- [Bugfix] Fix dependency error during `pip install tutor` due to urllib3 incompatibility
- [Bugfix] Fix user ID checking under Windows.
- [Bugfix] Fix template rendering for Windows users.
- [Improvement] Switch to `bcrypt` for htpasswd password generation, for better portability on Windows.
- [Improvement] In the openedx production docker image, add some jitter to the gunicorn worker restart process to prevent all workers from restarting at the same time.
- [Improvement] Mount config and setting files in read-only mode
- 💥[Improvement] Enable unit completion tracking by default.
- [Bugfix] Run `apt update` before `apt install` when installing deps in
the openedx Dockerfile
- [Improvement] Make all commands considerably faster
- [Improvement] Make it easier to override Mongodb connection parameters
- [Bugfix] Add support for .woff and .woff2 font files in themes (thanks @mrtndwrd!)
Tutor was making many calls to iter_installed (~100 on my machine with a
dozen installed plugins). Turns out it's useless to cache Plugin and
Renderer instances, as the config keeps changing all the time. Instead,
we cache the list of installed plugins, which does not change in the
course of a single run.
On my machine this speeds up `tutor config save` by 5x, going from 7.5s
to 1.3s.
Right now if I add .woff or .woff2 fonts files to an Indigo-based theme's `lms/static/fonts` directory, I get the following error:
```
$ /indigo-folder# make
tutor config render --extra-config ./config-totem.yml ./theme "$(tutor config printroot)/env/build/openedx/themes/indigo-totem"
Error loading template lms/static/fonts/NotoSans-Bold.woff
Traceback (most recent call last):
File "/home/maarten/.local/bin/tutor", line 8, in <module>
sys.exit(main())
File "/home/maarten/.local/lib/python3.8/site-packages/tutor/commands/cli.py", line 38, in main
cli() # pylint: disable=no-value-for-parameter
File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/decorators.py", line 27, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/maarten/.local/lib/python3.8/site-packages/tutor/commands/config.py", line 86, in render
renderer.render_all_to(dst)
File "/home/maarten/.local/lib/python3.8/site-packages/tutor/env.py", line 153, in render_all_to
rendered = self.render_file(template)
File "/home/maarten/.local/lib/python3.8/site-packages/tutor/env.py", line 137, in render_file
template = self.environment.get_template(path)
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 175, in get_source
contents = f.read().decode(self.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb4 in position 10: invalid start byte
make: *** [Makefile:2: render] Error 1
```
CORS headers are no longer handled by nginx, but directly by the LMS.
This makes it possible by 3rd-party apps to easily add domain names to
the CORS whitelist, even when they are not subdomains of the LMS.
- [Feature] Add ``images printtag`` command
- [Improvement] Make it possible to override individual contact email
addresses with plugins
- [Bugfix] Replace "no-reply@LMS_BASE" email address by regular contact
email address
- [Bugfix] Disable learner records globally by default
- [Improvement] Upgrade to the latest release of MySQL 5.6
- [Improvement] Non-plugin settings added by "set" directives are now
automatically removed when the plugin is disabled (#241)
- [Improvement] Add CORS basic configuration to LMS for subdomains of
the LMS
- [Feature] Add support for `images build --add-host` option (thanks
@grinderz!)
- [Bugfix] Fix podman compatibility by replacing `docker-compose rm`
command by `docker-compose stop` when stopping containers
- [Improvement] Improve plugin data deletion
- [Improvement] Introduce the `OPENEDX_COMMON_VERSION` setting
- [Bugfix] Make it possible to run init jobs without starting the entire
platform
- [Improvement] Reduce "openedx" Docker image size with static asset
de-duplication
This is done by explicitely listing job dependencies. Unfortunately,
it's not yet possible to move `init` before `start` in `quickstart`,
because some services, such as discovery, depend on the LMS, which takes
a few seconds to boot up. Actually, discovery also depends on ngins, as
it points to "local.overhang.io" when referring to the lms.
- [Bugfix] Upgrade all services to open-release/juniper.3
- [Bugfix] Fix upload of video transcripts to S3
- [Improvement] Memorize whether the user is running a production
platform during interactive configuration
