- [Feature] Add the `-m/--mount` option to `tutor dev quickstart`.
- [Bugfix] Fix `tutor dev start -m /path/to/frontend-app-learning` by introducing dev-specific `COMPOSE_DEV_TMP` and `COMPOSE_DEV_JOBS_TMP` filters (by @regisb).
- [Bugfix] Log the shell commands that Tutor executes more accurately. (by @kdmccormick)
- [Bugfix] `tutor dev quickstart` would fail under certain versions of docker-compose due to a bug in the logic that handled volume mounting. (by @kdmccormick)
- [Bugfix] The `tutor k8s start` command will succeed even when `k8s-override` and `kustomization-patches-strategic-merge` are not specified. (by @edazzocaisser)
- [BugFix] `kubectl wait` checks deployments instead of pods as it could hang indefinitely if there are extra pods in a broken state. (by @keithgg)
The -m/--mount option makes it possible to bind-mount volumes at runtime. The
volumes are declared in a local/docker-compose.tmp.yml file. The problem with
this approach is when we want to bind-mount a volume to a service which is
specific to the dev context. For instance: the "learning" service when the MFE
plugin is enabled.
In such a case, starting the service triggers a call to `docker-compose stop`
in the local context. This call fails because the "learning" service does not
exist in the local context. Note that this issue only seems to occur with
docker-compose v1.
To resolve this issue, we create two additional filters for
the dev context, which emulate the behaviour of the local context. With this approach, we convert the -m/--mount arguments right after they are parsed. Because they are parsed just once, we can get rid of the de-duplication logic initially introduced with the COMPOSE_CLI_MOUNTS context.
Close #711. Close also https://github.com/overhangio/tutor-mfe/issues/57.
Whenever Tutor executes a shell command, it logs out said
command in order to aid in end user understanding/debugging.
In some cases (notably, when running jobs in containers)
the logged command was not accurately quoted. The command
was run correctly, because it was passed in pieces to
``subprocess.Popen``, which correctly joins the pieces together
into a valid POSIX shell command; however, the logged version
of the command was constructed by simply joining the pieces
with spaces. This usually works, but breaks down when running
complex shell commands with nested quoting.
This commit changes the logging to use ``shlex.join``, which
joins command pieces together in a POSIX-compliant way,
presumably the same way as ``subprocess.Popen``.
Example:
tutor local importdemocourse
runs the shell command:
docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c 'echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup'
but the logged shell command was:
docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup
which will not run if copied and pasted back into the
user's terminal, as the importdemocourse shell script is unquoted.
When waiting for pods, it's possible that the deployment may be
complete but, because other pods may have been Evicted or Killed, the
wait wait condition completes.
In certain code paths, such as in `tutor local quickstart`,
`process_mount_points` is called more than once in the same process,
causing mounts to be added to `COMPOSE_LOCAL[_JOBS]_TMP` redundantly.
As a result, docker-compose[.jobs].tmp.yml was occasionally being
rendered with duplicate volume specifiers. Some versions of Docker
Compose ignored this; other versions warned or threw an error.
In order to make `process_mount_points` tolerant to being called
multiple times, we wrap its volume-adding callbacks within a new
hooks context. This allows us to clear said hooks context every
time `process_mount_points` is called, essentially making the
function idempotent.
Co-authored-by: Régis Behmo <regis@behmo.com>
- [Bugfix] Build openedx-dev Docker image even when the host user is root, for instance on Windows. (by @regisb)
- [Bugfix] Patch nutmeg.1 release with [LTI 1.3 fix](https://github.com/openedx/edx-platform/pull/30716). (by @ormsbee)
- [Improvement] Make it possible to override k8s resources in plugins using `k8s-override` patch. (by @foadlind)
Sometimes, the host user is root: this may happen when tutor is run with
"sudo" (which is not recommended) or on Windows. In such cases, building
the image should not fail, but default to a reasonable user. Also, when
we pass an invalid APP_USER_ID as a build arg, then we should fail with
an explicit message.
See this conversation:
https://discuss.overhang.io/t/problem-with-dev-image-build-useradd-uid-0-is-not-unique/2406
Currently there is no way for plugins to customize Kubernetes resources
defined in Tutor deployment manifests.
This change makes that possible by taking advantage of the strategic
merge patching mechanism in `kustomization.yml`.
Any resource definition in a `k8s-override` patch in a plugin will
override the resource defined by Tutor, provided that their names match.
Reference: https://github.com/overhangio/tutor/pull/675
- [Bugfix] Update problem with hint template so it works with newer python versions. (by @mariajgrimaldi)
- [Feature] Add default PYTHONBREAKPOINT to openedx/Dockerfile (by @Carlos-Muniz)
- [Bugfix] Fix smtp server port in `cms.yml` which was causing email sending failures in the Studio. (by @regisb)
- [Bugfix] Skip waiting for MongoDB if it is served using SRV records. (by @gabor-boros)
- [Improvement] Use `git am` instead of `cherry-pick` to simplify patching process.
- [Improvement] Tutor is now compatible with Docker Compose subcommand.
PYTHONBREAKPOINT has been exposed as an environment variable in
the openedx Dockerfile available to be changed in config.yml. The docs have also been changed to recommend using
breakpoint and explaining how PYTHONBREAKPOINT can be modified to use a
custom debugger.
Close https://github.com/overhangio/2u-tutor-adoption/issues/45
Incorrect format of cms.yml config file was causing the following error on course import:
cms-worker_1 | Traceback (most recent call last):
cms-worker_1 | File "/openedx/edx-platform/cms/djangoapps/cms_user_tasks/tasks.py", line 53, in send_task_complete_email
cms-worker_1 | mail.send_mail(subject, message, from_address, [dest_addr], fail_silently=False)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 61, in send_mail
cms-worker_1 | return mail.send()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 284, in send
cms-worker_1 | return self.get_connection(fail_silently).send_messages([self])
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
cms-worker_1 | new_conn_created = self.open()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 62, in open
cms-worker_1 | self.connection = self.connection_class(self.host, self.port, **connection_params)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 255, in __init__
cms-worker_1 | (code, msg) = self.connect(host, port)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 339, in connect
cms-worker_1 | self.sock = self._get_socket(host, port, self.timeout)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 310, in _get_socket
cms-worker_1 | return socket.create_connection((host, port), timeout,
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 787, in create_connection
cms-worker_1 | for res in getaddrinfo(host, port, 0, SOCK_STREAM):
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 918, in getaddrinfo
cms-worker_1 | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cms-worker_1 | socket.gaierror: [Errno -8] Servname not supported for ai_socktype
The reason was that the trailing comma "," was interpreted as being part of the email port.
- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
- 💥 [Feature] Persistent grades are now enabled by default.
- [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
- [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
- [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
Celery workers failed to start in development with the following stacktrace:
cms-worker_1 | Traceback (most recent call last):
cms-worker_1 | File "/openedx/venv/bin/celery", line 8, in <module>
cms-worker_1 | sys.exit(main())
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/__main__.py", line 16, in main
cms-worker_1 | _main()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 322, in main
cms-worker_1 | cmd.execute_from_commandline(argv)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 499, in execute_from_commandline
cms-worker_1 | super(CeleryCommand, self).execute_from_commandline(argv)))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
cms-worker_1 | return self.handle_argv(self.prog_name, argv[1:])
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 491, in handle_argv
cms-worker_1 | return self.execute(command, argv)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 415, in execute
cms-worker_1 | return cls(
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/worker.py", line 221, in run_from_argv
cms-worker_1 | *self.parse_options(prog_name, argv, command))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 428, in parse_options
cms-worker_1 | self.parser = self.create_parser(prog_name, command)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 440, in create_parser
cms-worker_1 | description=self._format_description(self.description),
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 462, in _format_description
cms-worker_1 | text.fill_paragraphs(text.dedent(description), width))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/utils/text.py", line 58, in fill_paragraphs
cms-worker_1 | return sep.join(fill(p, width) for p in s.split(sep))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/utils/text.py", line 58, in <genexpr>
cms-worker_1 | return sep.join(fill(p, width) for p in s.split(sep))
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 391, in fill
cms-worker_1 | return w.fill(text)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 363, in fill
cms-worker_1 | return "\n".join(self.wrap(text))
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 354, in wrap
cms-worker_1 | return self._wrap_chunks(chunks)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 248, in _wrap_chunks
cms-worker_1 | raise ValueError("invalid width %r (must be > 0)" % self.width)
cms-worker_1 | ValueError: invalid width -2 (must be > 0)
This issue was reported upstream here: https://github.com/celery/celery/issues/6302
It is caused by the `tty: true` statement, for some reason. It will be fixed in
Nutmeg, after celery is upgraded to 5.2.6.
Close #681.
- [Security] Apply logout redirect url security fix. (by @regisb)
- [Feature] Make it possible to force the rendering of a given template, even when the template path matches an ignore pattern. (by @regisb)
- 💥[Fix] Get rid of the `tutor config render` command, which is useless now that themes can be implemented as plugins. (by @regisb)
When rendering theme files in a plugin, the *.scss files are stored in a
"partials" subdirectory, which was ignored by the environment rendering logic.
To render these files, we move the path ignoring logic to a filter, which is a
list of regular expressions. Values in this filter can be overridden by another
filter.
See the corresponding issue in the indigo theme plugin:
https://github.com/overhangio/tutor-indigo/issues/24
- [Fix] Truncate site display name to 50 characters with a warning, fixing data too long error for long site names. (by @navinkarkera)
- [Feature] Add patch to allow overriding final openedx docker image CMD.
- [Fix] Ignore Python plugins that cannot be loaded. (by @regisb)
- [Improvement] Faster and more reliable builds with `npm clean-install` instead of `npm install`. (by @regisb. Thanks @ghassanmas!)
- [Fix] Fix 500 error during studio login. (by @regisb)
- [Fix] Fix updates for the Caddy deployment in multi-node Kubernetes clusters (#660). Previously, Caddy configuration updates might fail if the Kubernetes cluster had more than one worker node. (by @fghaas)
When running multiple concurrent versions of a plugin there are sometimes
version conflicts that prevent the plugin from being loaded. Prior to v1, Tutor
was correctly ignoring plugins that could not be loaded. During the transition
to v1 we lost that feature because we only captured TutorErrors.
When a Pod associated with a Deployment is updated (for example, due
to a change to its ConfigMap, or an updated image reference),
Kubernetes uses a ReplicaSet to spin up a Pod with the new
configuration, and once it is up, it tears down the old one.
In case of the Caddy Deployment, this is complicated by the fact that
it uses a Persistent Volume Claim (PVC), whose corresponding volume
uses a Read/Write-Once (RWO) configuration. This means that it can
only be used by multiple Pods if all those Pods all run on the same
Kubernetes worker node.
In order to enable rolling upgrades for the Caddy Deployment, we need
to ensure that its replacement Pod is scheduled on the same node as
the original Pod.
Thus, add a pod affinity rule that will force exactly that behavior.
Reference:
https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
The other Tutor services that use volumes (MySQL, Redis, Elasticsearch
and MongoDB) do not need this fix, since they all use the "Recreate"
deployment strategy: their Pods are all automatically torn down before
being replaced. This strategy is not needed for Caddy, and using a pod
affinity rule is less disruptive to the learner experience.
With this change, we want to better highlight the contributions of
developers to Tutor. We want to publicly acknowledge the positive impact
that individuals and companies have on the development of the platform.
to that end, each changelog entry can now be suffixed with the name of
the author (individual or company) who authored the change. These names
will find their way to the release notes for every release. Eventually,
we also want to spread these release notes more widely. For instance, we
could post new releases to the forum to notify the community of
important changes.
If you have contributed to Tutor in the past, feel free to open a PR and
append your name to the changes that you made. We will not be able to
update the release notes for every release out there, but your
contributions will be acknowledged from the changelog.
When mounting a directory in a dev-only container, such as the
"learning" mfe, docker-compose is failing because it is attempting to
run "docker-compose stop" in the local context -- which knows nothing
about the learning container.
To resolve this, we store tmp volumes either in the local or dev
docker-compose.yml, and load either one depending on the context.
- [Improvement] Add the `COMPOSE_PROJECT_STARTED` action and run `dev
stop` on `local start` (and vice versa).
- [Feature] Introduce `local/dev copyfrom` command to copy contents from
a container.
- [Bugfix] Fix a race condition that could prevent a newly provisioned
LMS container from starting due to a `FileExistsError` when creating
data folders.
- [Deprecation] Mark `tutor dev runserver` as deprecated in favor of
`tutor dev start`. Since `start` now supports bind-mounting and
breakpoint debugging, `runserver` is redundant and will be removed in a
future release.
- [Improvement] Allow breakpoint debugging when attached to a service
via `tutor dev start SERVICE`.
- [Security] Apply rate limiting security fix (see
[commit](b5723e416e)).
- [Feature] Introduce the ``-m/--mount`` option in ``local`` and ``dev``
commands to auto-magically bind-mount folders from the host.
- [Feature] Add `tutor dev quickstart` command, which is similar to
`tutor local quickstart`, except that it uses dev containers instead
of local production ones and includes some other small differences for
the convience of Open edX developers. This should remove some friction
from the Open edX development setup process, which previously required
that users provision using local producation containers (`tutor local
quickstart`) but then stop them and switch to dev containers (`tutor
local stop && tutor dev start -d`).
- 💥[Improvement] Make it possible to run `tutor k8s exec <command with
multiple arguments>` (#636). As a consequence, it is no longer
possible to run quoted commands: `tutor k8s exec "<some command>"`.
Instead, you should remove the quotes: `tutor k8s exec <some command>`.
- 💥[Deprecation] Drop support for the `TUTOR_EDX_PLATFORM_SETTINGS`
environment variable. It is now recommended to create a plugin
instead.
- 💥[Improvement] Complete overhaul of the plugin extension mechanism.
Tutor now has a hook-based Python API: actions can be triggered at
different points of the application life cycle and data can be modified
thanks to custom filters. The v0 plugin API is still supported, for
backward compatibility, but plugin developers are encouraged to migrate
their plugins to the new API. See the new plugin tutorial for more
information.
- [Improvement] Improved the output of `tutor plugins list`.
- [Feature] Add `tutor [dev|local|k8s] status` command, which provides
basic information about the platform's status.
Running `local start` while a dev platform is still running is a common sourse
of mistakes. Here we introduce a new action to automatically stop local and dev
projects whenever a project with a different name is started.
`copyfrom` copies data from a container to the local filesystem. It's similar
to bindmount, but less clunky, and more intuitive. Also, it plays along great
with `--mount`. Eventually we'll just get rid of the `bindmount` command and
the `--volume` option.
`tutor dev runserver` will be removed in a future release.
Developers are encouraged to use `tutor dev start` instead,
which is more flexible and provides a consistent interface
with `tutor local start`.
As part of this deprecation, we enable the `tty` and
`stdin_open` options on development docker-compose
services. This will allow developers to use `start`
for breakpoint debugging, which was previously only
availble via `runserver`. Several parallel PRs have
been merged in order to make the same change in the
development services of the official plugins.
Although `start` does not support the `--volume` option,
it supports a more-powerful `--mount` option. So, where
developers previously used:
tutor dev runserver --volume ...
to bind-mount host directories, they should now use:
tutor dev start --mount ...
Resolves https://github.com/overhangio/2u-tutor-adoption/issues/61
The `--mount` option is available both with `tutor local`
and `tutor dev` commands. It allows users to easily bind-mount containers from
the host to containers. Yes, I know, we already provide that possibility with
the `bindmount` command and the `--volume=/path/` option. But these suffer from
the following drawbacks:
- They are difficult to understand.
- The "bindmount" command name does not make much sense.
- It's not convenient to mount an arbitrary folder from the host to multiple
containers, such as the many lms/cms containers (web apps, celery workers and
job runners).
To address this situation, we now recommend to make use of --mount:
1. `--mount=service1[,service2,...]:/host/path:/container/path`: manually mount
`/host/path` to `/container/path` in container "service1" (and "service2").
2. `--mount=/host/path`: use the new v1 plugin API to discover plugins that
will detect this option and select the right containers in which to bind-mount
volumes. This is really nifty...
Close https://github.com/overhangio/2u-tutor-adoption/issues/43
Add `tutor dev quickstart` command, which is equivalent to
`tutor local quickstart`, but uses dev containers instead
of local production ones and includes some other small
differences for the convience of Open edX developers.
This should remove some friction
from the Open edX development setup process, which previously
required that users provision using local producation
containers but then stop them and switch to dev containers:
* tutor local quickstart
* tutor local stop
* tutor dev start -d
Document the command and its improved workflow in
./docs/tutorials/nightly.rst
Fixes overhangio/2u-tutor-adoption#58
The entrypoint in the "openedx" Docker image was used only to define the
DJANGO_SETTINGS_MODULE environment variable, based on SERVICE_VARIANT and
SETTINGS. We ditch SETTINGS in favour of defining explicitely
DJANGO_SETTINGS_MODULE.
The problem with the Docker entrypoint is that it was bypassed whenever we ran
`tutor local exec` or `tutor k8s exec`. By removing it we make it simpler for
end-users to run manage.py commands in kubernetes.
This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.
Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.
This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.
The plugin v0 API remains supported for backward compatibility of
existing plugins.
Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority
Close #499.
Previously, the `k8s exec` command did not support unknown "--options". This
made it impossible to launch, say, a django shell in the lms container.
While implementing this feature we saw an opportunity to simplify the way jobs
are handled in the k8s commands.
Close #636.
Another related issue is: https://github.com/overhangio/2u-tutor-adoption/issues/52
- [Security] Apply SAML security fix.
- [Improvement] In addition to the Docker build arguments
`EDX_PLATFORM_REPOSITORY` and `NPM_REGISTRY`, also support two corresponding
and identically-named `config.yml` values serving the same purpose.
Previously, the only way for Tutor users to use a fork of edx-platform
or a custom NPM registry was to use build args during the image build.
This is suboptimal in the case of automatically building images from
CI pipelines, which may want to auto-detect when an image needs to be
rebuilt based on config.yml changes.
In addition, the EDX_PLATFORM_VERSION build argument can already be
set via a corresponding config.yml parameter (OPENEDX_COMMON_VERSION),
so it's reasonable to follow that precedent and also introduce
config.yml parameters to correspond with the EDX_PLATFORM_REPOSITORY
and NPM_REGISTRY build arguments.
Thus, introduce two new configuration parameters:
- EDX_PLATFORM_REPOSITORY
- NPM_REGISTRY
These parameters can now optionally be used instead of the
aforementioned build args.
- [Bugfix] Fix dockerize on arm64 by switching to the [powerman/dockerize](https://github.com/powerman/dockerize) fork (#591).
- [Bugfix] Fix "Unexpected args" error during service initialization on Kubernetes (#611).
The version of dockerize that shipped with the "openedx" image was not
compatible with arm64. The original project is unmaintained, but there
is a fork that provides a version that is compatible with arm64.
This was tested on arm64 with buildx:
docker buildx build --tag=openedx --platform=linux/arm64 ~/.local/share/tutor/env/build/openedx
Close #591
- [Bugfix] Fix `local/k8s quickstart` commands when upgrading from an older release (#595).
- [Bugfix] Fix running the default exim-relay SMTP server on arm64 (#600).
- [Feature] Add `tutor k8s apply` comand, which is a direct interface with `kubectl apply`.
- [Feature] Add `openedx-dockerfile-minimal` patch, which you can use to install custom packages and run commands as root in the Docker image.
- [Security] Fix vulnerability in call to invalid enrollment API (see [commit](e9369cffde)).
- [Bugfix] Fix "Internal Server Error / AttributeError / object has no attribute 'get_metadata'" in learning MFE.
- [Improvement] Replace all links to github.com/edx by github.com/openedx, following the migration of all repositories.
- [Bugfix] Fix `k8s start caddy` command.
- [Bugfix] Fix authentication in development due to missing SameSite policy on session ID cookie.
- [Bugfix] Display properly themed favicon.ico image in LMS, Studio and microfrontends.
- [Bugfix] Fix "LazyStaticAbsoluteUrl is not JSON serializable" error when sending bulk emails.
- [Bugfix] Fix `tutor local importdemocourse` fails when platform is not up.
In development, it was no longer possible to authenticate to the lms. Ater
signing in, the session ID could not be dropped, and thus the user was not
signed in, although no error was logged -- just a warning in the browser
console.
This problem was caused by the fact that the SameSite policy was set to "None"
in development.
Previously, we were redirecting all /*favicon.ico requests to the default
favicon. This meant that the favicon might not necessarily be correctly themed,
most notably in MFEs. Here, we resolve this issue by redirecting to the
theme-agnostic theming/asset/* url. Also, we restrict the overly generic regexp
for favicon url matching. We verified that we did not miss any url by running
the following command on the demo server:
tutor local logs caddy | grep --only-matching "host.*favicon.ico" | sort | uniq
The LazyStaticAbsoluteUrl object was breaking bulk emails again with the
following stacktrace:
2022-01-11 13:50:10,591 ERROR 12 [celery.app.trace] [user None] [ip None] trace.py:255 - Task lms.djangoapps.instructor_task.tasks.send_bulk_course_email[26b93357-018a-408f-b3f7-b69722447c5b] raised unexpected: EncodeError(TypeError('Object of type LazyStaticAbsoluteUrl is not JSON serializable'))
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/kombu/serialization.py", line 50, in _reraise_errors
yield
File "/openedx/venv/lib/python3.8/site-packages/kombu/serialization.py", line 221, in dumps
payload = encoder(data)
File "/openedx/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 69, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/openedx/venv/lib/python3.8/site-packages/simplejson/__init__.py", line 398, in dumps
return cls(
File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 296, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 378, in iterencode
return _iterencode(o, 0)
File "/openedx/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 59, in default
return super(JSONEncoder, self).default(o)
File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 272, in default
raise TypeError('Object of type %s is not JSON serializable' %
TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable
The point of that lazy object was to link to the lms logo even when a custom
theme was enabled. Luckily, we no longer need this lazy evaluation because we
now have theme-agnostic urls that point to static asset (see
https://github.com/openedx/edx-platform/pull/29461).
See:
https://discuss.overhang.io/t/error-while-sending-bulk-emails-lazystaticabsoluteurl-is-not-json-serializable/2176/
upgrading.
- [Bugfix] During upgrade, make sure that environment is up-to-date
prior to prompting to rebuild the custom images.
- [Bugfix] Fix ownership of mysql data, in particular when upgrading a
Kubernetes cluster to Maple.
- [Bugfix] Ensure that ``tutor k8s upgrade`` is run during ``tutor k8s
quickstart``, when necessary.
- 💥[Bugfix] By default, detect the current version during ``tutor
k8s/local upgrade``.
- [Bugfix] Fix upgrading from Lilac to Maple on Kubernetes by deleting
deployments and services.
`upgrade` had several issues, which are summarized here:
https://discuss.overhang.io/t/confusing-instructions-during-upgrade/2281/7
- The docs say that you should run quickstart, but what most people will see is
the big command tutor local upgrade --from=lilac verbatim paragraph.
- The local upgrade command should be very explicit about the fact that users
need to run quickstart.
- Maybe the name of the local upgrade command should be improved.
- When upgrading tutor from one major release to the next, there should be a
more explicit warning to inform users of what they are doing (see this other
conversation 1)
- We should tell people that they almost certainly need to enable the tutor and
the mfe plugins, if they are not enabled during upgrade.
- A link to all of the breaking changes from the changelog should be
prominently displayed during upgrade.
- The docs should emphasize that upgrading from one major release to the next
is potentially a risky endeavor and that downgrading is not possible. The docs
should also link to the changelog.
This commit has grown slightly beyond the intended scope, but the changes should be mostly positive.
In theory, we can assign ownership of mysql data to just any user. But in
Lilac, mysql was running with user 999. When upgrading to Maple, on Kubernetes,
the fsGroupChangePolicy was causing a change of the data *group* (to 1000) but
not of the user. This was causing a crash with the following error:
[ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
When upgrading from Lilac, all services break with the following error:
Service "***" is invalid: spec.ports[0].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'
Upgrading deployments fails as well:
Deployment.apps "***" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"openedx-********", "app.kubernetes.io/managed-by":"tutor", "app.kubernetes.io/name":"***", "app.kubernetes.io/part-of":"openedx"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
That's because deployments and services need to be deleted as part of the Maple
upgrade. So that's what we do as part of `tutor k8s upgrade --from=lilac`. And
we take the opportunity to:
1. Run upgrade as part of quickstart, when necessary.
2. Default to lilac during `tutor k8s upgrade`.
Close #551.
- [Security] Upgrade Django to 3.2.11 in edx-platform.
- [Security] Prevent non-staff users from searching usernames by email by
abusing the logout url.
Without this patch, it is possible to search for account info including
username by using the email of a learner. This fix disallows searching using
email by regular users and restricts this feature to only staff and superusers.
Régis Behmo
Kyle McCormick
Keith Grootboom
Ed D'Azzo
David Ormsbee
Foad Lind
Gabor Boros
Maria Grimaldi
Carlos Muniz
iamcristye@outlook.com
ibrmora
Alejandro Cardenas
Navin Karkera
Daniel Rudnitski
Florian Haas
Braden MacDonald
Kyle McCormick
Florian Haas
Rebecca Graber
Emad Rad