Currently there is no way for plugins to customize Kubernetes resources
defined in Tutor deployment manifests.
This change makes that possible by taking advantage of the strategic
merge patching mechanism in `kustomization.yml`.
Any resource definition in a `k8s-override` patch in a plugin will
override the resource defined by Tutor, provided that their names match.
Reference: https://github.com/overhangio/tutor/pull/675
- [Bugfix] Update problem with hint template so it works with newer python versions. (by @mariajgrimaldi)
- [Feature] Add default PYTHONBREAKPOINT to openedx/Dockerfile (by @Carlos-Muniz)
- [Bugfix] Fix smtp server port in `cms.yml` which was causing email sending failures in the Studio. (by @regisb)
- [Bugfix] Skip waiting for MongoDB if it is served using SRV records. (by @gabor-boros)
- [Improvement] Use `git am` instead of `cherry-pick` to simplify patching process.
- [Improvement] Tutor is now compatible with Docker Compose subcommand.
PYTHONBREAKPOINT has been exposed as an environment variable in
the openedx Dockerfile available to be changed in config.yml. The docs have also been changed to recommend using
breakpoint and explaining how PYTHONBREAKPOINT can be modified to use a
custom debugger.
Close https://github.com/overhangio/2u-tutor-adoption/issues/45
Incorrect format of cms.yml config file was causing the following error on course import:
cms-worker_1 | Traceback (most recent call last):
cms-worker_1 | File "/openedx/edx-platform/cms/djangoapps/cms_user_tasks/tasks.py", line 53, in send_task_complete_email
cms-worker_1 | mail.send_mail(subject, message, from_address, [dest_addr], fail_silently=False)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 61, in send_mail
cms-worker_1 | return mail.send()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 284, in send
cms-worker_1 | return self.get_connection(fail_silently).send_messages([self])
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
cms-worker_1 | new_conn_created = self.open()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 62, in open
cms-worker_1 | self.connection = self.connection_class(self.host, self.port, **connection_params)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 255, in __init__
cms-worker_1 | (code, msg) = self.connect(host, port)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 339, in connect
cms-worker_1 | self.sock = self._get_socket(host, port, self.timeout)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 310, in _get_socket
cms-worker_1 | return socket.create_connection((host, port), timeout,
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 787, in create_connection
cms-worker_1 | for res in getaddrinfo(host, port, 0, SOCK_STREAM):
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 918, in getaddrinfo
cms-worker_1 | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cms-worker_1 | socket.gaierror: [Errno -8] Servname not supported for ai_socktype
The reason was that the trailing comma "," was interpreted as being part of the email port.
- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
- 💥 [Feature] Persistent grades are now enabled by default.
- [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
- [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
- [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
Celery workers failed to start in development with the following stacktrace:
cms-worker_1 | Traceback (most recent call last):
cms-worker_1 | File "/openedx/venv/bin/celery", line 8, in <module>
cms-worker_1 | sys.exit(main())
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/__main__.py", line 16, in main
cms-worker_1 | _main()
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 322, in main
cms-worker_1 | cmd.execute_from_commandline(argv)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 499, in execute_from_commandline
cms-worker_1 | super(CeleryCommand, self).execute_from_commandline(argv)))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
cms-worker_1 | return self.handle_argv(self.prog_name, argv[1:])
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 491, in handle_argv
cms-worker_1 | return self.execute(command, argv)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 415, in execute
cms-worker_1 | return cls(
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/worker.py", line 221, in run_from_argv
cms-worker_1 | *self.parse_options(prog_name, argv, command))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 428, in parse_options
cms-worker_1 | self.parser = self.create_parser(prog_name, command)
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 440, in create_parser
cms-worker_1 | description=self._format_description(self.description),
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 462, in _format_description
cms-worker_1 | text.fill_paragraphs(text.dedent(description), width))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/utils/text.py", line 58, in fill_paragraphs
cms-worker_1 | return sep.join(fill(p, width) for p in s.split(sep))
cms-worker_1 | File "/openedx/venv/lib/python3.8/site-packages/celery/utils/text.py", line 58, in <genexpr>
cms-worker_1 | return sep.join(fill(p, width) for p in s.split(sep))
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 391, in fill
cms-worker_1 | return w.fill(text)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 363, in fill
cms-worker_1 | return "\n".join(self.wrap(text))
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 354, in wrap
cms-worker_1 | return self._wrap_chunks(chunks)
cms-worker_1 | File "/opt/pyenv/versions/3.8.12/lib/python3.8/textwrap.py", line 248, in _wrap_chunks
cms-worker_1 | raise ValueError("invalid width %r (must be > 0)" % self.width)
cms-worker_1 | ValueError: invalid width -2 (must be > 0)
This issue was reported upstream here: https://github.com/celery/celery/issues/6302
It is caused by the `tty: true` statement, for some reason. It will be fixed in
Nutmeg, after celery is upgraded to 5.2.6.
Close #681.
I noticed `pip uninstall -y tutor` will not uninstall the plugins, so I made this PR. I know it's ugly, but I don't find any other way of doing it. Let me know if there are better choices 😊
- [Security] Apply logout redirect url security fix. (by @regisb)
- [Feature] Make it possible to force the rendering of a given template, even when the template path matches an ignore pattern. (by @regisb)
- 💥[Fix] Get rid of the `tutor config render` command, which is useless now that themes can be implemented as plugins. (by @regisb)
When rendering theme files in a plugin, the *.scss files are stored in a
"partials" subdirectory, which was ignored by the environment rendering logic.
To render these files, we move the path ignoring logic to a filter, which is a
list of regular expressions. Values in this filter can be overridden by another
filter.
See the corresponding issue in the indigo theme plugin:
https://github.com/overhangio/tutor-indigo/issues/24
- [Fix] Truncate site display name to 50 characters with a warning, fixing data too long error for long site names. (by @navinkarkera)
- [Feature] Add patch to allow overriding final openedx docker image CMD.
- [Fix] Ignore Python plugins that cannot be loaded. (by @regisb)
- [Improvement] Faster and more reliable builds with `npm clean-install` instead of `npm install`. (by @regisb. Thanks @ghassanmas!)
- [Fix] Fix 500 error during studio login. (by @regisb)
- [Fix] Fix updates for the Caddy deployment in multi-node Kubernetes clusters (#660). Previously, Caddy configuration updates might fail if the Kubernetes cluster had more than one worker node. (by @fghaas)
When running multiple concurrent versions of a plugin there are sometimes
version conflicts that prevent the plugin from being loaded. Prior to v1, Tutor
was correctly ignoring plugins that could not be loaded. During the transition
to v1 we lost that feature because we only captured TutorErrors.
Now that Tutor is the official community installation for Open edX, it no
longer makes sense to host a forum that is separate from the general Open edX
forum. Moving conversations there will encourage cross-communication between
projects and maintainers. This change is part of a larger overhaul described in
this Tutor enhancement proposal (TEP):
https://discuss.overhang.io/t/tep-rethinking-the-tutor-maintainers-program/2724
In the future, plugin maintainers should point their users to the Open edX
forum as well. They are encouraged to create dedicated "tutor-pluginnname" tags
on the forum and to set their notification level to "watching".
The default user in the mysql container is 'mysql',
so the `mysql` command tries to use the 'mysql' MySQL user by default.
But, in the MySQL dump instructions,
we are providing the MySQL root user's password, so we need
to specify `MYSQL_ROOT_USERNAME` as the MySQL user when invoking `mysql`.
When a Pod associated with a Deployment is updated (for example, due
to a change to its ConfigMap, or an updated image reference),
Kubernetes uses a ReplicaSet to spin up a Pod with the new
configuration, and once it is up, it tears down the old one.
In case of the Caddy Deployment, this is complicated by the fact that
it uses a Persistent Volume Claim (PVC), whose corresponding volume
uses a Read/Write-Once (RWO) configuration. This means that it can
only be used by multiple Pods if all those Pods all run on the same
Kubernetes worker node.
In order to enable rolling upgrades for the Caddy Deployment, we need
to ensure that its replacement Pod is scheduled on the same node as
the original Pod.
Thus, add a pod affinity rule that will force exactly that behavior.
Reference:
https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
The other Tutor services that use volumes (MySQL, Redis, Elasticsearch
and MongoDB) do not need this fix, since they all use the "Recreate"
deployment strategy: their Pods are all automatically torn down before
being replaced. This strategy is not needed for Caddy, and using a pod
affinity rule is less disruptive to the learner experience.
Commit 514e3fce22 made it so
that dev containers were to load mounts from
env/dev/docker-compose.tmp.yml. However, it did not update
the code to generates the docker-compose.tmp.yml files.
This manifested as mounts simply not working in dev mode.
Additionally, we make the docker-compose.jobs.tmp.yml files
follow the same local vs dev differentiation that
was introduced in 514e3fce22.
With this change, we want to better highlight the contributions of
developers to Tutor. We want to publicly acknowledge the positive impact
that individuals and companies have on the development of the platform.
to that end, each changelog entry can now be suffixed with the name of
the author (individual or company) who authored the change. These names
will find their way to the release notes for every release. Eventually,
we also want to spread these release notes more widely. For instance, we
could post new releases to the forum to notify the community of
important changes.
If you have contributed to Tutor in the past, feel free to open a PR and
append your name to the changes that you made. We will not be able to
update the release notes for every release out there, but your
contributions will be acknowledged from the changelog.
When mounting a directory in a dev-only container, such as the
"learning" mfe, docker-compose is failing because it is attempting to
run "docker-compose stop" in the local context -- which knows nothing
about the learning container.
To resolve this, we store tmp volumes either in the local or dev
docker-compose.yml, and load either one depending on the context.
- [Improvement] Add the `COMPOSE_PROJECT_STARTED` action and run `dev
stop` on `local start` (and vice versa).
- [Feature] Introduce `local/dev copyfrom` command to copy contents from
a container.
- [Bugfix] Fix a race condition that could prevent a newly provisioned
LMS container from starting due to a `FileExistsError` when creating
data folders.
- [Deprecation] Mark `tutor dev runserver` as deprecated in favor of
`tutor dev start`. Since `start` now supports bind-mounting and
breakpoint debugging, `runserver` is redundant and will be removed in a
future release.
- [Improvement] Allow breakpoint debugging when attached to a service
via `tutor dev start SERVICE`.
- [Security] Apply rate limiting security fix (see
[commit](b5723e416e)).
- [Feature] Introduce the ``-m/--mount`` option in ``local`` and ``dev``
commands to auto-magically bind-mount folders from the host.
- [Feature] Add `tutor dev quickstart` command, which is similar to
`tutor local quickstart`, except that it uses dev containers instead
of local production ones and includes some other small differences for
the convience of Open edX developers. This should remove some friction
from the Open edX development setup process, which previously required
that users provision using local producation containers (`tutor local
quickstart`) but then stop them and switch to dev containers (`tutor
local stop && tutor dev start -d`).
- 💥[Improvement] Make it possible to run `tutor k8s exec <command with
multiple arguments>` (#636). As a consequence, it is no longer
possible to run quoted commands: `tutor k8s exec "<some command>"`.
Instead, you should remove the quotes: `tutor k8s exec <some command>`.
- 💥[Deprecation] Drop support for the `TUTOR_EDX_PLATFORM_SETTINGS`
environment variable. It is now recommended to create a plugin
instead.
- 💥[Improvement] Complete overhaul of the plugin extension mechanism.
Tutor now has a hook-based Python API: actions can be triggered at
different points of the application life cycle and data can be modified
thanks to custom filters. The v0 plugin API is still supported, for
backward compatibility, but plugin developers are encouraged to migrate
their plugins to the new API. See the new plugin tutorial for more
information.
- [Improvement] Improved the output of `tutor plugins list`.
- [Feature] Add `tutor [dev|local|k8s] status` command, which provides
basic information about the platform's status.
Github release CI was running on ubuntu 18.04 withh python 3.6.
Installing tomli==2.0.1, which is required in dev, triggers a failure in
python 3.6 because it is no longer available.
