<rant>I attempted to actually run Tutor with Podman and I was sorely disappointed.
The only reliable source of docs that I found concerning the integration with
docker-compose is this blog post:
https://www.redhat.com/sysadmin/podman-docker-compose
There are no other official docs 😓
1. The instructions given in the blog post don't work out of the box. Launching
the podman service failed altogether on Ubuntu 20.04 and 20.10. It worked on
CentOS 8, but some parameters need to changed, such as the docker socket path.
2. After I got the podman service working, I managed to get an Open edX
platform running with tutor, but with the root user. Then, containers
complained that they could not write data to the bind-mounted volumes. I
attempted to run as a non-root user, and discovered that the podman socket is
only readable by root. This should explain why all commands from that blog post
are prefixed by sudo.
Long story short, I was hoping to update the tutorial. Instead, I'm just moving
it for the sake of better organisation. For the life of me, I do not understand
why some people would want to run Podman instead of Docker. Bad documentation
is an immediate turn-off for me. From my perspective, podman is mostly an
overblown marketina stunt.</rant>
There is too much information in each of the local/k8s/dev docs pages. The
"guides" that are listed in each one of those pages are moved either to "common
tasks" or to a dedicated "tutorials" section. This paves the way for more
comprehensive tutorials, where we describe how to run the latest master
branches of Open edX.
I am well aware that, as they stand, the tutorials are of poor quality and
should be rewritten. This is a task for another day/commit. For now, we only
move the contents to a separate part of the docs.
Also, we should add a "reference" section to the docs, where we add the result
of `tutor <subcommand> --help`.
Previously, the list of domain names to which a theme was assigned had to be
specified manually. Now, the themes are automatically assigned to the LMS and
the CMS, both in development and production modes.
See: https://discuss.overhang.io/t/no-activation-email-errors-logged-on-user-sign-up/1969
A 500 error was being triggered during user registration.
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 485, in dispatch
return super().dispatch(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/ratelimit/decorators.py", line 24, in _wrapped
return fn(request, *args, **kw)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 529, in post
response, user = self._create_account(request, data)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 572, in _create_account
user = create_account_with_params(request, data)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 236, in create_account_with_params
compose_and_send_activation_email(user, profile, registration)
File "/openedx/edx-platform/common/djangoapps/student/views/management.py", line 214, in compose_and_send_activation_email
send_activation_email.delay(str(msg))
File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 29, in __str__
return json.dumps(self, cls=MessageEncoder)
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 119, in default
return super().default(o) # pragma: no cover
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable
2021-09-28 05:21:52,174 ERROR 122 [django.request] [user 11] [ip XY.XY.XY.XY] log.py:222 - Internal Server Error: /api/user/v2/account/registration/
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 485, in dispatch
return super().dispatch(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/ratelimit/decorators.py", line 24, in _wrapped
return fn(request, *args, **kw)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 529, in post
response, user = self._create_account(request, data)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 572, in _create_account
user = create_account_with_params(request, data)
File "./openedx/core/djangoapps/user_authn/views/register.py", line 236, in create_account_with_params
compose_and_send_activation_email(user, profile, registration)
File "/openedx/edx-platform/common/djangoapps/student/views/management.py", line 214, in compose_and_send_activation_email
send_activation_email.delay(str(msg))
File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 29, in __str__
return json.dumps(self, cls=MessageEncoder)
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 119, in default
return super().default(o) # pragma: no cover
File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable
The reason for that was that edx-ace could not json-serialize the context to be
passed to the registration email renderer. That was caused by the
LazyStaticAbsoluteUrl object created to address missing logo in registration
email. To make sure that this object is serializable by
edx_ace.serialization.MessageEncoder, we add a trivial .to_json() method to the
LazyStaticAbsoluteUrl class.
This error could (at first) not be reproduced in development, because
AUTOMATIC_AUTH_FOR_TESTING is set to true in devstack settings.
When upgrading mongodb, the mongodb container takes a little while to become
ready. Running the "exec" command thus triggers an error:
docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setF
eatureCompatibilityVersion: "4.0" })
MongoDB shell version v4.0.24
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
2021-06-14T10:53:21.510+0000 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused:
connect@src/mongo/shell/mongo.js:356:17
@(connect):2:6
exception: connect failed
Error: Command failed with status 1: docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setFeatureCompatibilityVersion: "4.0" })
We add a "sleep" statement to the upgrade process to ensure that the mongodb
container is available.
Turns out, the authentication mechanism should only be defined if there
is an actual authentication. For now, because of the urgency, we
hardcode this auth_mech to ":scram". We'll add a way to override it if
necessary, in the future.
- [Improvement] Make it easier to run edx-platform unit tests.
- [Bugfix] Fix segmentation fault during `tutor config save` on Mac OS M1 (#473). Thanks @ghassanmas!
- [Bugfix] Fix a bug that prevented connecting to external MongoDB instances.
- [Improvement] Make sure that the logo included in email notifications (including discussion responses) is the same as the site logo.
- [Bugfix] Install IPython directly from pypi instead of installing it from source (the reason it was installed from source is no longer relevant). The effect of this shall speed up the process of building the openedx-dev Docker image.
- [Feature] Add "openedx-dockerfile-post-git-checkout" patch.
- [Improvement] In the "openedx" Docker images, convert git patches to cherry-picks for a cleaner source tree.
- 💥[Feature] Make it possible to override local job configuration. This deprecates the older model for running jobs which dates back from a long time ago.
It should be unnecessary to build a custom openedx-dev Docker image. All tests
can run from within the dev Docker image, with a couple additional environment
variables.
Previously, the logo included in emails was loaded from edX' CDN. Here, we make
sure that the logo is actually the same as the site logo. Because the logo may
be theme-specific, we need to compute the logo url at runtime, and use a
lazily-evaluated string.
Close #447.
Change the source of installation Ipyton, install it from PyPy instead of source, which shall speed up the process of building the docker image for openedx-dev
With "git patch", the resulting source tree was dirty, showing uncommitted
changes. Here, we replace "git patch" with "git cherry-pick". We avoid pulling
the entire remote repo by fetching individual commits. To do that, we need to
assign an identity to the git user.
- [Improvement] Upgrade all services to open-release/lilac.2.
- [Bugfix] Fix "`sh` is not a recognized command" in some plugins, including minio.
- [Improvement] Set the default contact mailing email address
- [Bugfix] Fix minio initialisation in Kubernetes.
- [Bugfix] Fix "Invalid command argument" during upgrade from Koa to Lilac.
- [Bugfix] Fix mysql initialisation in docker-compose==2.0.0beta4.
- [Improvement] Tutor is now published on pypi as "tutor".
Previously, the tutor-openedx package was loading tons of template data from
the MANIFEST.in. Turns out, we cannot ignore the MANIFEST.in file with
setuptools. So we need to move tutor-openedx to a separate, dedicated folder.
To auto-discover the package version, we copy it at runtime (in the make
command).
An issue with the latest release of docker-compose was reported here:
https://discuss.overhang.io/t/undefined-entrypoint-throws-error-in-docker-compose-2-0-0-beta-4/1716
The mysql-job definition had an empty entrypoint (`[]`). This was causing the following error:
the initiation of mysql fails with “services.mysql-job.entrypoint must be a string …
Error: Command failed with status 15”
I can't remember at all why we had to define an empty entrypoint. It probably
has to do with the fact that we could not run `sh -e -c "..."` commands in
mysql jobs. Similarly, the k8s job definition sets `command: []`. I tested both
local and k8s deployments without these definitions and they work just fine. So
I guess we can get rid of them.
The package maintainer of the "tutor" package was kind enough to
transfer ownership of the project to us. This is great, because we no
longer have to use the "openedx" suffix, which is trademarked.
For the time being, we keep maintaining the "tutor-openedx" package
which has a 1-to-1 dependency on the "tutor" package. In the future, we
expect that we will no longer push upgrades to tutor-openedx.
Here we add to the docs a few shameless plugs about Cairn -- because
it's really awesome!
We also add a few improvements to the wording, here and there.
- [Bugfix] Fix double pulling mongodb image when upgrading from Koa to Lilac.
- [Improvement] Better logging during `plugins disable`.
- [Bugfix] Fix "upstream sent too big header" error during login of existing users after a Koa to Lilac upgrade.
- [Feature] Added the ability to skip `config.yml` file modification while running `tutor config save` command with `-e` or `--env-only` flag.
- [Feature] Add new config value `FORUM_MONGODB_DATABASE` to set the forum database name
In config.yml the new value FORUM_MONGO_DB_DATABASE was added with `cs_comments_service` as default value.
In docker-entrypoint.sh of forum I changed the hardcoded `cs_commecnts_service` with the new config value.
Multiple .yml files changed to handle the new config value.
