Bumping the `OPENEDX_COMMON_VERSION` in the master branch usually
creates a conflict when we merge the change in the nightly branch. To
avoid this conflict, we add some logic to the `OPENEDX_COMMON_VERSION`.
This change should be invisible for most users.
This partially addresses issue #936.
This fix is for a rather serious issue that affects users who upgrade
from Olive to Palm. The client mysql charset and collation was
incorrectly set to utf8mb4, while the server stil runs utf8mb3. Only
users who run the mysql container are affected.
To resolve this issue, we explicitely configure the client to use the
utf8mb3 charset/collation.
Important note: users who have somehow managed to upgrade from olive to
Palm before may find themselves in an undefined state. They might have
to fix their mysql data manually. Same thing for users who launched Palm
from scratch; although, according to my preliinary tests, they should be
able to downgrade their connection from utf8mb4 to utf8mb3 without
issue.
In addition, we upgrade to mysql 8.1.0. Among many other fixes, this
avoids a server restart after the upgrade:
> An in-place upgrade from MySQL 5.7 to MySQL 8.0, without a server
> restart, could result in unexpected errors when executing queries on
> tables. This fix eliminates the need to restart the server between the
> upgrade and queries. (Bug #35410528)
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-34.html
See also the 8.1.0 release notes:
https://dev.mysql.com/doc/relnotes/mysql/8.1/en/news-8-1-0.html
Close #887.
This is an important change, where we get remove the previous `--mount`
option, and instead opt for persistent bind-mounts.
Persistent bind mounts have several advantages:
- They make it easier to remember which folders need to be bind-mounted.
- Code is *much* less clunky, as we no longer need to generate temporary
docker-compose files.
- They allow us to bind-mount host directories *at build time* using the
buildx `--build-context` option.
- The transition from development to production becomes much easier, as
images will automatically be built using the host repo.
The only drawback is that persistent bind-mounts are slightly less
portable: when a config.yml file is moved to a different folder, many
things will break if the repo is not checked out in the same path.
For instance, this is how to start working on a local fork of
edx-platform:
tutor config save --append MOUNTS=/path/to/edx-platform
And that's all there is to it. No, this fork will be used whenever we
run:
tutor images build openedx
tutor local start
tutor dev start
This change is made possible by huge improvements in the build time
performance. These improvements make it convenient to re-build Docker
images often.
Related issues:
https://github.com/openedx/wg-developer-experience/issues/71https://github.com/openedx/wg-developer-experience/issues/66https://github.com/openedx/wg-developer-experience/issues/166
Among other changes: ORA2 file uploads were stored in a folder named
"SET-ME-PLEASE (ex. bucket-name)" (sigh). With this change, the folder
should be automatically renamed to "openedxuploads". This issue has been
occuring since June 2019... (sigh²)
Close #707
Users want to be able to override the request `max_size` to upload
larger files. But they will not be able to if the patch is placed after
the `request` directive. So we move the patch statement before the
directive. Also, we wrap the `request_body` directives within `handle`
statements. If not, then different sizes are not managed properly.
To override the max upload size in the cms, add the following to the
"caddyfile-cms" patch:
handle_path /import/* {
request_body {
max_size 500MB
}
}
See discussion:
https://discuss.openedx.org/t/how-to-update-caddyfile-using-tutor-plugin/8944
When running:
tutor dev run -m /path/to/edx-platform lms
pip install -r requirements/edx/development.txt
I realised that I was re-installing packages that should already have been
present in the image. The reason for that was that I was running an outdated
version of the dev version of the openedx Docker image. This happens because
`tutor dev run` does not trigger an image re-build.
We solve this issue by pinning the openedx dev Docker image tag to the current
tutor version.
This change builds upon a previously proposed PR:
https://github.com/overhangio/tutor/pull/437
There was another long conversation about this topic here:
https://github.com/overhangio/tutor-forum/pull/10#issuecomment-1314799915
We could have supported the MongoDB auth/replica set/ssl parameters as part of
the MongoDB host URI, but then this URI is not supported in the forum plugin,
which uses an old version of the mongoid client. We were hoping that the client
would have been upgraded by now, but it's not been upgraded for a long time.
The changes introduced here are 100% backward-compatible. The forum plugin will
have to be updated to take into account the new parameters.
- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
- 💥 [Feature] Persistent grades are now enabled by default.
- [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
- [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
- [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
The `--mount` option is available both with `tutor local`
and `tutor dev` commands. It allows users to easily bind-mount containers from
the host to containers. Yes, I know, we already provide that possibility with
the `bindmount` command and the `--volume=/path/` option. But these suffer from
the following drawbacks:
- They are difficult to understand.
- The "bindmount" command name does not make much sense.
- It's not convenient to mount an arbitrary folder from the host to multiple
containers, such as the many lms/cms containers (web apps, celery workers and
job runners).
To address this situation, we now recommend to make use of --mount:
1. `--mount=service1[,service2,...]:/host/path:/container/path`: manually mount
`/host/path` to `/container/path` in container "service1" (and "service2").
2. `--mount=/host/path`: use the new v1 plugin API to discover plugins that
will detect this option and select the right containers in which to bind-mount
volumes. This is really nifty...
Close https://github.com/overhangio/2u-tutor-adoption/issues/43
PR #619 set the EDX_PLATFORM_VERSION build arg's default to
OPENEDX_COMMON_VERSION. While this works fine for setting a
non-default branch to run edx code from (say, "master"), it may break
if the user sets OPENEDX_COMMON_VERSION to a branch or tag name that
does not exist upstream in repositories *other than*
EDX_PLATFORM_REPOSITORY.
Thus, introduce a separate configuration parameter,
EDX_PLATFORM_VERSION, to match the build arg of the same name. Set its
default to OPENEDX_COMMON_VERSION.
This way, the user can deploy an arbitrarily-named fork of
edx-platform, while retaining the default OPENEDX_COMMON_VERSION
(like, for example "open-release/maple.3") for everything else.
Previously, the only way for Tutor users to use a fork of edx-platform
or a custom NPM registry was to use build args during the image build.
This is suboptimal in the case of automatically building images from
CI pipelines, which may want to auto-detect when an image needs to be
rebuilt based on config.yml changes.
In addition, the EDX_PLATFORM_VERSION build argument can already be
set via a corresponding config.yml parameter (OPENEDX_COMMON_VERSION),
so it's reasonable to follow that precedent and also introduce
config.yml parameters to correspond with the EDX_PLATFORM_REPOSITORY
and NPM_REGISTRY build arguments.
Thus, introduce two new configuration parameters:
- EDX_PLATFORM_REPOSITORY
- NPM_REGISTRY
These parameters can now optionally be used instead of the
aforementioned build args.
- A shared cookie domain between lms and cms is no longer recommended:
https://github.com/edx/edx-platform/blob/master/docs/guides/studio_oauth.rst
- refactor: clean mounted data folder in lms/cms. In Lilac, the
bind-mounted lms/data and cms/data folders are a mess because new
folders are created there for every new course organisation. These
folders are empty. As far as we know they are useless... With this
change we move these folders to a dedicated "modulestore" subdirectory;
which corresponds better to the initial intent of the fs_root setting.
- fix: frontend failure during login to the lms. See:
https://github.com/openedx/build-test-release-wg/issues/104
- feat: move all forum-related code to a dedicated plugin. Forum is an
optional feature, and as such it deserves its own plugin. Starting from
Maple, users will be able to install the forum from
https://github.com/overhangio/tutor-forum/
- migrate from DCS_* session cookie settings to SESSION_*. That's
because edx-platform no longer depends on django-cookies-samesite. Close
https://github.com/openedx/build-test-release-wg/issues/110
- get rid of tons of deprecation warnings in the lms/cms
- feat: make it possible to point to themed assets. Cherry-picking this
change makes it possible to point to themed assets with a theme-agnostic
url, notably from MFEs.
- Install all official plugins as part of the `tutor[full]` package.
- Don't print error messages about loading plugins during autocompletion.
- Prompt for image building when upgrading from one release to the next.
- Add `tutor local start --skip-build` option to skip building Docker images.
Close #450.
Close #545.
Previously, configuration management was very confusing because we kept mixing
"base" and "defaults" configuration:
- It was difficult to make the difference between core settings that were
necessary (e.g: passwords) as opposed to others that could simply be
defaulted to.
- The order of settings in config.yml mattered: config entries that depended on
other needed to be defined later. As a consequence, Tutor was not compatible
with Python 3.5, where dict entries are not sorted.
Previously, configuration management was very confusing because we kept mixing
"base" and "defaults" configuration:
- It was difficult to make the difference between core settings that were
necessary (e.g: passwords) as opposed to others that could simply be
defaulted to.
- The order of settings in config.yml mattered: config entries that depended on
other needed to be defined later. As a consequence, Tutor was not compatible
with Python 3.5, where dict entries are not sorted.
