These changes make to possible to run:
tutor mounts add /path/to/my-xblock
The xblock directory with then be auto-magically bind-mounted in the
"openedx" image at build time, and the lms*/cms* containers at run time.
This makes it effectively possible to work as a developer on
edx-platform requirements.
We take the opportunity to move some openedx-specific code to a
dedicated module.
Close https://github.com/openedx/wg-developer-experience/issues/177
On macOS, building the "openedx-dev" Docker image resulted in an image that required more than 600 GB of disk space. This was due to the `adduser` command which was called with a user ID of 2x10⁹ (on macOS only). This resulted in a very large /var/log/faillog file, hence the image size.
Related upstream discussion: https://github.com/moby/moby/issues/5419
Close https://github.com/openedx/wg-developer-experience/issues/178
On Oct. 10, py2neo package was abruptly removed from pypi, GitHub, and
the py2neo website now displays just a super funny meme: https://py2neo.org/
Yes, we should get rid of that dependency, but we are still supposed to
support existing users. So we install py2neo from our fork.
We upgrade nodeenv as an attempt to fix incomplete reads.
From time to time we face the following error:
#67 [linux/amd64 nodejs-requirements 2/4] RUN nodeenv /openedx/nodeenv --node=16.14.0 --prebuilt
#67 0.338 * Install prebuilt node (16.14.0) .Incomplete read while readingfrom https://nodejs.org/download/release/v16.14.0/node-v16.14.0-linux-x64.tar.gz#67 204.1 .
#67 204.1 Traceback (most recent call last):
#67 204.1 File "/openedx/venv/bin/nodeenv", line 8, in <module>
#67 204.1 sys.exit(main())
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 1104, in main
#67 204.1 create_environment(env_dir, args)
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 980, in create_environment
#67 204.1 install_node(env_dir, src_dir, args)
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 739, in install_node
#67 204.1 install_node_wrapped(env_dir, src_dir, args)
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 762, in install_node_wrapped
#67 204.1 download_node_src(node_url, src_dir, args)
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 602, in download_node_src
#67 204.1 with ctx as archive:
#67 204.1 File "/opt/pyenv/versions/3.8.15/lib/python3.8/contextlib.py", line 113, in __enter__
#67 204.1 return next(self.gen)
#67 204.1 File "/openedx/venv/lib/python3.8/site-packages/nodeenv.py", line 573, in tarfile_open
#67 204.1 tf = tarfile.open(*args, **kwargs)
#67 204.1 File "/opt/pyenv/versions/3.8.15/lib/python3.8/tarfile.py", line 1601, in open
#67 204.1 saved_pos = fileobj.tell()
#67 204.1 AttributeError: 'bytes' object has no attribute 'tell'
This change was added to 1.8.0 as an attempt to resolve the issue: https://github.com/ekalinin/nodeenv/pull/329
We are not sure it will work every time, but it can't hurt.
Among other changes: ORA2 file uploads were stored in a folder named
"SET-ME-PLEASE (ex. bucket-name)" (sigh). With this change, the folder
should be automatically renamed to "openedxuploads". This issue has been
occuring since June 2019... (sigh²)
Close #707
Before this commit, setting up an edx-platform development environment
took multiple steps:
tutor dev launch
tutor dev run --mount=/path/to/edx-platform lms bash
>> pip install -e .
>> npm clean-install
>> openedx-assets build --env=dev
This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:
tutor dev launch --mount=edx-platform
These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.
Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.
These bind-mounts:
* ../build/openedx/themes:/openedx/themes
* ../build/openedx/requirements:/openedx/requirements
existed in the dev lms and cms containers, but they did
not exist in the lms-job and cms-job containers.
This means that themes and requirements that were *built into the
image* would exist in the job containers, but live updates to the
themes and requirements would not apply.
To resolve this, we set ``volumes:`` on the lms-job and cms-job
services so that they match the volumes for the normal lms and
cms services.
Part of: https://github.com/openedx/wg-developer-experience/issues/146
Closes: https://github.com/openedx/wg-developer-experience/issues/152
This works around (but does not close) these related issues:
* https://github.com/openedx/wg-developer-experience/issues/150
* https://github.com/openedx/wg-developer-experience/issues/151
The `compilejsi18n` command was failing during image building because the
Open-edX package was not installed properly. The reason for that was an earlier
change where we got rid of the `pip install -r requirements/edx/local.in`
command. Installing the Open-edX package was part of this requirement file.
The local.in requirements file no longer exists, but we still need to `pip
install -e .` the edx-platform repo. To run this command we need both the
edx-platform repo and the virtualenv.
The good news is that there are no more local requirements in the base.txt
requirements file. This means that we no longer have to COPY the edx-platform
repo in the requirements installation step. Thus, changes in edx-platform will
no longer trigger a rebuild of the pip requirements; this means that re-builds
will be much faster when making changes to edx-platform.
Note that plugins that implemented the
"openedx-dockerfile-post-python-requirements" patch and that needed access to
the edx-platform repo will no longer work. Instead, these plugins should
implement the "openedx-dockerfile-pre-assets" patch. This scenario should be
very rare, though.
Close #726
The local requirements files does not exist since local requirements were all
removed from the edx-platform repo. As a consequence, the nightly build was
broken.
Strings could not be pulled from transifex because the file names were
incorrect. This is now fixed and we are now able to pull the i18n strings from
the nutmeg.2 tag.
Soon, running:
pip install -r ./requirements/edx/base.txt
in edx-platform will no longer install the local
project (that is, `-e .`). To prepare for that change,
we add the line:
pip install -e .
to the Dockerfile. This is backwards-compatible.
More details:
https://openedx.atlassian.net/browse/BOM-2575?focusedCommentId=613181
Sometimes, the host user is root: this may happen when tutor is run with
"sudo" (which is not recommended) or on Windows. In such cases, building
the image should not fail, but default to a reasonable user. Also, when
we pass an invalid APP_USER_ID as a build arg, then we should fail with
an explicit message.
See this conversation:
https://discuss.overhang.io/t/problem-with-dev-image-build-useradd-uid-0-is-not-unique/2406
PYTHONBREAKPOINT has been exposed as an environment variable in
the openedx Dockerfile available to be changed in config.yml. The docs have also been changed to recommend using
breakpoint and explaining how PYTHONBREAKPOINT can be modified to use a
custom debugger.
Close https://github.com/overhangio/2u-tutor-adoption/issues/45
- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
- 💥 [Feature] Persistent grades are now enabled by default.
- [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
- [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
- [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
The entrypoint in the "openedx" Docker image was used only to define the
DJANGO_SETTINGS_MODULE environment variable, based on SERVICE_VARIANT and
SETTINGS. We ditch SETTINGS in favour of defining explicitely
DJANGO_SETTINGS_MODULE.
The problem with the Docker entrypoint is that it was bypassed whenever we ran
`tutor local exec` or `tutor k8s exec`. By removing it we make it simpler for
end-users to run manage.py commands in kubernetes.
