6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-12 22:27:47 +00:00
Commit Graph

1491 Commits

Author SHA1 Message Date
Régis Behmo
2f442d7db3 fix: edx-platform requirement installation
The local requirements files does not exist since local requirements were all
removed from the edx-platform repo. As a consequence, the nightly build was
broken.
2022-09-30 12:06:28 +02:00
Carlos Muniz
fe901ab9de feat: deprecate "quickstart" and rename to "launch"
`quickstart` is being renamed to `launch` and deprecated in favor of
using `launch`. The `quickstart` function temporarily aliases to
`launch`. Further mentions of `quickstart` have been changed to
reference `launch` instead.

We are indicating that this change is breaking 💥 to encourage people to
migrate their scripts right away!
2022-09-30 12:05:35 +02:00
Overhang.IO
a7da98acc1 Merge remote-tracking branch 'origin/master' into nightly 2022-09-29 12:51:30 +00:00
Régis Behmo
dbb528aba4 chore: partial reqs upgrade
Oauthlib upgrade was suggested by dependabot:
https://github.com/overhangio/tutor/pull/723/files

We did not perform a full upgrade because sphinx-rtd-theme does not support
docutils 0.19, yet: https://github.com/readthedocs/sphinx_rtd_theme/issues/1323
2022-09-29 13:55:09 +02:00
Overhang.IO
45316bf752 Merge remote-tracking branch 'origin/master' into nightly 2022-09-12 05:22:31 +00:00
Kyle McCormick
98d756243f docs: add troubleshootoing docs for mysql connectivity errors
Adding docs to capture some common errors I've
been seeing in Slack. I've experienced these
myself as well.
2022-09-12 06:38:44 +02:00
Overhang.IO
435de0f21f Merge remote-tracking branch 'origin/master' into nightly 2022-09-08 10:03:17 +00:00
Régis Behmo
b4f905c2aa feat: upgrade i18n openedx strings to nutmeg.2
Strings could not be pulled from transifex because the file names were
incorrect. This is now fixed and we are now able to pull the i18n strings from
the nutmeg.2 tag.
2022-09-08 11:20:49 +02:00
Kyle McCormick
d9314b7507 build: prepare Dockerfile for common/lib removal
Soon, running:

  pip install -r ./requirements/edx/base.txt

in edx-platform will no longer install the local
project (that is, `-e .`). To prepare for that change,
we add the line:

  pip install -e .

to the Dockerfile. This is backwards-compatible.
More details:
https://openedx.atlassian.net/browse/BOM-2575?focusedCommentId=613181
2022-09-06 16:15:39 +02:00
Carlos Muniz
72c417da62 docs: add entry to CHANGELOG-nightly.md 2022-09-06 16:06:26 +02:00
Carlos Muniz
b879a9d17b feat: remove the implementation of dev runserver 2022-09-06 16:06:26 +02:00
Régis Behmo
3024bb257c Merge branch 'master' into nightly 2022-08-29 15:12:10 +02:00
Sofiane Bebert
3f1dd832e4 v14.0.5 (2022-08-29)
- [Bugfix] Fix MongoDB replica set connection error resulting from edx-platform's pymongo (3.10.1 -> 3.12.3) upgrade ([edx-platform#30569](https://github.com/openedx/edx-platform/pull/30569)). (by @ormsbee)
- [Feature] Upgrade all applications to open-release/nutmeg.2.
2022-08-29 10:10:29 +02:00
David Ormsbee
417feeb0f3 fix: replica set error from pymongo 3.10 -> 3.12 upgrade
The pymongo dependency for edx-platform was updated (3.10.1 to 3.12.3)
in https://github.com/openedx/edx-platform/pull/30569

This caused the following error when running the edx-platform database
migration split_modulestore_django.0002_data_migration as part of
`tutor dev quickstart`:

  pymongo.errors.ServerSelectionTimeoutError: client is configured to
  connect to a replica set named '' but this node belongs to a set named
  'None', Timeout: 30s, Topology Description: <TopologyDescription id:
  62bdbaf182687350acf1aeec, topology_type: Single, servers:
  [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt:
   None, error=ConfigurationError("client is configured to connect to a
   replica set named '' but this node belongs to a set named 'None'")>]>

This commit explicitly sets replicaSet to None to indicate that it's a
standalone MongoDB instance. I also had to remove the CONTENTSTORE entry
from auth.yml because edx-platform's devstack.py assumes it has a
non-null value (set in common.py), and devstack.py executes before
tutor's development.py can set this replicaSet value.
2022-08-21 09:37:26 +01:00
Sofiane Bebert
6674e267de feat: upgrade all apps to open-release/nutmeg.2 2022-08-21 09:15:00 +01:00
Overhang.IO
31fdcc60af Merge remote-tracking branch 'origin/master' into nightly 2022-07-29 18:40:39 +00:00
Régis Behmo
66bef2e14f v14.0.4 (2022-07-29)
- [Feature] Add the `-m/--mount` option to `tutor dev quickstart`.
- [Bugfix] Fix `tutor dev start -m /path/to/frontend-app-learning` by introducing dev-specific `COMPOSE_DEV_TMP` and `COMPOSE_DEV_JOBS_TMP` filters (by @regisb).
- [Bugfix] Log the shell commands that Tutor executes more accurately. (by @kdmccormick)
- [Bugfix] `tutor dev quickstart` would fail under certain versions of docker-compose due to a bug in the logic that handled volume mounting. (by @kdmccormick)
- [Bugfix] The `tutor k8s start` command will succeed even when `k8s-override` and `kustomization-patches-strategic-merge` are not specified. (by @edazzocaisser)
- [BugFix] `kubectl wait` checks deployments instead of pods as it could hang indefinitely if there are extra pods in a broken state. (by @keithgg)
2022-07-29 19:54:15 +02:00
Régis Behmo
52cf0cc1ca feat: add -m/--mount option to dev quickstart 2022-07-29 19:53:02 +02:00
Régis Behmo
a2a3c022b8 fix: bind-mount in dev-specific services
The -m/--mount option makes it possible to bind-mount volumes at runtime. The
volumes are declared in a local/docker-compose.tmp.yml file. The problem with
this approach is when we want to bind-mount a volume to a service which is
specific to the dev context. For instance: the "learning" service when the MFE
plugin is enabled.

In such a case, starting the service triggers a call to `docker-compose stop`
in the local context. This call fails because the "learning" service does not
exist in the local context. Note that this issue only seems to occur with
docker-compose v1.

To resolve this issue, we create two additional filters for
the dev context, which emulate the behaviour of the local context. With this approach, we convert the -m/--mount arguments right after they are parsed. Because they are parsed just once, we can get rid of the de-duplication logic initially introduced with the COMPOSE_CLI_MOUNTS context.

Close #711. Close also https://github.com/overhangio/tutor-mfe/issues/57.
2022-07-29 19:53:02 +02:00
Overhang.IO
cb1b22f54e Merge remote-tracking branch 'origin/master' into nightly 2022-07-25 20:51:54 +00:00
Kyle McCormick
8345b7ab93 temp: shim shlex.join for Python 3.7 compatability 2022-07-25 22:46:53 +02:00
Kyle McCormick
6a3138152f fix: accurately log shell commands when running jobs
Whenever Tutor executes a shell command, it logs out said
command in order to aid in end user understanding/debugging.

In some cases (notably, when running jobs in containers)
the logged command was not accurately quoted. The command
was run correctly, because it was passed in pieces to
``subprocess.Popen``, which correctly joins the pieces together
into a valid POSIX shell command; however, the logged version
of the command was constructed by simply joining the pieces
with spaces. This usually works, but breaks down when running
complex shell commands with nested quoting.

This commit changes the logging to use ``shlex.join``, which
joins command pieces together in a POSIX-compliant way,
presumably the same way as ``subprocess.Popen``.

Example:

    tutor local importdemocourse

runs the shell command:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c 'echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup'

but the logged shell command was:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup

which will not run if copied and pasted back into the
user's terminal, as the importdemocourse shell script is unquoted.
2022-07-25 22:46:53 +02:00
Keith Grootboom
55bc4c63fa fix: change kubectl wait to look at deployments not pods
When waiting for pods, it's possible that the deployment may be
complete but, because other pods may have been Evicted or Killed, the
wait wait condition completes.
2022-07-25 22:46:45 +02:00
Régis Behmo
8e188dc946 Merge branch 'master' into nightly 2022-07-25 16:13:05 +02:00
Kyle McCormick
0ae59a82a6
fix: avoid double-rendering mounts to docker-compose.tmp.yml (#669)
In certain code paths, such as in `tutor local quickstart`,
`process_mount_points` is called more than once in the same process,
causing mounts to be added to `COMPOSE_LOCAL[_JOBS]_TMP` redundantly.
As a result, docker-compose[.jobs].tmp.yml was occasionally being
rendered with duplicate volume specifiers. Some versions of Docker
Compose ignored this; other versions warned or threw an error.

In order to make `process_mount_points` tolerant to being called
multiple times, we wrap its volume-adding callbacks within a new
hooks context. This allows us to clear said hooks context every
time `process_mount_points` is called, essentially making the
function idempotent.

Co-authored-by: Régis Behmo <regis@behmo.com>
2022-07-25 15:58:49 +02:00
Carlos Muniz
d27e8d5ba7
docs: add missing config parameters from defaults.yaml
Document config parameters from defaults.yaml that were missing from the docs.


Co-authored-by: Carlos Muniz <cmuniz@trcil.org>
2022-07-25 15:43:40 +02:00
Ed D'Azzo
2e41de1af9 Added a record of the bugfix to the CHANGELOG 2022-07-25 15:25:25 +02:00
Ed D'Azzo
5e81a754c4 Fixed or syntax 2022-07-25 15:25:25 +02:00
Ed D'Azzo
67358c6b7a Omit patchesStrategicMerge when there are no custom resources 2022-07-25 15:25:25 +02:00
Kyle McCormick
9d1ce4717b docs: update test-running guide with new top-level xmodule package
edx-platform's ./common/lib/xmodule/xmodule folder has been moved to
./xmodule. The test-running instructions needed to be updated in order
to account for this new top-level folder.

For context on the edx-platform change, see:
https://discuss.openedx.org/t/breaking-apart-edx-platforms-common-lib-folder
2022-07-14 14:46:12 +02:00
David Ormsbee
bb907b4de3 v14.0.3 (2022-07-09)
- [Bugfix] Build openedx-dev Docker image even when the host user is root, for instance on Windows. (by @regisb)
- [Bugfix] Patch nutmeg.1 release with [LTI 1.3 fix](https://github.com/openedx/edx-platform/pull/30716). (by @ormsbee)
- [Improvement] Make it possible to override k8s resources in plugins using `k8s-override` patch. (by @foadlind)
2022-07-09 19:01:58 +02:00
Régis Behmo
5b5700eff4 fix: build openedx-dev image when host user is root
Sometimes, the host user is root: this may happen when tutor is run with
"sudo" (which is not recommended) or on Windows. In such cases, building
the image should not fail, but default to a reasonable user. Also, when
we pass an invalid APP_USER_ID as a build arg, then we should fail with
an explicit message.

See this conversation:
https://discuss.overhang.io/t/problem-with-dev-image-build-useradd-uid-0-is-not-unique/2406
2022-07-09 00:09:20 +02:00
David Ormsbee
ac3b4e3065 fix: apply LTI 1.3 XBlock patch fix 3.4.7 over nutmeg.1
The actual merge of this into open-release/nutmeg.master is here:
  https://github.com/openedx/edx-platform/pull/30716
2022-07-08 23:48:45 +02:00
David Ormsbee
6ac1c0d732 fix: replica set error from pymongo 3.10 -> 3.12 upgrade
The pymongo dependency for edx-platform was updated (3.10.1 to 3.12.3)
in https://github.com/openedx/edx-platform/pull/30569

This caused the following error when running the edx-platform database
migration split_modulestore_django.0002_data_migration as part of
`tutor dev quickstart`:

  pymongo.errors.ServerSelectionTimeoutError: client is configured to
  connect to a replica set named '' but this node belongs to a set named
  'None', Timeout: 30s, Topology Description: <TopologyDescription id:
  62bdbaf182687350acf1aeec, topology_type: Single, servers:
  [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt:
   None, error=ConfigurationError("client is configured to connect to a
   replica set named '' but this node belongs to a set named 'None'")>]>

This commit explicitly sets replicaSet to None to indicate that it's a
standalone MongoDB instance. I also had to remove the CONTENTSTORE entry
from auth.yml because edx-platform's devstack.py assumes it has a
non-null value (set in common.py), and devstack.py executes before
tutor's development.py can set this replicaSet value.
2022-07-05 16:44:12 +02:00
Overhang.IO
9d5e94b8d5 Merge remote-tracking branch 'origin/master' into nightly 2022-07-04 05:09:54 +00:00
Prakash Divy
9a63dc70ce docs: Update help command in local.rst 2022-07-04 06:30:29 +02:00
Overhang.IO
ede972a1da Merge remote-tracking branch 'origin/master' into nightly 2022-06-28 09:12:52 +00:00
Foad Lind
b8f773a5ef feat: Make it possible to customize k8s resources
Currently there is no way for plugins to customize Kubernetes resources
defined in Tutor deployment manifests.
This change makes that possible by taking advantage of the strategic
merge patching mechanism in `kustomization.yml`.
Any resource definition in a `k8s-override` patch in a plugin will
override the resource defined by Tutor, provided that their names match.

Reference: https://github.com/overhangio/tutor/pull/675
2022-06-28 10:30:24 +02:00
Overhang.IO
9d6d486c65 Merge remote-tracking branch 'origin/master' into nightly 2022-06-27 14:50:26 +00:00
Régis Behmo
4a808fa87b v14.0.2 (2022-06-27)
- [Bugfix] Update problem with hint template so it works with newer python versions. (by @mariajgrimaldi)
- [Feature] Add default PYTHONBREAKPOINT to openedx/Dockerfile (by @Carlos-Muniz)
- [Bugfix] Fix smtp server port in `cms.yml` which was causing email sending failures in the Studio. (by @regisb)
- [Bugfix] Skip waiting for MongoDB if it is served using SRV records. (by @gabor-boros)
- [Improvement] Use `git am` instead of `cherry-pick` to simplify patching process.
- [Improvement] Tutor is now compatible with Docker Compose subcommand.
2022-06-27 16:07:08 +02:00
Régis Behmo
570639a114 Merge branch 'master' into nightly 2022-06-27 10:02:34 +02:00
Gábor Boros
63bc184eab chore: remove extra empty line 2022-06-27 09:16:13 +02:00
Gabor Boros
eadb04fee9 fix: skip waiting for mongodb SRV records
Signed-off-by: Gabor Boros <gabor.brs@gmail.com>
2022-06-27 09:16:13 +02:00
Maria Grimaldi
4b8cf4ccc7 fix: update problem with hint according newer python versions 2022-06-27 09:15:56 +02:00
Overhang.IO
7f1162d29b Merge remote-tracking branch 'origin/master' into nightly 2022-06-22 10:09:09 +00:00
Régis Behmo
b13d4da1b1 fix: @lru_cache python 3.7 compatibility
In Python 3.7, which is used in the Tutor CI, the lru_cache needs an argument:
https://docs.python.org/3.7/library/functools.html#functools.lru_cache
On GitHub, CI did not catch this error because it is running Python 3.8.
2022-06-22 11:28:27 +02:00
Carlos Muniz
4dac13923f
feat: default to ipdb as PYTHONBREAKPOINT
PYTHONBREAKPOINT has been exposed as an environment variable in
the openedx Dockerfile available to be changed in config.yml. The docs have also been changed to recommend using
breakpoint and explaining how PYTHONBREAKPOINT can be modified to use a
custom debugger.

Close https://github.com/overhangio/2u-tutor-adoption/issues/45
2022-06-22 11:26:50 +02:00
iamcristye@outlook.com
a4aa01987b improvement: compatibility with Docker Compose subcommand
closes #641
2022-06-22 11:23:39 +02:00
Overhang.IO
2668633974 Merge remote-tracking branch 'origin/master' into nightly 2022-06-20 13:13:12 +00:00
Régis Behmo
602c381d6e fix: email sending from cms
Incorrect format of cms.yml config file was causing the following error on course import:

	cms-worker_1                 | Traceback (most recent call last):
	cms-worker_1                 |   File "/openedx/edx-platform/cms/djangoapps/cms_user_tasks/tasks.py", line 53, in send_task_complete_email
	cms-worker_1                 |     mail.send_mail(subject, message, from_address, [dest_addr], fail_silently=False)
	cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 61, in send_mail
	cms-worker_1                 |     return mail.send()
	cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 284, in send
	cms-worker_1                 |     return self.get_connection(fail_silently).send_messages([self])
	cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
	cms-worker_1                 |     new_conn_created = self.open()
	cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 62, in open
	cms-worker_1                 |     self.connection = self.connection_class(self.host, self.port, **connection_params)
	cms-worker_1                 |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 255, in __init__
	cms-worker_1                 |     (code, msg) = self.connect(host, port)
	cms-worker_1                 |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 339, in connect
	cms-worker_1                 |     self.sock = self._get_socket(host, port, self.timeout)
	cms-worker_1                 |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 310, in _get_socket
	cms-worker_1                 |     return socket.create_connection((host, port), timeout,
	cms-worker_1                 |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 787, in create_connection
	cms-worker_1                 |     for res in getaddrinfo(host, port, 0, SOCK_STREAM):
	cms-worker_1                 |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 918, in getaddrinfo
	cms-worker_1                 |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
	cms-worker_1                 | socket.gaierror: [Errno -8] Servname not supported for ai_socktype

The reason was that the trailing comma "," was interpreted as being part of the email port.
2022-06-20 14:33:48 +02:00