6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-24 02:55:28 +00:00
Commit Graph

1373 Commits

Author SHA1 Message Date
Régis Behmo
c9ac12fdd7 v14.2.1 (2022-11-24)
- [Improvement] Auto-completion of `plugins` and `config` arguments: `plugins enable/disable NAME`, `plugins install PATH`, `config save --set KEY=VAL`, `config save --unset KEY`, `config printvalue KEY`. (by @regisb)
- [Bugfix] Fix minimum click version (>= 8.0.0) when installing tutor from pip.
- [Improvement] Enable CORS by default for both LMS and CMS by moving those settings to the `common_all` partial. (by @arbrandes)
2022-11-24 16:25:34 +01:00
Régis Behmo
04258b9f9b docs: add changelog entry for click fix 2022-11-24 16:24:18 +01:00
Régis Behmo
29eb3398a2 feat: auto-complete config save/printroot arguments 2022-11-24 16:21:57 +01:00
Régis Behmo
ee09612326 feat: auto-complete plugins arguments
Support auto-completion of plugin name and path arguments in the `tutor
plugins` commands.
2022-11-24 16:21:57 +01:00
Régis Behmo
6069f91cce docs: fix nightly typo
Co-authored-by: Adolfo R. Brandes <arbrandes@tcril.org>
2022-11-24 16:15:15 +01:00
Régis Behmo
c886b72ae7 docs: migrate to scriv to manage changelog
Changelog management was starting to be a hassle:

- there were conflicts every time a PR was merged
- there were conflicts every time we merged the nightly branch in the new
  release branch, or vice versa.

Now, all changelog entries are stored as separate files in changelog.d,
including nightly. Nightly entries will be collected for every major release.
2022-11-24 16:15:15 +01:00
Régis Behmo
e63d5c32e3 fix: minimum click version is 8.0
An issue was reported here: https://app.slack.com/client/T02SNA1T6/CGE253B7V

CompletionItem was introduced in 8.0, so click needs to be upgraded when
we `pip install tutor`.
2022-11-23 18:24:05 +01:00
Adolfo R. Brandes
3e32f88c15 feat: common CORS settings for LMS and CMS
The ENABLE_CORS_HEADERS feature flag is already true for the LMS.
Instead of duplicating it for Studio via yaml settings, make this a
common Django setting to both LMS and CMS and all their environments.
2022-11-21 13:51:56 +01:00
Régis Behmo
5006f6edc9 v14.2.0 (2022-11-21)
- [Improvement] Auto-complete implicit `local/dev --mount /path/to/...` options. (by @regisb)
- 💥[Feature] Strong typing of action and filter hooks: this allows us to detect incorrect calls to `actions.add` or `filters.add` early. Strong typing forces us to break the `do` and `apply` API by removing the `context` named argument. Developers should replace `do(context=...)` by `do_from_context(..., )` (and similar for `apply`).
2022-11-21 10:10:44 +01:00
Régis Behmo
986c3e2305 chore: upgrade requirements 2022-11-21 10:10:07 +01:00
Régis Behmo
f8b5cbc657 feat: auto-complete --mount args
When typing `tutor local run --mount /path/to/edx-pl<TAB>`, the mount option
should be auto-completed to the full edx-platform repo path. That is, if shell
completion is enabled:
https://docs.tutor.overhang.io/install.html#shell-autocompletion

Here, we make sure that the implicit form of the `--mount` argument is properly
auto-completed. We are unable to get completion to work in the explicit form,
because args that include colons do not even reach the `shell_completion`
method.
2022-11-17 14:50:52 +01:00
Régis Behmo
33e4f33afe feat: strongly typed hooks
Now that the mypy bugs have been resolved, we are able to define more precisely
and cleanly the types of Actions and Filters.

Moreover, can now strongly type named actions and hooks (in consts.py). With
such a strong typing, we get early alerts of hooks called with incorrect
arguments, which is nothing short of awesome :)

This change breaks the hooks API by removing the `context=...` argument. The
reason for that is that we cannot insert arbitrary arguments between `P.args,
P.kwargs`: https://peps.python.org/pep-0612/#the-components-of-a-paramspec

> A function declared as def inner(a: A, b: B, *args: P.args, **kwargs:
> P.kwargs) -> R has type Callable[Concatenate[A, B, P], R]. Placing
> keyword-only parameters between the *args and **kwargs is forbidden.

Getting the documentation to build in nitpicky mode is quite difficult... We
need to add `nitpick_ignore` to the docs conf.py, otherwise sphinx complains
about many missing class references. This, despite upgrading almost all doc
requirements (except docutils).
2022-11-15 14:58:36 +01:00
Régis Behmo
8aeeb7e09c chore: replaced unnamed hook filters by named equivalents 2022-11-14 12:25:28 +01:00
Régis Behmo
fd2d3dea10 v14.1.2 (2022-11-02)
- [Security] Fix edx-platform XSS vulnerability on "next" parameter. (by
  @regisb)
2022-11-02 10:42:55 +01:00
Régis Behmo
b46a7b0fa8 sec: fix edx-platform xss vulnerability on "next" parameter 2022-11-02 10:42:10 +01:00
Régis Behmo
f95c394e38 fix: print proper command on check_output
Note that the `check_output` command is mostly unused.
2022-10-26 16:46:04 +02:00
Régis Behmo
7541433729 v14.1.1 (2022-10-25)
- [Security] Fix xblock ajax handler vulnerability. (by @regisb)
- [Improvement] Use web proxy gzip encoding to improve bandwidth. We
  observe a 75% size reduction on the LMS dashboard. (by @ghassanmas)
2022-10-26 12:28:16 +02:00
Régis Behmo
1161f925a4 feat: add py.typed for typing extensions
Extensions which use the tutor API need to know that the tutor package
is typed. For that we add the py.typed file.

Also, we fix the signature of get_typed.

See:
https://mypy.readthedocs.io/en/stable/installed_packages.html#installed-packages
2022-10-26 12:28:16 +02:00
Régis Behmo
7eeccfb5f5 chore: mark Python 3.6 as deprecated 2022-10-26 12:24:07 +02:00
Régis Behmo
3ba5365537 sec: fix xblock ajax handler vulnerability 2022-10-25 18:56:40 +02:00
Ghassan Maslamani
7d32179038 feat: use encode gzip for caddy
Compressing assests would lead to readuce transfer size.
  As testing with frontend-app-learning/Olive, the network traffic
  before was about ~4MB, after this it became ~1MB.

  This change was suggested by Google Lighthouse[1], there are of
  course more suggestion but this was one the easiest and one of most
  impactful.

  Also check orignal PR overhangio/tutor-mfe/pull/64 for more
  info.
  [1]: https://web.dev/uses-text-compression
2022-10-19 10:46:01 +02:00
Régis Behmo
7453e70fa0 v14.1.0 (2022-10-10)
- [Improvement] Upgrade Scorm XBlock to v14.0.0. (by @regisb)
- 💥[Improvement] The Richie plugin was transferred to the Openfun organization; thus, it is no longer officially supported and it is removed from the default set of plugins that ships with
`pip install tutor[full]` or the Tutor pre-compiled binary. Users are encouraged to uninstall the `tutor-richie` Python package and install the `tutor-contrib-richie` package instead.
- [Feature] Upgrade edx-platform i18n strings to nutmeg.2. (by @regisb)
2022-10-10 10:14:36 +02:00
Régis Behmo
c0fb64714f feat: bump scorm to v14.0.0 2022-10-04 09:25:41 +02:00
Michael Wheeler
fd037fcb4d Remove warning about building tutor-mfe on ARM systems 2022-10-03 14:15:24 +02:00
Frank Anderson
871d293719 Update local.rst 2022-10-03 14:09:34 +02:00
Frank Anderson
55684ebd49 Updated local.rst with changes from #716 2022-10-03 14:09:34 +02:00
Frank Anderson
5fdede8511 Fix incorrect information about running config save --interactive
Running config save --interactive doesn't create a config.yml file in the current directory. It instead creates it in the project root.
2022-10-03 14:09:34 +02:00
Régis Behmo
020f6d1137 depr: richie plugin is no longer official
The richie plugin was transferred to the openfun GitHub org. As a consequence,
it is no longer part of the list of official plugins.

See: https://github.com/overhangio/tutor-contrib-richie/pull/5
2022-09-30 12:09:31 +02:00
Régis Behmo
dbb528aba4 chore: partial reqs upgrade
Oauthlib upgrade was suggested by dependabot:
https://github.com/overhangio/tutor/pull/723/files

We did not perform a full upgrade because sphinx-rtd-theme does not support
docutils 0.19, yet: https://github.com/readthedocs/sphinx_rtd_theme/issues/1323
2022-09-29 13:55:09 +02:00
Kyle McCormick
98d756243f docs: add troubleshootoing docs for mysql connectivity errors
Adding docs to capture some common errors I've
been seeing in Slack. I've experienced these
myself as well.
2022-09-12 06:38:44 +02:00
Régis Behmo
b4f905c2aa feat: upgrade i18n openedx strings to nutmeg.2
Strings could not be pulled from transifex because the file names were
incorrect. This is now fixed and we are now able to pull the i18n strings from
the nutmeg.2 tag.
2022-09-08 11:20:49 +02:00
Sofiane Bebert
3f1dd832e4 v14.0.5 (2022-08-29)
- [Bugfix] Fix MongoDB replica set connection error resulting from edx-platform's pymongo (3.10.1 -> 3.12.3) upgrade ([edx-platform#30569](https://github.com/openedx/edx-platform/pull/30569)). (by @ormsbee)
- [Feature] Upgrade all applications to open-release/nutmeg.2.
2022-08-29 10:10:29 +02:00
David Ormsbee
417feeb0f3 fix: replica set error from pymongo 3.10 -> 3.12 upgrade
The pymongo dependency for edx-platform was updated (3.10.1 to 3.12.3)
in https://github.com/openedx/edx-platform/pull/30569

This caused the following error when running the edx-platform database
migration split_modulestore_django.0002_data_migration as part of
`tutor dev quickstart`:

  pymongo.errors.ServerSelectionTimeoutError: client is configured to
  connect to a replica set named '' but this node belongs to a set named
  'None', Timeout: 30s, Topology Description: <TopologyDescription id:
  62bdbaf182687350acf1aeec, topology_type: Single, servers:
  [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt:
   None, error=ConfigurationError("client is configured to connect to a
   replica set named '' but this node belongs to a set named 'None'")>]>

This commit explicitly sets replicaSet to None to indicate that it's a
standalone MongoDB instance. I also had to remove the CONTENTSTORE entry
from auth.yml because edx-platform's devstack.py assumes it has a
non-null value (set in common.py), and devstack.py executes before
tutor's development.py can set this replicaSet value.
2022-08-21 09:37:26 +01:00
Sofiane Bebert
6674e267de feat: upgrade all apps to open-release/nutmeg.2 2022-08-21 09:15:00 +01:00
Régis Behmo
66bef2e14f v14.0.4 (2022-07-29)
- [Feature] Add the `-m/--mount` option to `tutor dev quickstart`.
- [Bugfix] Fix `tutor dev start -m /path/to/frontend-app-learning` by introducing dev-specific `COMPOSE_DEV_TMP` and `COMPOSE_DEV_JOBS_TMP` filters (by @regisb).
- [Bugfix] Log the shell commands that Tutor executes more accurately. (by @kdmccormick)
- [Bugfix] `tutor dev quickstart` would fail under certain versions of docker-compose due to a bug in the logic that handled volume mounting. (by @kdmccormick)
- [Bugfix] The `tutor k8s start` command will succeed even when `k8s-override` and `kustomization-patches-strategic-merge` are not specified. (by @edazzocaisser)
- [BugFix] `kubectl wait` checks deployments instead of pods as it could hang indefinitely if there are extra pods in a broken state. (by @keithgg)
2022-07-29 19:54:15 +02:00
Régis Behmo
52cf0cc1ca feat: add -m/--mount option to dev quickstart 2022-07-29 19:53:02 +02:00
Régis Behmo
a2a3c022b8 fix: bind-mount in dev-specific services
The -m/--mount option makes it possible to bind-mount volumes at runtime. The
volumes are declared in a local/docker-compose.tmp.yml file. The problem with
this approach is when we want to bind-mount a volume to a service which is
specific to the dev context. For instance: the "learning" service when the MFE
plugin is enabled.

In such a case, starting the service triggers a call to `docker-compose stop`
in the local context. This call fails because the "learning" service does not
exist in the local context. Note that this issue only seems to occur with
docker-compose v1.

To resolve this issue, we create two additional filters for
the dev context, which emulate the behaviour of the local context. With this approach, we convert the -m/--mount arguments right after they are parsed. Because they are parsed just once, we can get rid of the de-duplication logic initially introduced with the COMPOSE_CLI_MOUNTS context.

Close #711. Close also https://github.com/overhangio/tutor-mfe/issues/57.
2022-07-29 19:53:02 +02:00
Kyle McCormick
8345b7ab93 temp: shim shlex.join for Python 3.7 compatability 2022-07-25 22:46:53 +02:00
Kyle McCormick
6a3138152f fix: accurately log shell commands when running jobs
Whenever Tutor executes a shell command, it logs out said
command in order to aid in end user understanding/debugging.

In some cases (notably, when running jobs in containers)
the logged command was not accurately quoted. The command
was run correctly, because it was passed in pieces to
``subprocess.Popen``, which correctly joins the pieces together
into a valid POSIX shell command; however, the logged version
of the command was constructed by simply joining the pieces
with spaces. This usually works, but breaks down when running
complex shell commands with nested quoting.

This commit changes the logging to use ``shlex.join``, which
joins command pieces together in a POSIX-compliant way,
presumably the same way as ``subprocess.Popen``.

Example:

    tutor local importdemocourse

runs the shell command:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c 'echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup'

but the logged shell command was:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup

which will not run if copied and pasted back into the
user's terminal, as the importdemocourse shell script is unquoted.
2022-07-25 22:46:53 +02:00
Keith Grootboom
55bc4c63fa fix: change kubectl wait to look at deployments not pods
When waiting for pods, it's possible that the deployment may be
complete but, because other pods may have been Evicted or Killed, the
wait wait condition completes.
2022-07-25 22:46:45 +02:00
Kyle McCormick
0ae59a82a6
fix: avoid double-rendering mounts to docker-compose.tmp.yml (#669)
In certain code paths, such as in `tutor local quickstart`,
`process_mount_points` is called more than once in the same process,
causing mounts to be added to `COMPOSE_LOCAL[_JOBS]_TMP` redundantly.
As a result, docker-compose[.jobs].tmp.yml was occasionally being
rendered with duplicate volume specifiers. Some versions of Docker
Compose ignored this; other versions warned or threw an error.

In order to make `process_mount_points` tolerant to being called
multiple times, we wrap its volume-adding callbacks within a new
hooks context. This allows us to clear said hooks context every
time `process_mount_points` is called, essentially making the
function idempotent.

Co-authored-by: Régis Behmo <regis@behmo.com>
2022-07-25 15:58:49 +02:00
Carlos Muniz
d27e8d5ba7
docs: add missing config parameters from defaults.yaml
Document config parameters from defaults.yaml that were missing from the docs.


Co-authored-by: Carlos Muniz <cmuniz@trcil.org>
2022-07-25 15:43:40 +02:00
Ed D'Azzo
2e41de1af9 Added a record of the bugfix to the CHANGELOG 2022-07-25 15:25:25 +02:00
Ed D'Azzo
5e81a754c4 Fixed or syntax 2022-07-25 15:25:25 +02:00
Ed D'Azzo
67358c6b7a Omit patchesStrategicMerge when there are no custom resources 2022-07-25 15:25:25 +02:00
David Ormsbee
bb907b4de3 v14.0.3 (2022-07-09)
- [Bugfix] Build openedx-dev Docker image even when the host user is root, for instance on Windows. (by @regisb)
- [Bugfix] Patch nutmeg.1 release with [LTI 1.3 fix](https://github.com/openedx/edx-platform/pull/30716). (by @ormsbee)
- [Improvement] Make it possible to override k8s resources in plugins using `k8s-override` patch. (by @foadlind)
2022-07-09 19:01:58 +02:00
Régis Behmo
5b5700eff4 fix: build openedx-dev image when host user is root
Sometimes, the host user is root: this may happen when tutor is run with
"sudo" (which is not recommended) or on Windows. In such cases, building
the image should not fail, but default to a reasonable user. Also, when
we pass an invalid APP_USER_ID as a build arg, then we should fail with
an explicit message.

See this conversation:
https://discuss.overhang.io/t/problem-with-dev-image-build-useradd-uid-0-is-not-unique/2406
2022-07-09 00:09:20 +02:00
David Ormsbee
ac3b4e3065 fix: apply LTI 1.3 XBlock patch fix 3.4.7 over nutmeg.1
The actual merge of this into open-release/nutmeg.master is here:
  https://github.com/openedx/edx-platform/pull/30716
2022-07-08 23:48:45 +02:00
Prakash Divy
9a63dc70ce docs: Update help command in local.rst 2022-07-04 06:30:29 +02:00
Foad Lind
b8f773a5ef feat: Make it possible to customize k8s resources
Currently there is no way for plugins to customize Kubernetes resources
defined in Tutor deployment manifests.
This change makes that possible by taking advantage of the strategic
merge patching mechanism in `kustomization.yml`.
Any resource definition in a `k8s-override` patch in a plugin will
override the resource defined by Tutor, provided that their names match.

Reference: https://github.com/overhangio/tutor/pull/675
2022-06-28 10:30:24 +02:00