6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-10-31 18:52:34 +00:00
Commit Graph

1382 Commits

Author SHA1 Message Date
Régis Behmo
c0d79463ff v14.2.2 (2022-11-29)
- [Bugfix] Fix `jinja2.exceptions.TemplateSyntaxError: expected token 'end of statement block', got '|'` error by bumping the minimum required version of the Jinja2 package.
- [Feature] Add support for MongoDB SSL, authentication source, mechanism and replica set via the `MONGODB_USE_SSL`, `MONGODB_AUTH_MECHANISM`, `MONGODB_AUTH_SOURCE`, `MONGODB_REPLICA_SET` settings. (by @zakum1 and @regisb)
- [Bugfix] Fix tag of "openedx" development Docker image. Previously, this Docker tag did not include the Tutor version. As a consequence, a different cached image could be used in some cases. For instance: when running `tutor dev run` commands. Now, the image tag is "openedx-dev:TUTOR_VERSION".
- [Bugfix] Fix name of Swahili locale: it is "sw-ke" and not "sw" (by @regisb).
- [Security] Apply drag-n-drop v2 xblock [security patch](https://discuss.openedx.org/t/upcoming-security-release-xblock-drag-and-drop-v2/8768/7). (by @regisb)
2022-11-29 11:27:35 +01:00
Régis Behmo
4a6896d892 docs: adjust web proxy tutorial 2022-11-29 09:49:57 +01:00
Daniel Herrmann
bb137c16d6 Added more info to required headers and a pitfall regarding misleading wording in the quickstart dialogue 2022-11-29 09:49:57 +01:00
Régis Behmo
0e8f55798c sec: fix XSS vulnerability in drag-n-drop v2 xblock
Vulnerability is fixed by upgrading the xblock from v2.3.5 to v3.0.0.
See announcement:
https://discuss.openedx.org/t/upcoming-security-release-xblock-drag-and-drop-v2/8768
2022-11-29 09:05:38 +01:00
Régis Behmo
ff0e8f7140 fix: pin openedx dev Docker image tag with tutor version
When running:

    tutor dev run -m /path/to/edx-platform lms
    pip install -r requirements/edx/development.txt

I realised that I was re-installing packages that should already have been
present in the image. The reason for that was that I was running an outdated
version of the dev version of the openedx Docker image. This happens because
`tutor dev run` does not trigger an image re-build.

We solve this issue by pinning the openedx dev Docker image tag to the current
tutor version.
2022-11-29 09:02:50 +01:00
james
7b72a5a910 feat: support for mongodb SSL+auth source/mech+replica set
This change builds upon a previously proposed PR:
https://github.com/overhangio/tutor/pull/437

There was another long conversation about this topic here:
https://github.com/overhangio/tutor-forum/pull/10#issuecomment-1314799915

We could have supported the MongoDB auth/replica set/ssl parameters as part of
the MongoDB host URI, but then this URI is not supported in the forum plugin,
which uses an old version of the mongoid client. We were hoping that the client
would have been upgraded by now, but it's not been upgraded for a long time.

The changes introduced here are 100% backward-compatible. The forum plugin will
have to be updated to take into account the new parameters.
2022-11-28 13:09:13 +01:00
Régis Behmo
b8ca06a42c fix: Swahili locale name
Note that this locale ships with edx-platform by default.
2022-11-28 12:23:02 +01:00
Régis Behmo
13104809c1 docs: add missing changelog entry 2022-11-28 10:07:02 +01:00
Régis Behmo
5929ecbc2e fix: TemplateSyntaxError with old jinja2
The following syntax is only supported in jinja2>=2.10:

    {% set jwt_rsa_key | rsa_import_key %}{{ JWT_RSA_PRIVATE_KEY }}{% endset %}

Thus, we bump the minimal working version of jinja2 in the base requirements.

See discussion: https://discuss.openedx.org/t/error-while-tutor-local-quickstart/8796
2022-11-28 10:03:59 +01:00
Régis Behmo
c9ac12fdd7 v14.2.1 (2022-11-24)
- [Improvement] Auto-completion of `plugins` and `config` arguments: `plugins enable/disable NAME`, `plugins install PATH`, `config save --set KEY=VAL`, `config save --unset KEY`, `config printvalue KEY`. (by @regisb)
- [Bugfix] Fix minimum click version (>= 8.0.0) when installing tutor from pip.
- [Improvement] Enable CORS by default for both LMS and CMS by moving those settings to the `common_all` partial. (by @arbrandes)
2022-11-24 16:25:34 +01:00
Régis Behmo
04258b9f9b docs: add changelog entry for click fix 2022-11-24 16:24:18 +01:00
Régis Behmo
29eb3398a2 feat: auto-complete config save/printroot arguments 2022-11-24 16:21:57 +01:00
Régis Behmo
ee09612326 feat: auto-complete plugins arguments
Support auto-completion of plugin name and path arguments in the `tutor
plugins` commands.
2022-11-24 16:21:57 +01:00
Régis Behmo
6069f91cce docs: fix nightly typo
Co-authored-by: Adolfo R. Brandes <arbrandes@tcril.org>
2022-11-24 16:15:15 +01:00
Régis Behmo
c886b72ae7 docs: migrate to scriv to manage changelog
Changelog management was starting to be a hassle:

- there were conflicts every time a PR was merged
- there were conflicts every time we merged the nightly branch in the new
  release branch, or vice versa.

Now, all changelog entries are stored as separate files in changelog.d,
including nightly. Nightly entries will be collected for every major release.
2022-11-24 16:15:15 +01:00
Régis Behmo
e63d5c32e3 fix: minimum click version is 8.0
An issue was reported here: https://app.slack.com/client/T02SNA1T6/CGE253B7V

CompletionItem was introduced in 8.0, so click needs to be upgraded when
we `pip install tutor`.
2022-11-23 18:24:05 +01:00
Adolfo R. Brandes
3e32f88c15 feat: common CORS settings for LMS and CMS
The ENABLE_CORS_HEADERS feature flag is already true for the LMS.
Instead of duplicating it for Studio via yaml settings, make this a
common Django setting to both LMS and CMS and all their environments.
2022-11-21 13:51:56 +01:00
Régis Behmo
5006f6edc9 v14.2.0 (2022-11-21)
- [Improvement] Auto-complete implicit `local/dev --mount /path/to/...` options. (by @regisb)
- 💥[Feature] Strong typing of action and filter hooks: this allows us to detect incorrect calls to `actions.add` or `filters.add` early. Strong typing forces us to break the `do` and `apply` API by removing the `context` named argument. Developers should replace `do(context=...)` by `do_from_context(..., )` (and similar for `apply`).
2022-11-21 10:10:44 +01:00
Régis Behmo
986c3e2305 chore: upgrade requirements 2022-11-21 10:10:07 +01:00
Régis Behmo
f8b5cbc657 feat: auto-complete --mount args
When typing `tutor local run --mount /path/to/edx-pl<TAB>`, the mount option
should be auto-completed to the full edx-platform repo path. That is, if shell
completion is enabled:
https://docs.tutor.overhang.io/install.html#shell-autocompletion

Here, we make sure that the implicit form of the `--mount` argument is properly
auto-completed. We are unable to get completion to work in the explicit form,
because args that include colons do not even reach the `shell_completion`
method.
2022-11-17 14:50:52 +01:00
Régis Behmo
33e4f33afe feat: strongly typed hooks
Now that the mypy bugs have been resolved, we are able to define more precisely
and cleanly the types of Actions and Filters.

Moreover, can now strongly type named actions and hooks (in consts.py). With
such a strong typing, we get early alerts of hooks called with incorrect
arguments, which is nothing short of awesome :)

This change breaks the hooks API by removing the `context=...` argument. The
reason for that is that we cannot insert arbitrary arguments between `P.args,
P.kwargs`: https://peps.python.org/pep-0612/#the-components-of-a-paramspec

> A function declared as def inner(a: A, b: B, *args: P.args, **kwargs:
> P.kwargs) -> R has type Callable[Concatenate[A, B, P], R]. Placing
> keyword-only parameters between the *args and **kwargs is forbidden.

Getting the documentation to build in nitpicky mode is quite difficult... We
need to add `nitpick_ignore` to the docs conf.py, otherwise sphinx complains
about many missing class references. This, despite upgrading almost all doc
requirements (except docutils).
2022-11-15 14:58:36 +01:00
Régis Behmo
8aeeb7e09c chore: replaced unnamed hook filters by named equivalents 2022-11-14 12:25:28 +01:00
Régis Behmo
fd2d3dea10 v14.1.2 (2022-11-02)
- [Security] Fix edx-platform XSS vulnerability on "next" parameter. (by
  @regisb)
2022-11-02 10:42:55 +01:00
Régis Behmo
b46a7b0fa8 sec: fix edx-platform xss vulnerability on "next" parameter 2022-11-02 10:42:10 +01:00
Régis Behmo
f95c394e38 fix: print proper command on check_output
Note that the `check_output` command is mostly unused.
2022-10-26 16:46:04 +02:00
Régis Behmo
7541433729 v14.1.1 (2022-10-25)
- [Security] Fix xblock ajax handler vulnerability. (by @regisb)
- [Improvement] Use web proxy gzip encoding to improve bandwidth. We
  observe a 75% size reduction on the LMS dashboard. (by @ghassanmas)
2022-10-26 12:28:16 +02:00
Régis Behmo
1161f925a4 feat: add py.typed for typing extensions
Extensions which use the tutor API need to know that the tutor package
is typed. For that we add the py.typed file.

Also, we fix the signature of get_typed.

See:
https://mypy.readthedocs.io/en/stable/installed_packages.html#installed-packages
2022-10-26 12:28:16 +02:00
Régis Behmo
7eeccfb5f5 chore: mark Python 3.6 as deprecated 2022-10-26 12:24:07 +02:00
Régis Behmo
3ba5365537 sec: fix xblock ajax handler vulnerability 2022-10-25 18:56:40 +02:00
Ghassan Maslamani
7d32179038 feat: use encode gzip for caddy
Compressing assests would lead to readuce transfer size.
  As testing with frontend-app-learning/Olive, the network traffic
  before was about ~4MB, after this it became ~1MB.

  This change was suggested by Google Lighthouse[1], there are of
  course more suggestion but this was one the easiest and one of most
  impactful.

  Also check orignal PR overhangio/tutor-mfe/pull/64 for more
  info.
  [1]: https://web.dev/uses-text-compression
2022-10-19 10:46:01 +02:00
Régis Behmo
7453e70fa0 v14.1.0 (2022-10-10)
- [Improvement] Upgrade Scorm XBlock to v14.0.0. (by @regisb)
- 💥[Improvement] The Richie plugin was transferred to the Openfun organization; thus, it is no longer officially supported and it is removed from the default set of plugins that ships with
`pip install tutor[full]` or the Tutor pre-compiled binary. Users are encouraged to uninstall the `tutor-richie` Python package and install the `tutor-contrib-richie` package instead.
- [Feature] Upgrade edx-platform i18n strings to nutmeg.2. (by @regisb)
2022-10-10 10:14:36 +02:00
Régis Behmo
c0fb64714f feat: bump scorm to v14.0.0 2022-10-04 09:25:41 +02:00
Michael Wheeler
fd037fcb4d Remove warning about building tutor-mfe on ARM systems 2022-10-03 14:15:24 +02:00
Frank Anderson
871d293719 Update local.rst 2022-10-03 14:09:34 +02:00
Frank Anderson
55684ebd49 Updated local.rst with changes from #716 2022-10-03 14:09:34 +02:00
Frank Anderson
5fdede8511 Fix incorrect information about running config save --interactive
Running config save --interactive doesn't create a config.yml file in the current directory. It instead creates it in the project root.
2022-10-03 14:09:34 +02:00
Régis Behmo
020f6d1137 depr: richie plugin is no longer official
The richie plugin was transferred to the openfun GitHub org. As a consequence,
it is no longer part of the list of official plugins.

See: https://github.com/overhangio/tutor-contrib-richie/pull/5
2022-09-30 12:09:31 +02:00
Régis Behmo
dbb528aba4 chore: partial reqs upgrade
Oauthlib upgrade was suggested by dependabot:
https://github.com/overhangio/tutor/pull/723/files

We did not perform a full upgrade because sphinx-rtd-theme does not support
docutils 0.19, yet: https://github.com/readthedocs/sphinx_rtd_theme/issues/1323
2022-09-29 13:55:09 +02:00
Kyle McCormick
98d756243f docs: add troubleshootoing docs for mysql connectivity errors
Adding docs to capture some common errors I've
been seeing in Slack. I've experienced these
myself as well.
2022-09-12 06:38:44 +02:00
Régis Behmo
b4f905c2aa feat: upgrade i18n openedx strings to nutmeg.2
Strings could not be pulled from transifex because the file names were
incorrect. This is now fixed and we are now able to pull the i18n strings from
the nutmeg.2 tag.
2022-09-08 11:20:49 +02:00
Sofiane Bebert
3f1dd832e4 v14.0.5 (2022-08-29)
- [Bugfix] Fix MongoDB replica set connection error resulting from edx-platform's pymongo (3.10.1 -> 3.12.3) upgrade ([edx-platform#30569](https://github.com/openedx/edx-platform/pull/30569)). (by @ormsbee)
- [Feature] Upgrade all applications to open-release/nutmeg.2.
2022-08-29 10:10:29 +02:00
David Ormsbee
417feeb0f3 fix: replica set error from pymongo 3.10 -> 3.12 upgrade
The pymongo dependency for edx-platform was updated (3.10.1 to 3.12.3)
in https://github.com/openedx/edx-platform/pull/30569

This caused the following error when running the edx-platform database
migration split_modulestore_django.0002_data_migration as part of
`tutor dev quickstart`:

  pymongo.errors.ServerSelectionTimeoutError: client is configured to
  connect to a replica set named '' but this node belongs to a set named
  'None', Timeout: 30s, Topology Description: <TopologyDescription id:
  62bdbaf182687350acf1aeec, topology_type: Single, servers:
  [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt:
   None, error=ConfigurationError("client is configured to connect to a
   replica set named '' but this node belongs to a set named 'None'")>]>

This commit explicitly sets replicaSet to None to indicate that it's a
standalone MongoDB instance. I also had to remove the CONTENTSTORE entry
from auth.yml because edx-platform's devstack.py assumes it has a
non-null value (set in common.py), and devstack.py executes before
tutor's development.py can set this replicaSet value.
2022-08-21 09:37:26 +01:00
Sofiane Bebert
6674e267de feat: upgrade all apps to open-release/nutmeg.2 2022-08-21 09:15:00 +01:00
Régis Behmo
66bef2e14f v14.0.4 (2022-07-29)
- [Feature] Add the `-m/--mount` option to `tutor dev quickstart`.
- [Bugfix] Fix `tutor dev start -m /path/to/frontend-app-learning` by introducing dev-specific `COMPOSE_DEV_TMP` and `COMPOSE_DEV_JOBS_TMP` filters (by @regisb).
- [Bugfix] Log the shell commands that Tutor executes more accurately. (by @kdmccormick)
- [Bugfix] `tutor dev quickstart` would fail under certain versions of docker-compose due to a bug in the logic that handled volume mounting. (by @kdmccormick)
- [Bugfix] The `tutor k8s start` command will succeed even when `k8s-override` and `kustomization-patches-strategic-merge` are not specified. (by @edazzocaisser)
- [BugFix] `kubectl wait` checks deployments instead of pods as it could hang indefinitely if there are extra pods in a broken state. (by @keithgg)
2022-07-29 19:54:15 +02:00
Régis Behmo
52cf0cc1ca feat: add -m/--mount option to dev quickstart 2022-07-29 19:53:02 +02:00
Régis Behmo
a2a3c022b8 fix: bind-mount in dev-specific services
The -m/--mount option makes it possible to bind-mount volumes at runtime. The
volumes are declared in a local/docker-compose.tmp.yml file. The problem with
this approach is when we want to bind-mount a volume to a service which is
specific to the dev context. For instance: the "learning" service when the MFE
plugin is enabled.

In such a case, starting the service triggers a call to `docker-compose stop`
in the local context. This call fails because the "learning" service does not
exist in the local context. Note that this issue only seems to occur with
docker-compose v1.

To resolve this issue, we create two additional filters for
the dev context, which emulate the behaviour of the local context. With this approach, we convert the -m/--mount arguments right after they are parsed. Because they are parsed just once, we can get rid of the de-duplication logic initially introduced with the COMPOSE_CLI_MOUNTS context.

Close #711. Close also https://github.com/overhangio/tutor-mfe/issues/57.
2022-07-29 19:53:02 +02:00
Kyle McCormick
8345b7ab93 temp: shim shlex.join for Python 3.7 compatability 2022-07-25 22:46:53 +02:00
Kyle McCormick
6a3138152f fix: accurately log shell commands when running jobs
Whenever Tutor executes a shell command, it logs out said
command in order to aid in end user understanding/debugging.

In some cases (notably, when running jobs in containers)
the logged command was not accurately quoted. The command
was run correctly, because it was passed in pieces to
``subprocess.Popen``, which correctly joins the pieces together
into a valid POSIX shell command; however, the logged version
of the command was constructed by simply joining the pieces
with spaces. This usually works, but breaks down when running
complex shell commands with nested quoting.

This commit changes the logging to use ``shlex.join``, which
joins command pieces together in a POSIX-compliant way,
presumably the same way as ``subprocess.Popen``.

Example:

    tutor local importdemocourse

runs the shell command:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c 'echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup'

but the logged shell command was:

    docker-compose -f /home/kyle/.local/share/tutor/env/local/docker-compose.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.yml -f /home/kyle/.local/share/tutor/env/local/docker-compose.jobs.tmp.yml run --rm cms-job sh -e -c echo "Loading settings $DJANGO_SE... (several more script lines) ...eindex_course --all --setup

which will not run if copied and pasted back into the
user's terminal, as the importdemocourse shell script is unquoted.
2022-07-25 22:46:53 +02:00
Keith Grootboom
55bc4c63fa fix: change kubectl wait to look at deployments not pods
When waiting for pods, it's possible that the deployment may be
complete but, because other pods may have been Evicted or Killed, the
wait wait condition completes.
2022-07-25 22:46:45 +02:00
Kyle McCormick
0ae59a82a6
fix: avoid double-rendering mounts to docker-compose.tmp.yml (#669)
In certain code paths, such as in `tutor local quickstart`,
`process_mount_points` is called more than once in the same process,
causing mounts to be added to `COMPOSE_LOCAL[_JOBS]_TMP` redundantly.
As a result, docker-compose[.jobs].tmp.yml was occasionally being
rendered with duplicate volume specifiers. Some versions of Docker
Compose ignored this; other versions warned or threw an error.

In order to make `process_mount_points` tolerant to being called
multiple times, we wrap its volume-adding callbacks within a new
hooks context. This allows us to clear said hooks context every
time `process_mount_points` is called, essentially making the
function idempotent.

Co-authored-by: Régis Behmo <regis@behmo.com>
2022-07-25 15:58:49 +02:00