6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-13 06:37:46 +00:00
Commit Graph

1247 Commits

Author SHA1 Message Date
Crist
d49d166383 Fix: double pulling mongodb image when upgrading from Koa to Lilac. ref: https://discuss.overhang.io/t/double-pulling-mongo-images-when-upgrading-from-koa-to-lilac/1625 2021-06-22 12:28:26 +02:00
Shimul Chowdhury
31a41f81c3 feat: env only mode for config generation
chore: added -e flag to the changelog
2021-06-22 12:25:29 +02:00
Régis Behmo
e7d3e722ce fix: "upstream sent too big header" nginx errors
It was reported that existing users could not sign-in after a Koa -> Lilac
upgrade. This is due to the fact that cookies exceed the 4kb limit, which is
rejected by nginx. The fix consists in increasing this limit to 8kb:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size

This issue was reported here: https://discuss.overhang.io/t/lilac-upgrade-login-issues/1617
Note that I was not able to reproduce the issue. The fix was tested by the
issue reporter.
2021-06-22 08:58:50 +02:00
Régis Behmo
775a20b156 ci: fix sync with private repository 2021-06-14 08:16:40 +02:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
915551268c v11.3.1 (2021-06-08)
- [Improvement] Avoid permission issues in Kubernetes/Openshift for users who do not have the rights to edit their namespace.
- [Improvement] Better Kubernetes object creation.
2021-06-08 20:17:44 +02:00
Régis Behmo
4a451844e6 fix: avoid namespace edition for k8s users without access rights
In most cases, it makes very little sense to edit the namespace that an
application is running in. Quite often, users are granted access to just one
namespace and don't have the necessary rights to edit the namespace -- and for
good security reasons. In such cases, the k8s namespace object already exists
and there is no need for the user to edit or create it. Here, what we do is
that we create the namespace only if it does not exist. This should solve quite
a few permission issues, notably for Openshift users.
2021-06-08 19:54:09 +02:00
Régis Behmo
9b6b770dd7 feat: better object creation in "k8s start"
When running "tutor k8s start", it's unnecessary to attempt to create volumes
and namespace multiple times.
2021-06-08 19:54:09 +02:00
Régis Behmo
900572fdae v11.3.0 (2021-05-18)
- 💥[Security] Disable python-evaluated input by default as we don't run codejail.
- [Bugfix] Fix missing discussion notifications.
- [Improvement] Better error logging when loading a plugin from an incompatible version.
2021-06-06 14:40:24 +02:00
Régis Behmo
e990291d16 feat: upgrade pinned requirements to click 8+
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.

We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
2021-06-06 14:38:59 +02:00
Régis Behmo
a79d194bbc security: disable python-evaluated input by default
As described in issue #284, tutor does not come with codejail enabled out of
the box. Actually, we don't even have a working plugin, yet. To prevent users
from running unsafe code, we explicitely disable python-evaluated input by
disabling the "python" interpreter. This might break some courses; thus, this
is a non-backward compatible change.
2021-06-06 14:37:46 +02:00
liupengcheng
58c77ad3fb fix: discussion email notification not work.
Discussion email notification relies on edx_ace and celery routing
key for ace is not set properly by default, which causes the send
mail task is never consumed.

Close #439.
2021-05-25 10:00:45 +02:00
Régis Behmo
1b9334e37e fix: sync with private repo in CI 2021-05-20 12:48:10 +02:00
Régis Behmo
4cc48c966f feat: better logging during plugin loading failure
When upgrading Tutor plugins to the next release, I often end up with a
virtualenv that contains plugins that depend on different versions of
tutor-openedx. This causes a crash that did not log the name of the responsible
package. For instance:

    Traceback (most recent call last):
      File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
        load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 37, in main
        add_plugin_commands(cli)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/plugins.py", line 137, in add_plugin_commands
        for plugin in plugins.iter_installed():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 401, in iter_installed
        yield from Plugins.iter_installed()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 362, in iter_installed
        for plugin in PluginClass.iter_installed():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 199, in iter_installed
        for plugin in cls.iter_load():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 233, in iter_load
        yield cls(entrypoint)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 220, in __init__
        super().__init__(entrypoint.name, entrypoint.load())
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2442, in load
        self.require(*args, **kwargs)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2465, in require
        items = working_set.resolve(reqs, env, installer, extras=self.extras)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
        raise VersionConflict(dist, req).with_context(dependent_req)
    pkg_resources.VersionConflict: (tutor-openedx 11.2.11 (/home/regis/projets/overhang/repos/overhang/tutor), Requirement.parse('tutor-openedx<13.0.0,>=12.0.0'))

In this commit, we introduce an error log that displays the name and location
of the package. E.g:

    Failed to load entrypoint 'minio = tutorminio.plugin' from distribution tutor-minio 12.0.0
2021-05-20 12:22:52 +02:00
Régis Behmo
553b08636a fix: make binary release compatible with Ubuntu 18.04 in CI
Switching from Travis CI to Github Actions made us switch from Ubuntu 16.04 to
20.04. This causes errors for users running older versions of Ubuntu. I don't
think we can support Ubuntu 16.04 anymore, as it has gone out of support, but
we can at least attempt to support 18.04.

For reference, the reported error seems to be:

    [17871] Error loading Python lib '/tmp/_MEIa1GHWz/libpython3.6m.so.1.0':
    dlopen: /lib/x86_64-linux-gnu/libm.so.6: version GLIBC_2.29’ not found
    (required by /tmp/_MEIa1GHWz/libpython3.6m.so.1.0)`

See: https://discuss.overhang.io/t/error-upgrading-from-v11-2-9-to-v11-2-10/1557
2021-05-19 23:43:06 +02:00
Régis Behmo
ed1e2e49fe v11.2.11 (2021-05-18)
- [Feature] Add redis database configuration for both cache and celery.
2021-05-18 10:05:21 +02:00
liupengcheng
028162fea4 feat: Make openedx redis db configurable.
Redis db 0 is used for celery broker and db 1 is used for cache
by default.
2021-05-18 09:35:36 +02:00
Régis Behmo
5658b46715 v11.2.10 (2021-05-17)
- [Security] Apply Django security patches by upgrading from 2.2.20 to 2.2.23.
- [Bugfix] Fix video unit completion (see [pull
  request](https://github.com/edx/edx-platform/pull/27230)).
2021-05-17 21:29:52 +02:00
Régis Behmo
6af6390fb1 refactor: migrate from Travis.ci to GitHub Actions
GitHub Actions now performs the following tasks:

- run tests on every PR
- sync with git.overhang.io on push
- build binary releases on tags

Travis.CI was completely removed from this repo.
2021-05-17 21:29:52 +02:00
Régis Behmo
26d14457ec security: upgrade Django to 2.2.23
Also, fix video unit completion in LMS.
2021-05-17 14:25:13 +02:00
Régis Behmo
306d270a3a v11.2.9 (2021-05-12)
- [Bugfix] Fix crashing installation because of a major release of all
Pallets projects.
- [Bugfix] Fix crash in `local quickstart -p` command.
- [Bugfix] Fix 502 error on request to lms with header larger than the
maximum uwsgi buffer size.
2021-05-12 10:14:27 +02:00
Régis Behmo
1a5e2e005b fix: requirements of pallets project
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/

We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.

The following error was being raised:

    $ tutor plugins list
    Traceback (most recent call last):
      File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
	from tutor.commands.cli import main
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
	import click_repl
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
	import click._bashcomplete
    ModuleNotFoundError: No module named 'click._bashcomplete'

Close #430.
2021-05-12 10:12:31 +02:00
Régis Behmo
6d92fe2d4a fix: crash during local quickstart -p
When running `tutor local quickstart -p` we were getting the following error:

    Usage: custom [OPTIONS] ARGS...
    Try 'custom --help' for help.

    Error: Missing argument 'ARGS...'.

The docker-compose command sometimes accept a single command ("pull") with zero
argument.

See: https://discuss.overhang.io/t/local-quickstart-not-working-when-pullimages-enabled/1526
2021-05-07 17:01:09 +02:00
wey-gu
452cd229f3 fix: 502 error on request to lms with large header
According to [1], request to uwsgi with header
larger than the default value 4096 will end up
with below error as #426:
  invalid request block size: 4123 (max 4096)...skip
  hr_instance_read(): Connection reset by peer
  [plugins/http/http.c line 647]

This commit fixes it by changing the maximum
buffer size as 4096

Also: Similar issue was already identifed and
fixed in [2] on tutor-ecommerce

[1] https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
[2] 6df2c99362

Close #426
2021-05-01 16:32:19 +02:00
Régis Behmo
dfd3d179e9 v11.2.8 (2021-04-27)
- [Bugfix] Fix parsing of YAML-formatted values in ``config save --set
  KEY=VALUE`` commands, in use for instance with Ecommerce.
2021-04-27 09:26:01 +02:00
Eric Herrera
389dd96fdd fix: YamlParamType supports line terminators
This fix allows using a multiple line formatted Yaml string as input for setting a Tutor config value.
2021-04-25 09:14:21 +02:00
Régis Behmo
2e0f136a23 v11.2.7 (2021-04-23)
- [Security] Apply security patch
  [27394](https://github.com/edx/edx-platform/pull/27394).
- [Feature] Add patches to extend python requirements installation process in
  openedx and openedx-dev Dockerfiles.
- [Improvement] Apply edx-platform patches during Docker image build using
  tutor patch 'openedx-dockerfile-git-patches-default'.
2021-04-23 20:24:08 +02:00
Régis Behmo
b211cb0a0f security: apply security patch 27394
See: https://discuss.openedx.org/t/security-patch-for-xss-fixes/4820
2021-04-23 20:23:11 +02:00
liupengcheng
96be0b7401 docs: fix redis port configuration. 2021-04-20 09:37:34 +02:00
Régis Behmo
336cb79fa8 refactor: better config type checking
I stumbled upon a bug that should have been detected by the type
checking. Turns out, considering that config is of type Dict[str, Any]
means that we can use just any method on all config values -- which is
terrible. I discovered this after I set `config["PLUGINS"] = None`:
this triggered a crash when I enabled a plugin.
We resolve this by making the Config type more explicit. We also take
the opportunity to remove a few cast statements.
2021-04-18 18:02:02 +02:00
Régis Behmo
887ba31e09 code: lint unused imports
Unused imports are only warnings in pylint; here we make sure that they
are processed as errors.
2021-04-18 18:02:02 +02:00
Eric Herrera
17088e2fc6 feat: Conditional edx-platform patching
During Docker images build process, apply custom edx-platform patches when tutor patch 'openedx-dockerfile-git-patches-default' is defined or apply current release patches in other case. It avoids possible conflicts between the actually used edx-platform version and the current release patches.
2021-04-15 07:32:39 +02:00
Eric Herrera
a05486e7b0 feat: openedx Dockerfile python requirements extension patch
Add patches to extend python requirements installation process in openedx and openedx-dev Dockerfiles
2021-04-15 07:32:39 +02:00
Calvin
9a16391d45 docs: mention MySQL external version 2021-04-14 21:57:36 +02:00
Régis Behmo
e532f34909 v11.2.6 (2021-04-09)
- [Improvement] Upgrade all services to open-release/koa.3.
- [Feature] Make it possible to build the openedx Docker image with a custom openedx-i18n version with the ``--build-arg OPENEDX_I18N_VERSION=custom`` command line argument.
2021-04-09 10:16:17 +02:00
Régis Behmo
4a65f24ae7 feat: upgrade all services to open-release/koa.3
We remove security patches and custom fixes which are now part of koa.3.

We take the opportunity to make it possible to build the openedx Docker image
without relying on a corresponding openedx-i18n repo tag: often, we want to
test whether the image simply builds successfully, and we don't need up-to-date
translations. For those cases, it's now possible to pass the `-a
OPENEDX_I18N_VERSION=oldertag` build argument.
2021-04-09 10:13:34 +02:00
Régis Behmo
5cff808ee7 docs: non-ascii locales must specify a content-type
See: https://discuss.overhang.io/t/translating-login-registration-page/1378/3

This should also address issue #410.
2021-04-06 12:08:30 +02:00
Régis Behmo
f2ab27199e v11.2.5 (2021-03-30)
- [Bugfix] Fix edge case where `PLUGINS` entry is null in config.yml.
- [Bugfix] Fix missing py2neo dependency in `images build openedx` (#411).
2021-03-30 09:25:39 +02:00
Régis Behmo
c01f4476b8 fix: TypeError when PLUGINS is None
When the PLUGINS config entry is None (`PLUGINS:`), the following error
was being triggered:

  File "/.../tutor/tutor/plugins.py",
  line 304, in is_enabled
      return name in config.get(CONFIG_KEY, [])
      TypeError: argument of type 'NoneType' is not iterable
2021-03-30 09:23:16 +02:00
Régis Behmo
8db1495497 fix: missing py2neo dependency breaks images build openedx
All releases of py2neo prior to 4.0.0 were stripped from Pypi, so we
need to install this dependency from source.

See:
https://github.com/openedx/build-test-release-wg/issues/37
https://github.com/edx/edx-platform/pull/27168

Close #411.
2021-03-30 09:13:44 +02:00
Régis Behmo
740e6baf2e fix: stricter type checking when loading plugins
This allows us to get rid of a few `cast(...)` statements.

This kind of check would have avoided this issue:
https://discuss.overhang.io/t/cant-enable-keycloak-oauth2-backend-with-yml-plugin/1380
2021-03-29 09:48:53 +02:00
Régis Behmo
d184bb2bda fix: a few typing minor issues 2021-03-29 09:06:13 +02:00
Régis Behmo
1c5dd4626e docs: link to the maintainers team handbook 2021-03-25 17:13:22 +01:00
Régis Behmo
98ed40c800 v11.2.4 (2021-03-17)
- [Bugfix] Fix "Invalid Request" error during SAML authentication
(thanks @Ant1x!).
- [Feature] Add `images build --pull` option to update base images.
- [Improvement] Annotate types all over the Tutor code base.
- [Bugfix] Fix parsing of YAML CLI arguments that include equal "="
signs.
- [Bugfix] Fix minor edge case in `long_to_base64` utility function.
- [Improvement] Add openedx patches to add settings during build
process.
2021-03-17 18:21:16 +01:00
Régis Behmo
f688a279dc Actually remove the images build --pull option
This option is mostly useless to us in CI, as it attempts (and fails) to
update the base image when building the dev image. For good reason: the
base image of the dev image is the latest openedx, which has not been
pushed to docker.io at the time of building -_-
2021-03-17 18:21:16 +01:00
Régis Behmo
27a7893c45 Clean some minor stuff from Makefile 2021-03-17 18:21:16 +01:00
Régis Behmo
b00bae9e72 minor: add missing changelog entry 2021-03-17 17:33:06 +01:00
Régis Behmo
b557cf9b43 Simplify the release process
Instead of pushing to multiple remotes, push to just one. It will then
sync changes to the other.
2021-03-17 17:33:06 +01:00
Ant1x
0c88d73914 Add X-Forwarded-Port header 2021-03-17 17:10:12 +01:00
Régis Behmo
914dbc53be Add images build --pull option to pull base images on build
This will be useful in CI, where we want to automatically update base
images as soon as a security fix comes out.
2021-03-17 16:02:46 +01:00