6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-05 04:48:00 +00:00
Commit Graph

432 Commits

Author SHA1 Message Date
liupengcheng
028162fea4 feat: Make openedx redis db configurable.
Redis db 0 is used for celery broker and db 1 is used for cache
by default.
2021-05-18 09:35:36 +02:00
Régis Behmo
5658b46715 v11.2.10 (2021-05-17)
- [Security] Apply Django security patches by upgrading from 2.2.20 to 2.2.23.
- [Bugfix] Fix video unit completion (see [pull
  request](https://github.com/edx/edx-platform/pull/27230)).
2021-05-17 21:29:52 +02:00
Régis Behmo
26d14457ec security: upgrade Django to 2.2.23
Also, fix video unit completion in LMS.
2021-05-17 14:25:13 +02:00
Régis Behmo
306d270a3a v11.2.9 (2021-05-12)
- [Bugfix] Fix crashing installation because of a major release of all
Pallets projects.
- [Bugfix] Fix crash in `local quickstart -p` command.
- [Bugfix] Fix 502 error on request to lms with header larger than the
maximum uwsgi buffer size.
2021-05-12 10:14:27 +02:00
Régis Behmo
1a5e2e005b fix: requirements of pallets project
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/

We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.

The following error was being raised:

    $ tutor plugins list
    Traceback (most recent call last):
      File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
	from tutor.commands.cli import main
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
	import click_repl
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
	import click._bashcomplete
    ModuleNotFoundError: No module named 'click._bashcomplete'

Close #430.
2021-05-12 10:12:31 +02:00
Régis Behmo
6d92fe2d4a fix: crash during local quickstart -p
When running `tutor local quickstart -p` we were getting the following error:

    Usage: custom [OPTIONS] ARGS...
    Try 'custom --help' for help.

    Error: Missing argument 'ARGS...'.

The docker-compose command sometimes accept a single command ("pull") with zero
argument.

See: https://discuss.overhang.io/t/local-quickstart-not-working-when-pullimages-enabled/1526
2021-05-07 17:01:09 +02:00
wey-gu
452cd229f3 fix: 502 error on request to lms with large header
According to [1], request to uwsgi with header
larger than the default value 4096 will end up
with below error as #426:
  invalid request block size: 4123 (max 4096)...skip
  hr_instance_read(): Connection reset by peer
  [plugins/http/http.c line 647]

This commit fixes it by changing the maximum
buffer size as 4096

Also: Similar issue was already identifed and
fixed in [2] on tutor-ecommerce

[1] https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
[2] 6df2c99362

Close #426
2021-05-01 16:32:19 +02:00
Régis Behmo
dfd3d179e9 v11.2.8 (2021-04-27)
- [Bugfix] Fix parsing of YAML-formatted values in ``config save --set
  KEY=VALUE`` commands, in use for instance with Ecommerce.
2021-04-27 09:26:01 +02:00
Eric Herrera
389dd96fdd fix: YamlParamType supports line terminators
This fix allows using a multiple line formatted Yaml string as input for setting a Tutor config value.
2021-04-25 09:14:21 +02:00
Régis Behmo
2e0f136a23 v11.2.7 (2021-04-23)
- [Security] Apply security patch
  [27394](https://github.com/edx/edx-platform/pull/27394).
- [Feature] Add patches to extend python requirements installation process in
  openedx and openedx-dev Dockerfiles.
- [Improvement] Apply edx-platform patches during Docker image build using
  tutor patch 'openedx-dockerfile-git-patches-default'.
2021-04-23 20:24:08 +02:00
Régis Behmo
b211cb0a0f security: apply security patch 27394
See: https://discuss.openedx.org/t/security-patch-for-xss-fixes/4820
2021-04-23 20:23:11 +02:00
Eric Herrera
17088e2fc6 feat: Conditional edx-platform patching
During Docker images build process, apply custom edx-platform patches when tutor patch 'openedx-dockerfile-git-patches-default' is defined or apply current release patches in other case. It avoids possible conflicts between the actually used edx-platform version and the current release patches.
2021-04-15 07:32:39 +02:00
Eric Herrera
a05486e7b0 feat: openedx Dockerfile python requirements extension patch
Add patches to extend python requirements installation process in openedx and openedx-dev Dockerfiles
2021-04-15 07:32:39 +02:00
Régis Behmo
e532f34909 v11.2.6 (2021-04-09)
- [Improvement] Upgrade all services to open-release/koa.3.
- [Feature] Make it possible to build the openedx Docker image with a custom openedx-i18n version with the ``--build-arg OPENEDX_I18N_VERSION=custom`` command line argument.
2021-04-09 10:16:17 +02:00
Régis Behmo
4a65f24ae7 feat: upgrade all services to open-release/koa.3
We remove security patches and custom fixes which are now part of koa.3.

We take the opportunity to make it possible to build the openedx Docker image
without relying on a corresponding openedx-i18n repo tag: often, we want to
test whether the image simply builds successfully, and we don't need up-to-date
translations. For those cases, it's now possible to pass the `-a
OPENEDX_I18N_VERSION=oldertag` build argument.
2021-04-09 10:13:34 +02:00
Régis Behmo
f2ab27199e v11.2.5 (2021-03-30)
- [Bugfix] Fix edge case where `PLUGINS` entry is null in config.yml.
- [Bugfix] Fix missing py2neo dependency in `images build openedx` (#411).
2021-03-30 09:25:39 +02:00
Régis Behmo
c01f4476b8 fix: TypeError when PLUGINS is None
When the PLUGINS config entry is None (`PLUGINS:`), the following error
was being triggered:

  File "/.../tutor/tutor/plugins.py",
  line 304, in is_enabled
      return name in config.get(CONFIG_KEY, [])
      TypeError: argument of type 'NoneType' is not iterable
2021-03-30 09:23:16 +02:00
Régis Behmo
8db1495497 fix: missing py2neo dependency breaks images build openedx
All releases of py2neo prior to 4.0.0 were stripped from Pypi, so we
need to install this dependency from source.

See:
https://github.com/openedx/build-test-release-wg/issues/37
https://github.com/edx/edx-platform/pull/27168

Close #411.
2021-03-30 09:13:44 +02:00
Régis Behmo
98ed40c800 v11.2.4 (2021-03-17)
- [Bugfix] Fix "Invalid Request" error during SAML authentication
(thanks @Ant1x!).
- [Feature] Add `images build --pull` option to update base images.
- [Improvement] Annotate types all over the Tutor code base.
- [Bugfix] Fix parsing of YAML CLI arguments that include equal "="
signs.
- [Bugfix] Fix minor edge case in `long_to_base64` utility function.
- [Improvement] Add openedx patches to add settings during build
process.
2021-03-17 18:21:16 +01:00
Régis Behmo
f688a279dc Actually remove the images build --pull option
This option is mostly useless to us in CI, as it attempts (and fails) to
update the base image when building the dev image. For good reason: the
base image of the dev image is the latest openedx, which has not been
pushed to docker.io at the time of building -_-
2021-03-17 18:21:16 +01:00
Régis Behmo
b00bae9e72 minor: add missing changelog entry 2021-03-17 17:33:06 +01:00
Régis Behmo
914dbc53be Add images build --pull option to pull base images on build
This will be useful in CI, where we want to automatically update base
images as soon as a security fix comes out.
2021-03-17 16:02:46 +01:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Eric Herrera
6ca863e04c Add patches that enable adding settings during build process (collectsatic and i18ncompile). 2021-03-08 13:09:53 +01:00
Régis Behmo
070b75caa4 fix: parsing of YAML config params with "=" signs
Close #405.
2021-03-08 12:24:40 +01:00
Régis Behmo
de872282b2 fix: minor edge case in long_to_base64 utility function.
This was detected by mypy.
2021-02-25 09:54:46 +01:00
Régis Behmo
59e368a449 v11.2.3 (2021-02-20)
- [Bugfix] Make LMS celery workers actually process LMS tasks, and not
  CMS tasks.
2021-02-20 12:50:49 +01:00
Régis Behmo
89dae24b83 minor changelog formatting
Sentences end with a dot ".".
2021-02-20 12:49:30 +01:00
Régis Behmo
39924121be fix: make LMS process LMS tasks, and not CMS tasks
lms-worker was configured to run CMS tasks instead of LMS tasks. I'm not
sure what tasks were being dismissed, and what is the actual production
impact.
2021-02-20 12:45:17 +01:00
Régis Behmo
6c72877256 v11.2.2 (2021-02-17)
- [Security] Apply security patch [26592](https://github.com/edx/edx-platform/pull/26592)
2021-02-17 20:01:53 +01:00
Régis Behmo
8ad45e6157 Apply security patch 26592
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-9/4297
2021-02-17 20:00:44 +01:00
Régis Behmo
8932ea0541 v11.2.1 (2021-02-10)
- [Bugfix] Actually persist Redis data
2021-02-16 12:40:04 +01:00
Régis Behmo
28afa890bf fix: actually persist redis data
Redis data was not actually persisted, because the redis configuration file was
not mounted from the right location. In order to mount redis data in a
host-mounted directory, the working directory has to be properly set.

The problem was occurring both with docker-compose and Kubernetes.

Close #404.
2021-02-16 12:26:54 +01:00
Régis Behmo
731946cce6 v11.2.0 (2021-02-10)
- [Bugfix] Upgrade all services to open-release/koa.2
2021-02-10 00:18:00 +01:00
Régis Behmo
02f9d8db44 Upgrade to open-release/koa.2 2021-02-10 00:15:54 +01:00
Régis Behmo
1d5886baee v11.1.5 (2021-02-009)
- [Security] Apply security patch [26432](https://github.com/edx/edx-platform/pull/26432).
- [Bugfix] Print warnings to stderr
2021-02-09 09:28:39 +01:00
Régis Behmo
9c559cd30b Apply security patch 26432
See: https://github.com/edx/edx-platform/pull/26432/commits
2021-02-09 09:26:43 +01:00
Régis Behmo
73dfb5a616 Print warnings to stderr
When tutor is run as root, the `tutor config printroot` and `tutor
config printvalue` commands used to write to stdout, which was causing
issues when combined with `tutor config render` and `tutor local
settheme` commands.

See discussion:
https://discuss.overhang.io/t/error-while-install-indigo-theme/1056
2021-02-05 15:44:19 +01:00
Régis Behmo
6bed4f6d4a v11.1.4 (2021-02-04)
- [Security] Apply security patch
  [26358](https://github.com/edx/edx-platform/pull/26358).
2021-02-04 17:02:40 +01:00
Régis Behmo
dca4eb6a32 Apply security patch 26358 2021-02-04 08:55:26 +01:00
Régis Behmo
0f5feeb4eb v11.1.3 (2021-01-31)
- [Security] Apply security patch
  [26253](https://github.com/edx/edx-platform/pull/26253).
2021-01-31 10:58:46 +01:00
Régis Behmo
ed66ccdcf4 Apply security patch #26029
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-6/4162
2021-01-31 10:57:53 +01:00
Régis Behmo
56be99f854 v11.1.2 (2021-01-29)
- [Bugfix] Fix django setting value and static asset loading in openedx-dev
  image
2021-01-29 08:44:22 +01:00
Adolfo R. Brandes
d4b067483d Fix default settings on development environment
First, allow using custom Django settings on a development
environment (as documented but not implemented), setting it to the
correct value of `tutor.development`.  Prior to this, `tutor dev
runserver lms` would default to `tutor.production` when on a custom edX
branch.

Second, fix the documentation so the correct environment variable is
described, at the same time removing an option that doesn't seem to work.

See discussion: https://discuss.overhang.io/t/koa-dev-lms-doesnt-find-static-content/1250
2021-01-29 08:43:46 +01:00
Régis Behmo
6fa1db1e4f v11.1.1 (2021-01-20)
- [Feature] Add a `tutor images build --target=...` argument for [multi-stage
  Docker
builds](https://docs.docker.com/develop/develop-images/multistage-build/).
- [Feature] Create a test version of the openedx-dev Docker image for running
  edx-platform unit tests.
- [Security] Apply security patch
  [26112](https://github.com/edx/edx-platform/pull/26112)
- [Bugfix] Fix `local exec` command which crashed with a `AttributeError`.
2021-01-20 14:38:08 +01:00
Régis Behmo
8d803fb08b Get edx-platform unit tests to run
We manage to get unit tests to run in a dedicated openedx-test container. Only
35 tests are failing (out of 17k). I suspect these tests are also failing in
the devstack.
2021-01-20 14:37:10 +01:00
Régis Behmo
3ab800f2e2 Apply security patch 26112 2021-01-20 14:35:47 +01:00
Régis Behmo
df3d432518 Fix AttributeError on "local exec"
`local exec` was failing with the following stacktrace:

    Traceback (most recent call last):
      File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
        load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 38, in main
        cli()  # pylint: disable=no-value-for-parameter
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)                                                     File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)                                                                   File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 237, in ex
    ecute                                                                                                    dc_command.callback("exec", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 271, in dc_command
        volumes, non_volume_args = bindmounts.parse_volumes(args)                                          File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/bindmounts.py", line 73, in parse_vol
    umes
        context = custom_docker_compose.make_context("custom", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 700, in make_context    self.parse_args(ctx, args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1045, in parse_args
        opts, args, param_order = parser.parse_args(args=args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 269, in parse_args
        self._process_args_for_options(state)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 289, in _process_a
    rgs_for_options
        arg = state.rargs.pop(0)
    AttributeError: 'tuple' object has no attribute 'pop'
2021-01-19 11:51:37 +01:00
Régis Behmo
b648e413e6 v11.1.0 (2021-01-13)
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
  runserver` commands on Mac OS. (again, see #392)
- [Feature] Automatically bind-mount volumes from the `volumes/` directory with
  the `--volume=/...` syntax.
2021-01-13 12:32:48 +01:00
Régis Behmo
b09d2ea1ed Fix "Read-only file system" error in dev (again)
Looks like the `-perm /u=w` argument incorrectly includes files from the
read-only filesystem. This should be fixed by the `-writable` option.

Close #392
2021-01-13 12:27:34 +01:00