6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-14 06:58:21 +00:00
Commit Graph

622 Commits

Author SHA1 Message Date
Régis Behmo
89b4ce1202 security: fix enrollment API vulnerability 2022-02-01 17:40:53 +01:00
Régis Behmo
123786193e fix: object has no attribute 'get_metadata' error in learning MFE
See: https://github.com/openedx/edx-platform/pull/29741
2022-02-01 17:40:53 +01:00
Régis Behmo
9a4ea986e1 feat: migrate from edx to openedx GitHub org
edX has completed the migration of all repos from the "edx" to the "openedx"
organization. As a consequence, we change all the links in the repo.
2022-02-01 17:40:53 +01:00
Régis Behmo
bc87f78866 fix: k8s start caddy command
Caddy was not properly starting because its associated configmap was not
starting.

Close #577.
2022-01-31 08:37:35 +01:00
Régis Behmo
5958af86e3 v13.1.2 (2022-01-30)
- [Bugfix] Fix auto-renewal of certificates revoked by Let's Encrypt (see [discussion](https://community.letsencrypt.org/t/questions-about-renewing-before-tls-alpn-01-revocations/170449/21)).
2022-01-30 21:36:25 +01:00
Régis Behmo
7c3e54335d fix: let's encrypt auto-revoked certificates
Let's Encrypt started mass-revoking certificates. For some reason, we
did not upgrade to the latest version of Caddy in the Maple upgrade, so
we are affected by this issue. Users should upgrade now.

See:
https://discuss.overhang.io/t/lets-encrypt-revoked-certificate-did-anyone-experienced-this/2391
https://discuss.overhang.io/t/ssl-error-in-the-browser/2390
https://community.letsencrypt.org/t/questions-about-renewing-before-tls-alpn-01-revocations/170449/21
2022-01-30 21:33:31 +01:00
Régis Behmo
bbba7b6c4e v13.1.1 (2022-01-25)
- [Bugfix] Fix authentication in development due to missing SameSite policy on session ID cookie.
- [Bugfix] Display properly themed favicon.ico image in LMS, Studio and microfrontends.
- [Bugfix] Fix "LazyStaticAbsoluteUrl is not JSON serializable" error when sending bulk emails.
- [Bugfix] Fix `tutor local importdemocourse` fails when platform is not up.
2022-01-25 08:38:14 +01:00
Régis Behmo
26119983d4 fix: lms authenticaton in development
In development, it was no longer possible to authenticate to the lms. Ater
signing in, the session ID could not be dropped, and thus the user was not
signed in, although no error was logged -- just a warning in the browser
console.
This problem was caused by the fact that the SameSite policy was set to "None"
in development.
2022-01-25 08:36:52 +01:00
Régis Behmo
1172ab6f1f fix: display themed favicon across all apps
Previously, we were redirecting all /*favicon.ico requests to the default
favicon. This meant that the favicon might not necessarily be correctly themed,
most notably in MFEs. Here, we resolve this issue by redirecting to the
theme-agnostic theming/asset/* url. Also, we restrict the overly generic regexp
for favicon url matching. We verified that we did not miss any url by running
the following command on the demo server:

    tutor local logs caddy | grep --only-matching "host.*favicon.ico" | sort | uniq
2022-01-25 08:30:06 +01:00
Régis Behmo
405aaac189 fix: broken bulk emails because of LazyStaticAbsoluteUrl (again)
The LazyStaticAbsoluteUrl object was breaking bulk emails again with the
following stacktrace:

    2022-01-11 13:50:10,591 ERROR 12 [celery.app.trace] [user None] [ip None] trace.py:255 - Task lms.djangoapps.instructor_task.tasks.send_bulk_course_email[26b93357-018a-408f-b3f7-b69722447c5b] raised unexpected: EncodeError(TypeError('Object of type LazyStaticAbsoluteUrl is not JSON serializable'))
    Traceback (most recent call last):
      File "/openedx/venv/lib/python3.8/site-packages/kombu/serialization.py", line 50, in _reraise_errors
	yield
      File "/openedx/venv/lib/python3.8/site-packages/kombu/serialization.py", line 221, in dumps
	payload = encoder(data)
      File "/openedx/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 69, in dumps
	return _dumps(s, cls=cls or _default_encoder,
      File "/openedx/venv/lib/python3.8/site-packages/simplejson/__init__.py", line 398, in dumps
	return cls(
      File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 296, in encode
	chunks = self.iterencode(o, _one_shot=True)
      File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 378, in iterencode
	return _iterencode(o, 0)
      File "/openedx/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 59, in default
	return super(JSONEncoder, self).default(o)
      File "/openedx/venv/lib/python3.8/site-packages/simplejson/encoder.py", line 272, in default
	raise TypeError('Object of type %s is not JSON serializable' %
    TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable

The point of that lazy object was to link to the lms logo even when a custom
theme was enabled. Luckily, we no longer need this lazy evaluation because we
now have theme-agnostic urls that point to static asset (see
https://github.com/openedx/edx-platform/pull/29461).

See:
https://discuss.overhang.io/t/error-while-sending-bulk-emails-lazystaticabsoluteurl-is-not-json-serializable/2176/
2022-01-25 08:27:36 +01:00
iamcristye@outlook.com
d953e99ae2 fix: tutor local importdemocourse fails when platform is not up 2022-01-20 11:51:24 +01:00
Régis Behmo
d45b36394a - [Improvement] Provide much more comprehensive instructions when
upgrading.
- [Bugfix] During upgrade, make sure that environment is up-to-date
prior to prompting to rebuild the custom images.
- [Bugfix] Fix ownership of mysql data, in particular when upgrading a
Kubernetes cluster to Maple.
- [Bugfix] Ensure that ``tutor k8s upgrade`` is run during ``tutor k8s
quickstart``, when necessary.
- 💥[Bugfix] By default, detect the current version during ``tutor
k8s/local upgrade``.
- [Bugfix] Fix upgrading from Lilac to Maple on Kubernetes by deleting
deployments and services.
2022-01-08 19:09:18 +01:00
Régis Behmo
4dc772d1e4 fix: attempt to make upgrade much clearer
`upgrade` had several issues, which are summarized here:
https://discuss.overhang.io/t/confusing-instructions-during-upgrade/2281/7

- The docs say that you should run quickstart, but what most people will see is
the big command tutor local upgrade --from=lilac verbatim paragraph.
- The local upgrade command should be very explicit about the fact that users
need to run quickstart.
- Maybe the name of the local upgrade command should be improved.
- When upgrading tutor from one major release to the next, there should be a
more explicit warning to inform users of what they are doing (see this other
conversation 1)
- We should tell people that they almost certainly need to enable the tutor and
the mfe plugins, if they are not enabled during upgrade.
- A link to all of the breaking changes from the changelog should be
prominently displayed during upgrade.
- The docs should emphasize that upgrading from one major release to the next
is potentially a risky endeavor and that downgrading is not possible. The docs
should also link to the changelog.

This commit has grown slightly beyond the intended scope, but the changes should be mostly positive.
2022-01-08 19:07:26 +01:00
Régis Behmo
9fc928a711 fix: update env prior to rebuilding images during upgrade
The fact that the environment was not up-to-date was causing errors and
confusion:
https://discuss.overhang.io/t/install-maple-importerror-cannot-import-name-removedindjango40warning/2255/6
2022-01-08 19:07:26 +01:00
Régis Behmo
59291eed52 fix: ownership of mysql data
In theory, we can assign ownership of mysql data to just any user. But in
Lilac, mysql was running with user 999. When upgrading to Maple, on Kubernetes,
the fsGroupChangePolicy was causing a change of the data *group* (to 1000) but
not of the user. This was causing a crash with the following error:

    [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
2022-01-08 19:07:26 +01:00
Régis Behmo
70dbfcb6fb fix: upgrade from Lilac on k8s
When upgrading from Lilac, all services break with the following error:

    Service "***" is invalid: spec.ports[0].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'

Upgrading deployments fails as well:

    Deployment.apps "***" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"openedx-********", "app.kubernetes.io/managed-by":"tutor", "app.kubernetes.io/name":"***", "app.kubernetes.io/part-of":"openedx"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

That's because deployments and services need to be deleted as part of the Maple
upgrade. So that's what we do as part of `tutor k8s upgrade --from=lilac`. And
we take the opportunity to:

1. Run upgrade as part of quickstart, when necessary.
2. Default to lilac during `tutor k8s upgrade`.

Close #551.
2022-01-08 19:07:26 +01:00
Régis Behmo
f4d866db81 v13.0.3 (2022-01-04)
- [Security] Upgrade Django to 3.2.11 in edx-platform.
- [Security] Prevent non-staff users from searching usernames by email by
  abusing the logout url.
2022-01-04 17:33:37 +01:00
Régis Behmo
699ef4c6d5 security: upgrade django to 3.2.11 in edx-platform
See release notes: https://docs.djangoproject.com/en/3.2/releases/3.2.11/
2022-01-04 16:38:23 +01:00
Régis Behmo
622e7cc097 security: prevent searching user emails by name (again)
This time, the security fix addresses an issue in the logout url.
2022-01-04 15:43:30 +01:00
Régis Behmo
94941d0387 v13.0.2
- [Security] Prevent non-staff users from searching usernames by email.
2021-12-22 00:46:13 +01:00
Régis Behmo
553000e5f5 security: fix search username by email
Without this patch, it is possible to search for account info including
username by using the email of a learner. This fix disallows searching using
email by regular users and restricts this feature to only staff and superusers.
2021-12-22 00:45:26 +01:00
Régis Behmo
4bfaa80bd3 v13.0.1
- [Fix] Missing requirements file in `pip install tutor[full]`.
2021-12-20 23:00:12 +01:00
Régis Behmo
14c40376ec fix: missing file in pip install tutor[full]
The missing requirement file is causing the install from pypi to fail.
2021-12-20 22:59:21 +01:00
Régis Behmo
7c157eccd5 feat: upgrade to Maple
- A shared cookie domain between lms and cms is no longer recommended:
https://github.com/edx/edx-platform/blob/master/docs/guides/studio_oauth.rst
- refactor: clean mounted data folder in lms/cms. In Lilac, the
bind-mounted lms/data and cms/data folders are a mess because new
folders are created there for every new course organisation.  These
folders are empty. As far as we know they are useless... With this
change we move these folders to a dedicated "modulestore" subdirectory;
which corresponds better to the initial intent of the fs_root setting.
- fix: frontend failure during login to the lms. See:
https://github.com/openedx/build-test-release-wg/issues/104
- feat: move all forum-related code to a dedicated plugin. Forum is an
optional feature, and as such it deserves its own plugin. Starting from
Maple, users will be able to install the forum from
https://github.com/overhangio/tutor-forum/
- migrate from DCS_* session cookie settings to SESSION_*. That's
because edx-platform no longer depends on django-cookies-samesite. Close
https://github.com/openedx/build-test-release-wg/issues/110
- get rid of tons of deprecation warnings in the lms/cms
- feat: make it possible to point to themed assets. Cherry-picking this
change makes it possible to point to themed assets with a theme-agnostic
url, notably from MFEs.
- Install all official plugins as part of the `tutor[full]` package.
- Don't print error messages about loading plugins during autocompletion.
- Prompt for image building when upgrading from one release to the next.
- Add `tutor local start --skip-build` option to skip building Docker images.

Close #450.
Close #545.
2021-12-20 21:21:36 +01:00
Braden MacDonald
1f46ed39c5 fix: openedx image wasn't building on ARM64 due to missing libgeos-dev 2021-12-14 12:10:49 +01:00
Régis Behmo
e89ff49223 v12.2.0
- 💥[Improvement] Fail on incorrect image name argument in `images
build/pull/push/printtag` commands.
- [Bugfix] Remove trailing slashes in docker-compose files for
[compatibility with docker-compose v2 in
WSL](https://github.com/docker/compose/issues/8558).
- [Improvement] `settheme` now works with preview domain.
- [Feature] Allow specifying extra pip packages through config.yml.
2021-12-08 11:29:52 +01:00
Régis Behmo
79c6d122e2 fix: incorrect "from" address in course bulk emails
Upstream PR: https://github.com/edx/edx-platform/pull/29001
See discussion:
https://discuss.overhang.io/t/sending-bulk-email-triggers-smtprecipientsrefused-error/1923
Related issue:
https://github.com/openedx/build-test-release-wg/issues/102
2021-12-08 11:29:52 +01:00
Régis Behmo
c3e6f57b08 feat: fail images ... [image] commands on incorrect image arg
Previously, `tutor images build doesnotexist` was a no-op. This was
counter-intuitive -- in particular, when users have a typo in their command.

Now, the `images build/pull/push/printtag` commands fail if the corresponding
image was not found.

See discussion: https://discuss.overhang.io/t/return-something-when-tutor-images-build-contains-a-typo/2174
2021-12-02 09:34:16 +01:00
Maxim Beder
19157d94bc feat: allow to specify extra pip packages in config
Added OPENEDX_EXTRA_PIP_REQUIREMENTS setting, which allows to specify
extra pip packages that should be installed.

Moved "openedx-scorm-xblock" package from Dockerfile to the new setting
in the config.yml.
2021-11-30 11:32:40 +01:00
iamcristye@outlook.com
5604cd4bf8 Improvement: settheme now works with preview domain 2021-11-29 18:21:16 +01:00
Sofiane Bebert
852759ae35 fix: remove trailing slashes in volume paths for docker-compose v2 compatibility
close #522
2021-11-22 14:10:28 +01:00
Régis Behmo
a4dcb9eaf2 v12.1.7 (2021-11-18)
- [Security] Timed exam security fix [29347](https://github.com/edx/edx-platform/pull/29347).
- [Feature] Add [tutor-richie](https://github.com/overhangio/tutor-richie) to the plugins that are bundled with the tutor binary.
- [Improvement] Make `tutor plugins list` print plugins sorted by name.
- [Improvement] Ignore Python plugins which cannot be loaded.
- [Bugfix] When configured with `RUN_FORUM: false`, omit forum-related [Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/) from the manifests that `tutor k8s` generates. (#525)
2021-11-18 11:04:15 +01:00
Régis Behmo
aa931b33f4 security: applied timed exam fix 2021-11-18 10:55:50 +01:00
Florian Haas
0c3a160fce fix: Include the forum job only if RUN_FORUM is set
When running "tutor k8s" we can disable the forum deployment and
service by setting RUN_FORUM to false, but the corresponding job is
added to jobs.yml unconditionally.

Add a conditional to tutor/templates/k8s/jobs.yml so that the forum
job definition is only included when RUN_FORUM is true.

Fixes #525.
2021-11-18 10:22:54 +01:00
Régis Behmo
0a4a2bbac2 feat: add richie to bundled plugins
For more information, checkout the plugin docs:
https://github.com/overhangio/tutor-richie
2021-11-09 11:48:32 +01:00
Régis Behmo
485f47f6d0 fix: always print plugins sorted by name
`tutor plugins list` used to print plugins in random oredr. To be honest
this has always bothered me.
2021-11-08 10:40:37 +01:00
Régis Behmo
d9d08ad0f7 improvement: don't crash on unloadable plugins
In some cases, plugins declare an entry point but cannot be loaded.
This is the case when they depend on a version of tutor that is not the
one that is currently installed. This use case is very frequent when
working on multiple versions at the same time (i.e: right now, while we
are working on the Maple release). In such cases, it's best just to
ignore the plugin entirely rather than having to re-install all plugins
in the virtualenv.
2021-11-08 10:40:37 +01:00
Régis Behmo
0a2abe32dc v12.1.6 (2021-11-02)
- [Improvement] Upgrade all services to open-release/lilac.3.
- [Feature] Make it possible to override job configuration in
development: if they exist, `dev/docker-compose.jobs.yml` and
`dev/docker-compose.jobs.override.yml` will be loaded when running jobs.
- [Improvement] Faster `tutor local start` by building only necessary
images.
2021-11-02 15:55:10 +01:00
Régis Behmo
c9a08a5e18 chore: upgrade to open-release/lilac.3 2021-11-02 11:56:24 +01:00
Régis Behmo
d73d6732d5 feat: make it possible to override jobs in dev
Previously, job declarations were always loaded from local/docker-compose.yml
and local/docker-compose.jobs.yml. This meant that it was not possible to
override job declarations in dev mode. It is now the case, with
dev/docker-compose.jobs.yml and dev/docker-compose.jobs.override.yml. Neither
of these files exist yet... But who knows? we might need this feature one day.
In any case the code is much cleaner now.
2021-11-01 17:21:43 +01:00
Régis Behmo
079fb1c9ec fix: bypass build to accelerate "local start"
Previously, we were building all images every time we ran a "local start"
command. This was causing unnecessary rebuild. Here, instead, we make use of
the `docker-compose up --build`. This means that only the required images will
be rebuilt.
2021-11-01 17:00:11 +01:00
Peter Parkkali
2549aef4dc fix: require at least 4 GB RAM on macOS for 'local quickstart' only
Limits the memory chek to the 'local quickstart' command, makes error
handling more accurate and adds warning messages for some conditions.
Also adds a mention of this in troubleshooting.rst.
2021-10-25 20:17:38 +02:00
Peter Parkkali
fb2aeefd91 fix: require at least 4 GB RAM on macOS for local commands
Adds a check in the 'local' command group that requires at least
4 GB of RAM to be allocated to Docker when running any of the
local subcommands on macOS. This addresses a common issue where
Docker's default setting (2 GB) causes startup to crash with
misleading error messages.
2021-10-25 20:17:38 +02:00
Régis Behmo
01d374d2b1 v12.1.5 (2021-10-25)
- 💥[Improvement] Change the `settheme` command such that, by default, a custom theme is assigned to the LMS and the CMS, both in production and development mode.
2021-10-25 17:54:40 +02:00
Régis Behmo
df6a1c3b4e 💥 improvement: better defaults for the settheme commands
Previously, the list of domain names to which a theme was assigned had to be
specified manually. Now, the themes are automatically assigned to the LMS and
the CMS, both in development and production modes.
2021-10-14 12:59:57 +02:00
Régis Behmo
5a4e3792f4 v12.1.4 (2021-10-11)
- [Feature] Add configuration setting `PREVIEW_LMS_BASE` for custom
preview domain.
- [Improvement] Enable milestones application flag `MILESTONES_APP` and
prerequisite courses feature flags `ENABLE_PREREQUISITE_COURSES` by
default.
2021-10-11 12:14:22 +02:00
Kevin Valencia
72d01a05dc improvement: enable prerequisites course feature by default
Set MILESTONES_APP and ENABLE_PREREQUISITE_COURSES to true as default.

See: https://discuss.overhang.io/t/course-prerequisites/162
2021-10-07 16:11:02 +02:00
Crist
e3788257db Feature: configuration setting PREVIEW_LMS_BASE
discussion: https://discuss.overhang.io/t/new-settings-variable/1973/4
2021-10-04 14:51:14 +02:00
Shimul Chowdhury
791c0161df feat: ability to pass docker build options via env variable TUTOR_DOCKER_BUILD_ARGS
feat: use --docker-arg flag instead of env variable to pass docker build options

chore: remove unused import

chore: added changelog

feat: added shorthand -d for --docker-arg

fix: typo
2021-09-30 12:42:27 +02:00
Régis Behmo
926263deb3 v12.1.3 (2021-09-28)
- [Bugfix] Fix 500 error during user registration.
- [Bugfix] Fix Mongodb compatibility version upgrade when upgrading from Koa to Lilac.
2021-09-28 12:42:12 +02:00
Régis Behmo
530b26a5ff fix: 500 error during user registration
See: https://discuss.overhang.io/t/no-activation-email-errors-logged-on-user-sign-up/1969

A 500 error was being triggered during user registration.

    Traceback (most recent call last):
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
        response = get_response(request)
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
        response = self.process_exception_by_middleware(e, request)
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
        response = wrapped_callback(request, *callback_args, **callback_kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        return view_func(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
        return self.dispatch(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
        return view(request, *args, **kwargs)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 485, in dispatch
        return super().dispatch(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
        response = self.handle_exception(exc)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
        self.raise_uncaught_exception(exc)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
        raise exc
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
        response = handler(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        return view_func(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/ratelimit/decorators.py", line 24, in _wrapped
        return fn(request, *args, **kw)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 529, in post
        response, user = self._create_account(request, data)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 572, in _create_account
        user = create_account_with_params(request, data)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 236, in create_account_with_params
        compose_and_send_activation_email(user, profile, registration)
      File "/openedx/edx-platform/common/djangoapps/student/views/management.py", line 214, in compose_and_send_activation_email
        send_activation_email.delay(str(msg))
      File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 29, in __str__
        return json.dumps(self, cls=MessageEncoder)
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/__init__.py", line 234, in dumps
        return cls(
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 199, in encode
        chunks = self.iterencode(o, _one_shot=True)
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 257, in iterencode
        return _iterencode(o, 0)
      File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 119, in default
        return super().default(o)   # pragma: no cover
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 179, in default
        raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable
    2021-09-28 05:21:52,174 ERROR 122 [django.request] [user 11] [ip XY.XY.XY.XY] log.py:222 - Internal Server Error: /api/user/v2/account/registration/
    Traceback (most recent call last):
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
        response = get_response(request)
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
        response = self.process_exception_by_middleware(e, request)
      File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
        response = wrapped_callback(request, *callback_args, **callback_kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        return view_func(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
        return self.dispatch(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
        return view(request, *args, **kwargs)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 485, in dispatch
        return super().dispatch(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
        response = self.handle_exception(exc)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
        self.raise_uncaught_exception(exc)
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
        raise exc
      File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
        response = handler(request, *args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        return view_func(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 45, in _wrapper
        return bound_method(*args, **kwargs)
      File "/openedx/venv/lib/python3.8/site-packages/ratelimit/decorators.py", line 24, in _wrapped
        return fn(request, *args, **kw)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 529, in post
        response, user = self._create_account(request, data)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 572, in _create_account
        user = create_account_with_params(request, data)
      File "./openedx/core/djangoapps/user_authn/views/register.py", line 236, in create_account_with_params
        compose_and_send_activation_email(user, profile, registration)
      File "/openedx/edx-platform/common/djangoapps/student/views/management.py", line 214, in compose_and_send_activation_email
        send_activation_email.delay(str(msg))
      File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 29, in __str__
        return json.dumps(self, cls=MessageEncoder)
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/__init__.py", line 234, in dumps
        return cls(
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 199, in encode
        chunks = self.iterencode(o, _one_shot=True)
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 257, in iterencode
        return _iterencode(o, 0)
      File "/openedx/venv/lib/python3.8/site-packages/edx_ace/serialization.py", line 119, in default
        return super().default(o)   # pragma: no cover
      File "/opt/pyenv/versions/3.8.6/lib/python3.8/json/encoder.py", line 179, in default
        raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type LazyStaticAbsoluteUrl is not JSON serializable

The reason for that was that edx-ace could not json-serialize the context to be
passed to the registration email renderer. That was caused by the
LazyStaticAbsoluteUrl object created to address missing logo in registration
email. To make sure that this object is serializable by
edx_ace.serialization.MessageEncoder, we add a trivial .to_json() method to the
LazyStaticAbsoluteUrl class.

This error could (at first) not be reproduced in development, because
AUTOMATIC_AUTH_FOR_TESTING is set to true in devstack settings.
2021-09-28 12:41:06 +02:00
Régis Behmo
72cf5fe30e fix: mongodb upgrade during koa -> lilac migration
When upgrading mongodb, the mongodb container takes a little while to become
ready. Running the "exec" command thus triggers an error:

    docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setF
    eatureCompatibilityVersion: "4.0" })
    MongoDB shell version v4.0.24
    connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
    2021-06-14T10:53:21.510+0000 E QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused:
    connect@src/mongo/shell/mongo.js:356:17
    @(connect):2:6
    exception: connect failed
    Error: Command failed with status 1: docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setFeatureCompatibilityVersion: "4.0" })

We add a "sleep" statement to the upgrade process to ensure that the mongodb
container is available.
2021-09-21 09:18:10 +02:00
Régis Behmo
9754719277 v12.1.2 2021-09-18 10:27:03 +02:00
Régis Behmo
bf5d2b80d8 fix: forum connection to mongodb (again)
Turns out, the authentication mechanism should only be defined if there
is an actual authentication. For now, because of the urgency, we
hardcode this auth_mech to ":scram". We'll add a way to override it if
necessary, in the future.
2021-09-18 10:19:02 +02:00
Régis Behmo
9263add948 v12.1.1 (2021-09-17)
- [Bugfix] Fix forum starting issue: "NoMethodError: undefined method
`encode' for nil:NilClass".
2021-09-17 16:52:34 +02:00
Régis Behmo
22d42cea45 fix: Revert "Remove MONGOID_AUTH_MECH setting from forum Dockerfile."
This reverts commit 750bdca04d.

The MONGOID_AUTH_MECH environment variable is actually necessary and
required by the ruby gem. Thus, we should not remove it, but keep it to
its default value.

See: https://github.com/overhangio/tutor/pull/493#issuecomment-921758422
https://discuss.overhang.io/t/edx-forum-mongodb-authentication/927/18
2021-09-17 16:51:24 +02:00
Régis Behmo
922c8bb373 v12.1.0 (2021-09-17)
- [Improvement] Make it easier to run edx-platform unit tests.
- [Bugfix] Fix segmentation fault during `tutor config save` on Mac OS M1 (#473). Thanks @ghassanmas!
- [Bugfix] Fix a bug that prevented connecting to external MongoDB instances.
- [Improvement] Make sure that the logo included in email notifications (including discussion responses) is the same as the site logo.
- [Bugfix] Install IPython directly from pypi instead of installing it from source (the reason it was installed from source is no longer relevant). The effect of this shall speed up the process of building the openedx-dev Docker image.
- [Feature] Add "openedx-dockerfile-post-git-checkout" patch.
- [Improvement] In the "openedx" Docker images, convert git patches to cherry-picks for a cleaner source tree.
- 💥[Feature] Make it possible to override local job configuration. This deprecates the older model for running jobs which dates back from a long time ago.
2021-09-17 10:52:15 +02:00
Régis Behmo
010279a0d9 feat: make it easier to run edx-platform unit tests
It should be unnecessary to build a custom openedx-dev Docker image. All tests
can run from within the dev Docker image, with a couple additional environment
variables.
2021-09-17 10:48:00 +02:00
Régis Behmo
b651ebc3c4 fix: segmentation fault on Mac OS M1 (Big Sur)
On Mac OS M1 (ARM processor), when the GMP library is available, pycryptodome
crashes with a segmentation fault during the generation of the RSA keys.

Upstream issue: https://github.com/Legrandin/pycryptodome/issues/506
Upstream fix: https://github.com/Legrandin/pycryptodome/pull/541

Close #473.
2021-09-17 10:44:52 +02:00
Régis Behmo
eef3c15c17 improvement: correctly set email logo
Previously, the logo included in emails was loaded from edX' CDN. Here, we make
sure that the logo is actually the same as the site logo. Because the logo may
be theme-specific, we need to compute the logo url at runtime, and use a
lazily-evaluated string.

Close #447.
2021-09-17 10:39:34 +02:00
Andrés González
750bdca04d Remove MONGOID_AUTH_MECH setting from forum Dockerfile. 2021-09-16 22:06:09 +02:00
Ghassan Maslamani
b1e91e3626 BugFix: fixes slow installation close #488
Change the source of installation Ipyton, install it from PyPy instead of source, which shall speed up the process of building the docker image for openedx-dev
2021-09-16 08:16:18 +02:00
Régis Behmo
6d18ccc632 feat: add "openedx-dockerfile-post-git-checkout" patch
This will be convenient for plugins which need to patch edx-platform.
2021-09-14 17:49:02 +02:00
Régis Behmo
ca3aabedce feat: cleaner git tree in openedx Docker image
With "git patch", the resulting source tree was dirty, showing uncommitted
changes. Here, we replace "git patch" with "git cherry-pick". We avoid pulling
the entire remote repo by fetching individual commits. To do that, we need to
assign an identity to the git user.
2021-09-14 17:49:02 +02:00
Régis Behmo
d3bfc4fb87 feat: make it possible to override local docker-compose job
Previously, it was possible to override docker-compose services, but not jobs. This requirement has appeared because some people need to override project-wide MTU settings.

See: https://discuss.overhang.io/t/problem-fetching-saml-idp-metadata/1330/23
2021-09-09 09:03:37 +02:00
Régis Behmo
ecf26fccda v12.0.4 (2021-08-12)
- [Security] Apply security patch [28442](https://github.com/edx/edx-platform/pull/28442).
2021-08-12 16:47:57 +02:00
Régis Behmo
c33bd7c892 Apply security patch #28442
See: https://discuss.openedx.org/t/security-patch-for-xxe-vulnerability-in-course-import/5575
2021-08-12 16:47:12 +02:00
Régis Behmo
eeb08f63d9 v12.0.3 (2021-08-10)
- [Improvement] Upgrade all services to open-release/lilac.2.
- [Bugfix] Fix "`sh` is not a recognized command" in some plugins, including minio.
- [Improvement] Set the default contact mailing email address
- [Bugfix] Fix minio initialisation in Kubernetes.
2021-08-10 12:30:06 +02:00
Régis Behmo
41b65b80d5 Upgrade to open-release/lilac.2 2021-08-10 12:26:30 +02:00
Sofiane Bebert
4bf61d6d35 fix: set default CONTACT_MAILING_ADDRESS 2021-08-10 10:46:38 +02:00
Régis Behmo
f1773c0ba9 Revert "fix: empty entrypoints in docker-compose=2.0.0.beta4"
This reverts commit 6f04223d01.

It turns out that we cannot ignore "command: []" statements in k8s manifests.
That's because there is no way to clear entrypoint in k8s manifests, but
sometimes we do need to bypass the entrypoints. For instance, the minio client
container sets "mc" as the default entrypoint, which does not work with our job
logic.

As a consequence, Tutor becomes incompatible with docker-compose 2.0.0.beta4.
The "entrypoint must be a string" error is actually an upstream bug:
https://github.com/docker/compose-cli/issues/1848

See:

The corresponding minio issue: https://github.com/overhangio/tutor-minio/issues/9
The previous conversation about empty entrypoints: https://discuss.overhang.io/t/undefined-entrypoint-throws-error-in-docker-compose-2-0-0-beta-4/1716
2021-08-10 09:16:55 +02:00
Régis Behmo
c678638ea2 v12.0.2 (2021-07-06)
- [Bugfix] Fix "Invalid command argument" during upgrade from Koa to Lilac.
- [Bugfix] Fix mysql initialisation in docker-compose==2.0.0beta4.
- [Improvement] Tutor is now published on pypi as "tutor".
2021-07-06 09:50:11 +02:00
Crist Ye
db5852e9c4 fix: mongodb upgrade error when upgrading from koa
ref: https://discuss.overhang.io/t/mongo-db-error-warning-on-koa-lilac-upgrade/1744

Close #469
2021-07-06 08:42:13 +02:00
Régis Behmo
6f04223d01 fix: empty entrypoints in docker-compose=2.0.0.beta4
An issue with the latest release of docker-compose was reported here:
https://discuss.overhang.io/t/undefined-entrypoint-throws-error-in-docker-compose-2-0-0-beta-4/1716

The mysql-job definition had an empty entrypoint (`[]`). This was causing the following error:

    the initiation of mysql fails with “services.mysql-job.entrypoint must be a string …
    Error: Command failed with status 15”

I can't remember at all why we had to define an empty entrypoint. It probably
has to do with the fact that we could not run `sh -e -c "..."` commands in
mysql jobs. Similarly, the k8s job definition sets `command: []`. I tested both
local and k8s deployments without these definitions and they work just fine. So
I guess we can get rid of them.
2021-07-05 10:01:37 +02:00
Régis Behmo
c15eec53dc feat: switch the package name from "tutor-openedx" to "tutor"
The package maintainer of the "tutor" package was kind enough to
transfer ownership of the project to us. This is great, because we no
longer have to use the "openedx" suffix, which is trademarked.

For the time being, we keep maintaining the "tutor-openedx" package
which has a 1-to-1 dependency on the "tutor" package. In the future, we
expect that we will no longer push upgrades to tutor-openedx.
2021-07-03 11:07:37 +02:00
Régis Behmo
5cc64bf9ee v12.0.1 (2021-06-22)
- [Bugfix] Fix double pulling mongodb image when upgrading from Koa to Lilac.
- [Improvement] Better logging during `plugins disable`.
- [Bugfix] Fix "upstream sent too big header" error during login of existing users after a Koa to Lilac upgrade.
- [Feature] Added the ability to skip `config.yml` file modification while running `tutor config save` command with `-e` or `--env-only` flag.
- [Feature] Add new config value `FORUM_MONGODB_DATABASE` to set the forum database name
2021-06-22 12:47:18 +02:00
ChrisChV
d6af5ef55b feat: FORUM_MONGODB_DATABASE config val added to fix the hardcoded forum database name
In config.yml the new value FORUM_MONGO_DB_DATABASE was added with `cs_comments_service` as default value.
In docker-entrypoint.sh of forum I changed the hardcoded `cs_commecnts_service` with the new config value.
Multiple .yml files changed to handle the new config value.
2021-06-22 12:38:23 +02:00
Régis Behmo
ef189e7f67 fix: better logging during plugins disable
When disable a plugin that set config entried, such as the minio plugin, tutor was logging the following:

    Disabling plugin minio...
        Removed config entry OPENEDX_AWS_ACCESS_KEY=openedx
        Removed config entry OPENEDX_AWS_SECRET_ACCESS_KEY={{ MINIO_AWS_SECRET_ACCESS_KEY }}
        Plugin disabled

The config values were not rendered during printing, which is a shame, because
the whole point of this log line is to warn users of passwords/secrets that are
being removed. Here, we make sure that the config values are properly rendered.
The new logs are now:

    Disabling plugin minio...
        Removing config entry OPENEDX_AWS_ACCESS_KEY=openedx
        Removing config entry OPENEDX_AWS_SECRET_ACCESS_KEY=64vpCVLxhDxBuNjakSrX4CQg
        Plugin disabled
2021-06-22 12:28:58 +02:00
Crist
d49d166383 Fix: double pulling mongodb image when upgrading from Koa to Lilac. ref: https://discuss.overhang.io/t/double-pulling-mongo-images-when-upgrading-from-koa-to-lilac/1625 2021-06-22 12:28:26 +02:00
Shimul Chowdhury
31a41f81c3 feat: env only mode for config generation
chore: added -e flag to the changelog
2021-06-22 12:25:29 +02:00
Régis Behmo
e7d3e722ce fix: "upstream sent too big header" nginx errors
It was reported that existing users could not sign-in after a Koa -> Lilac
upgrade. This is due to the fact that cookies exceed the 4kb limit, which is
rejected by nginx. The fix consists in increasing this limit to 8kb:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size

This issue was reported here: https://discuss.overhang.io/t/lilac-upgrade-login-issues/1617
Note that I was not able to reproduce the issue. The fix was tested by the
issue reporter.
2021-06-22 08:58:50 +02:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
915551268c v11.3.1 (2021-06-08)
- [Improvement] Avoid permission issues in Kubernetes/Openshift for users who do not have the rights to edit their namespace.
- [Improvement] Better Kubernetes object creation.
2021-06-08 20:17:44 +02:00
Régis Behmo
4a451844e6 fix: avoid namespace edition for k8s users without access rights
In most cases, it makes very little sense to edit the namespace that an
application is running in. Quite often, users are granted access to just one
namespace and don't have the necessary rights to edit the namespace -- and for
good security reasons. In such cases, the k8s namespace object already exists
and there is no need for the user to edit or create it. Here, what we do is
that we create the namespace only if it does not exist. This should solve quite
a few permission issues, notably for Openshift users.
2021-06-08 19:54:09 +02:00
Régis Behmo
9b6b770dd7 feat: better object creation in "k8s start"
When running "tutor k8s start", it's unnecessary to attempt to create volumes
and namespace multiple times.
2021-06-08 19:54:09 +02:00
Régis Behmo
900572fdae v11.3.0 (2021-05-18)
- 💥[Security] Disable python-evaluated input by default as we don't run codejail.
- [Bugfix] Fix missing discussion notifications.
- [Improvement] Better error logging when loading a plugin from an incompatible version.
2021-06-06 14:40:24 +02:00
Régis Behmo
a79d194bbc security: disable python-evaluated input by default
As described in issue #284, tutor does not come with codejail enabled out of
the box. Actually, we don't even have a working plugin, yet. To prevent users
from running unsafe code, we explicitely disable python-evaluated input by
disabling the "python" interpreter. This might break some courses; thus, this
is a non-backward compatible change.
2021-06-06 14:37:46 +02:00
liupengcheng
58c77ad3fb fix: discussion email notification not work.
Discussion email notification relies on edx_ace and celery routing
key for ace is not set properly by default, which causes the send
mail task is never consumed.

Close #439.
2021-05-25 10:00:45 +02:00
Régis Behmo
4cc48c966f feat: better logging during plugin loading failure
When upgrading Tutor plugins to the next release, I often end up with a
virtualenv that contains plugins that depend on different versions of
tutor-openedx. This causes a crash that did not log the name of the responsible
package. For instance:

    Traceback (most recent call last):
      File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
        load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 37, in main
        add_plugin_commands(cli)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/plugins.py", line 137, in add_plugin_commands
        for plugin in plugins.iter_installed():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 401, in iter_installed
        yield from Plugins.iter_installed()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 362, in iter_installed
        for plugin in PluginClass.iter_installed():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 199, in iter_installed
        for plugin in cls.iter_load():
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 233, in iter_load
        yield cls(entrypoint)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/plugins.py", line 220, in __init__
        super().__init__(entrypoint.name, entrypoint.load())
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2442, in load
        self.require(*args, **kwargs)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2465, in require
        items = working_set.resolve(reqs, env, installer, extras=self.extras)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
        raise VersionConflict(dist, req).with_context(dependent_req)
    pkg_resources.VersionConflict: (tutor-openedx 11.2.11 (/home/regis/projets/overhang/repos/overhang/tutor), Requirement.parse('tutor-openedx<13.0.0,>=12.0.0'))

In this commit, we introduce an error log that displays the name and location
of the package. E.g:

    Failed to load entrypoint 'minio = tutorminio.plugin' from distribution tutor-minio 12.0.0
2021-05-20 12:22:52 +02:00
Régis Behmo
ed1e2e49fe v11.2.11 (2021-05-18)
- [Feature] Add redis database configuration for both cache and celery.
2021-05-18 10:05:21 +02:00
liupengcheng
028162fea4 feat: Make openedx redis db configurable.
Redis db 0 is used for celery broker and db 1 is used for cache
by default.
2021-05-18 09:35:36 +02:00
Régis Behmo
5658b46715 v11.2.10 (2021-05-17)
- [Security] Apply Django security patches by upgrading from 2.2.20 to 2.2.23.
- [Bugfix] Fix video unit completion (see [pull
  request](https://github.com/edx/edx-platform/pull/27230)).
2021-05-17 21:29:52 +02:00
Régis Behmo
26d14457ec security: upgrade Django to 2.2.23
Also, fix video unit completion in LMS.
2021-05-17 14:25:13 +02:00
Régis Behmo
306d270a3a v11.2.9 (2021-05-12)
- [Bugfix] Fix crashing installation because of a major release of all
Pallets projects.
- [Bugfix] Fix crash in `local quickstart -p` command.
- [Bugfix] Fix 502 error on request to lms with header larger than the
maximum uwsgi buffer size.
2021-05-12 10:14:27 +02:00
Régis Behmo
1a5e2e005b fix: requirements of pallets project
All pallets project requirement had a major upgrade today:
https://palletsprojects.com/blog/flask-2-0-released/

We are not yet compatible with click 8.0 and others. In particular,
click-repl imports modules which are no longer available. Until we can
upgrade, we add constraints to the requirements files.

The following error was being raised:

    $ tutor plugins list
    Traceback (most recent call last):
      File "/home/data/regis/tmp/testtutor/bin/tutor", line 5, in <module>
	from tutor.commands.cli import main
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/tutor/commands/cli.py", line 6, in <module>
	import click_repl
      File "/home/data/regis/tmp/testtutor/lib/python3.6/site-packages/click_repl/__init__.py", line 6, in <module>
	import click._bashcomplete
    ModuleNotFoundError: No module named 'click._bashcomplete'

Close #430.
2021-05-12 10:12:31 +02:00
Régis Behmo
6d92fe2d4a fix: crash during local quickstart -p
When running `tutor local quickstart -p` we were getting the following error:

    Usage: custom [OPTIONS] ARGS...
    Try 'custom --help' for help.

    Error: Missing argument 'ARGS...'.

The docker-compose command sometimes accept a single command ("pull") with zero
argument.

See: https://discuss.overhang.io/t/local-quickstart-not-working-when-pullimages-enabled/1526
2021-05-07 17:01:09 +02:00
wey-gu
452cd229f3 fix: 502 error on request to lms with large header
According to [1], request to uwsgi with header
larger than the default value 4096 will end up
with below error as #426:
  invalid request block size: 4123 (max 4096)...skip
  hr_instance_read(): Connection reset by peer
  [plugins/http/http.c line 647]

This commit fixes it by changing the maximum
buffer size as 4096

Also: Similar issue was already identifed and
fixed in [2] on tutor-ecommerce

[1] https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
[2] 6df2c99362

Close #426
2021-05-01 16:32:19 +02:00
Régis Behmo
dfd3d179e9 v11.2.8 (2021-04-27)
- [Bugfix] Fix parsing of YAML-formatted values in ``config save --set
  KEY=VALUE`` commands, in use for instance with Ecommerce.
2021-04-27 09:26:01 +02:00
Eric Herrera
389dd96fdd fix: YamlParamType supports line terminators
This fix allows using a multiple line formatted Yaml string as input for setting a Tutor config value.
2021-04-25 09:14:21 +02:00
Régis Behmo
2e0f136a23 v11.2.7 (2021-04-23)
- [Security] Apply security patch
  [27394](https://github.com/edx/edx-platform/pull/27394).
- [Feature] Add patches to extend python requirements installation process in
  openedx and openedx-dev Dockerfiles.
- [Improvement] Apply edx-platform patches during Docker image build using
  tutor patch 'openedx-dockerfile-git-patches-default'.
2021-04-23 20:24:08 +02:00
Régis Behmo
b211cb0a0f security: apply security patch 27394
See: https://discuss.openedx.org/t/security-patch-for-xss-fixes/4820
2021-04-23 20:23:11 +02:00
Eric Herrera
17088e2fc6 feat: Conditional edx-platform patching
During Docker images build process, apply custom edx-platform patches when tutor patch 'openedx-dockerfile-git-patches-default' is defined or apply current release patches in other case. It avoids possible conflicts between the actually used edx-platform version and the current release patches.
2021-04-15 07:32:39 +02:00
Eric Herrera
a05486e7b0 feat: openedx Dockerfile python requirements extension patch
Add patches to extend python requirements installation process in openedx and openedx-dev Dockerfiles
2021-04-15 07:32:39 +02:00
Régis Behmo
e532f34909 v11.2.6 (2021-04-09)
- [Improvement] Upgrade all services to open-release/koa.3.
- [Feature] Make it possible to build the openedx Docker image with a custom openedx-i18n version with the ``--build-arg OPENEDX_I18N_VERSION=custom`` command line argument.
2021-04-09 10:16:17 +02:00
Régis Behmo
4a65f24ae7 feat: upgrade all services to open-release/koa.3
We remove security patches and custom fixes which are now part of koa.3.

We take the opportunity to make it possible to build the openedx Docker image
without relying on a corresponding openedx-i18n repo tag: often, we want to
test whether the image simply builds successfully, and we don't need up-to-date
translations. For those cases, it's now possible to pass the `-a
OPENEDX_I18N_VERSION=oldertag` build argument.
2021-04-09 10:13:34 +02:00
Régis Behmo
f2ab27199e v11.2.5 (2021-03-30)
- [Bugfix] Fix edge case where `PLUGINS` entry is null in config.yml.
- [Bugfix] Fix missing py2neo dependency in `images build openedx` (#411).
2021-03-30 09:25:39 +02:00
Régis Behmo
c01f4476b8 fix: TypeError when PLUGINS is None
When the PLUGINS config entry is None (`PLUGINS:`), the following error
was being triggered:

  File "/.../tutor/tutor/plugins.py",
  line 304, in is_enabled
      return name in config.get(CONFIG_KEY, [])
      TypeError: argument of type 'NoneType' is not iterable
2021-03-30 09:23:16 +02:00
Régis Behmo
8db1495497 fix: missing py2neo dependency breaks images build openedx
All releases of py2neo prior to 4.0.0 were stripped from Pypi, so we
need to install this dependency from source.

See:
https://github.com/openedx/build-test-release-wg/issues/37
https://github.com/edx/edx-platform/pull/27168

Close #411.
2021-03-30 09:13:44 +02:00
Régis Behmo
98ed40c800 v11.2.4 (2021-03-17)
- [Bugfix] Fix "Invalid Request" error during SAML authentication
(thanks @Ant1x!).
- [Feature] Add `images build --pull` option to update base images.
- [Improvement] Annotate types all over the Tutor code base.
- [Bugfix] Fix parsing of YAML CLI arguments that include equal "="
signs.
- [Bugfix] Fix minor edge case in `long_to_base64` utility function.
- [Improvement] Add openedx patches to add settings during build
process.
2021-03-17 18:21:16 +01:00
Régis Behmo
f688a279dc Actually remove the images build --pull option
This option is mostly useless to us in CI, as it attempts (and fails) to
update the base image when building the dev image. For good reason: the
base image of the dev image is the latest openedx, which has not been
pushed to docker.io at the time of building -_-
2021-03-17 18:21:16 +01:00
Régis Behmo
b00bae9e72 minor: add missing changelog entry 2021-03-17 17:33:06 +01:00
Régis Behmo
914dbc53be Add images build --pull option to pull base images on build
This will be useful in CI, where we want to automatically update base
images as soon as a security fix comes out.
2021-03-17 16:02:46 +01:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Eric Herrera
6ca863e04c Add patches that enable adding settings during build process (collectsatic and i18ncompile). 2021-03-08 13:09:53 +01:00
Régis Behmo
070b75caa4 fix: parsing of YAML config params with "=" signs
Close #405.
2021-03-08 12:24:40 +01:00
Régis Behmo
de872282b2 fix: minor edge case in long_to_base64 utility function.
This was detected by mypy.
2021-02-25 09:54:46 +01:00
Régis Behmo
59e368a449 v11.2.3 (2021-02-20)
- [Bugfix] Make LMS celery workers actually process LMS tasks, and not
  CMS tasks.
2021-02-20 12:50:49 +01:00
Régis Behmo
89dae24b83 minor changelog formatting
Sentences end with a dot ".".
2021-02-20 12:49:30 +01:00
Régis Behmo
39924121be fix: make LMS process LMS tasks, and not CMS tasks
lms-worker was configured to run CMS tasks instead of LMS tasks. I'm not
sure what tasks were being dismissed, and what is the actual production
impact.
2021-02-20 12:45:17 +01:00
Régis Behmo
6c72877256 v11.2.2 (2021-02-17)
- [Security] Apply security patch [26592](https://github.com/edx/edx-platform/pull/26592)
2021-02-17 20:01:53 +01:00
Régis Behmo
8ad45e6157 Apply security patch 26592
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-9/4297
2021-02-17 20:00:44 +01:00
Régis Behmo
8932ea0541 v11.2.1 (2021-02-10)
- [Bugfix] Actually persist Redis data
2021-02-16 12:40:04 +01:00
Régis Behmo
28afa890bf fix: actually persist redis data
Redis data was not actually persisted, because the redis configuration file was
not mounted from the right location. In order to mount redis data in a
host-mounted directory, the working directory has to be properly set.

The problem was occurring both with docker-compose and Kubernetes.

Close #404.
2021-02-16 12:26:54 +01:00
Régis Behmo
731946cce6 v11.2.0 (2021-02-10)
- [Bugfix] Upgrade all services to open-release/koa.2
2021-02-10 00:18:00 +01:00
Régis Behmo
02f9d8db44 Upgrade to open-release/koa.2 2021-02-10 00:15:54 +01:00
Régis Behmo
1d5886baee v11.1.5 (2021-02-009)
- [Security] Apply security patch [26432](https://github.com/edx/edx-platform/pull/26432).
- [Bugfix] Print warnings to stderr
2021-02-09 09:28:39 +01:00
Régis Behmo
9c559cd30b Apply security patch 26432
See: https://github.com/edx/edx-platform/pull/26432/commits
2021-02-09 09:26:43 +01:00
Régis Behmo
73dfb5a616 Print warnings to stderr
When tutor is run as root, the `tutor config printroot` and `tutor
config printvalue` commands used to write to stdout, which was causing
issues when combined with `tutor config render` and `tutor local
settheme` commands.

See discussion:
https://discuss.overhang.io/t/error-while-install-indigo-theme/1056
2021-02-05 15:44:19 +01:00
Régis Behmo
6bed4f6d4a v11.1.4 (2021-02-04)
- [Security] Apply security patch
  [26358](https://github.com/edx/edx-platform/pull/26358).
2021-02-04 17:02:40 +01:00
Régis Behmo
dca4eb6a32 Apply security patch 26358 2021-02-04 08:55:26 +01:00
Régis Behmo
0f5feeb4eb v11.1.3 (2021-01-31)
- [Security] Apply security patch
  [26253](https://github.com/edx/edx-platform/pull/26253).
2021-01-31 10:58:46 +01:00
Régis Behmo
ed66ccdcf4 Apply security patch #26029
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-6/4162
2021-01-31 10:57:53 +01:00
Régis Behmo
56be99f854 v11.1.2 (2021-01-29)
- [Bugfix] Fix django setting value and static asset loading in openedx-dev
  image
2021-01-29 08:44:22 +01:00
Adolfo R. Brandes
d4b067483d Fix default settings on development environment
First, allow using custom Django settings on a development
environment (as documented but not implemented), setting it to the
correct value of `tutor.development`.  Prior to this, `tutor dev
runserver lms` would default to `tutor.production` when on a custom edX
branch.

Second, fix the documentation so the correct environment variable is
described, at the same time removing an option that doesn't seem to work.

See discussion: https://discuss.overhang.io/t/koa-dev-lms-doesnt-find-static-content/1250
2021-01-29 08:43:46 +01:00
Régis Behmo
6fa1db1e4f v11.1.1 (2021-01-20)
- [Feature] Add a `tutor images build --target=...` argument for [multi-stage
  Docker
builds](https://docs.docker.com/develop/develop-images/multistage-build/).
- [Feature] Create a test version of the openedx-dev Docker image for running
  edx-platform unit tests.
- [Security] Apply security patch
  [26112](https://github.com/edx/edx-platform/pull/26112)
- [Bugfix] Fix `local exec` command which crashed with a `AttributeError`.
2021-01-20 14:38:08 +01:00
Régis Behmo
8d803fb08b Get edx-platform unit tests to run
We manage to get unit tests to run in a dedicated openedx-test container. Only
35 tests are failing (out of 17k). I suspect these tests are also failing in
the devstack.
2021-01-20 14:37:10 +01:00
Régis Behmo
3ab800f2e2 Apply security patch 26112 2021-01-20 14:35:47 +01:00
Régis Behmo
df3d432518 Fix AttributeError on "local exec"
`local exec` was failing with the following stacktrace:

    Traceback (most recent call last):
      File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
        load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 38, in main
        cli()  # pylint: disable=no-value-for-parameter
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)                                                     File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)                                                                   File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 237, in ex
    ecute                                                                                                    dc_command.callback("exec", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 271, in dc_command
        volumes, non_volume_args = bindmounts.parse_volumes(args)                                          File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/bindmounts.py", line 73, in parse_vol
    umes
        context = custom_docker_compose.make_context("custom", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 700, in make_context    self.parse_args(ctx, args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1045, in parse_args
        opts, args, param_order = parser.parse_args(args=args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 269, in parse_args
        self._process_args_for_options(state)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 289, in _process_a
    rgs_for_options
        arg = state.rargs.pop(0)
    AttributeError: 'tuple' object has no attribute 'pop'
2021-01-19 11:51:37 +01:00
Régis Behmo
b648e413e6 v11.1.0 (2021-01-13)
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
  runserver` commands on Mac OS. (again, see #392)
- [Feature] Automatically bind-mount volumes from the `volumes/` directory with
  the `--volume=/...` syntax.
2021-01-13 12:32:48 +01:00
Régis Behmo
b09d2ea1ed Fix "Read-only file system" error in dev (again)
Looks like the `-perm /u=w` argument incorrectly includes files from the
read-only filesystem. This should be fixed by the `-writable` option.

Close #392
2021-01-13 12:27:34 +01:00
Régis Behmo
1c927c6e96 Automatically bind-mount volumes from volumes/
This introduces a new dev/local command:

    tutor dev bindmount CONTAINER PATH

And a new volume syntax:

    tutor dev run --volume=PATH CONTAINER

This syntax automatically bind-mounts folders from the tutorroot/volumes
directory, which is pretty nifty.
2021-01-12 22:43:06 +01:00
Régis Behmo
b015890857 v11.0.7 (2021-01-11)
- [Security] Apply security patch [26029](https://github.com/edx/edx-platform/pull/26029)
2021-01-11 14:20:42 +01:00
Régis Behmo
0435483e14 Apply security patch #26029
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-4/3997
2021-01-11 14:19:50 +01:00
Régis Behmo
d2ec64fe70 v11.0.6 (2021-01-05)
- [Security] Apply security patch
[25974](https://github.com/edx/edx-platform/pull/25974)
2021-01-05 14:06:07 +01:00
Régis Behmo
813a83b029 Apply security patch #25974
See https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-3/3944
2021-01-05 14:05:31 +01:00
Régis Behmo
75af640dbf v11.0.5 (2020-12-30)
- [Bugfix] Fix "Invalid type for parameter ContentType" error on js
upload in Scorm xblock ([openedx-scorm-xblock #16](https://github.com/overhangio/openedx-scorm-xblock/issues/16))
2020-12-30 16:17:34 +01:00
Régis Behmo
3bdc038971 Fix "Invalid type for parameter ContentType" error in scorm xblock
Close https://github.com/overhangio/openedx-scorm-xblock/issues/16
See upstream PR: https://github.com/edx/edx-platform/pull/25957
See discussion: https://discuss.overhang.io/t/issue-with-scorm/1126
2020-12-30 12:21:55 +01:00
Régis Behmo
1c9e112114 v11.0.4 (2020-12-17)
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
runserver` commands on Mac OS. (#392)
2020-12-17 19:12:30 +01:00
Régis Behmo
661d238600 Fix "Read-only file system" errors
Close #392.
2020-12-17 19:10:42 +01:00
Régis Behmo
09ee675ae2 v11.0.3 (2020-12-15)
- [Bugfix] Fix upload of video transcripts to S3 (again)
2020-12-15 12:54:01 +01:00