6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-04 20:37:52 +00:00
Commit Graph

322 Commits

Author SHA1 Message Date
Régis Behmo
39d146d477 Make it possible to run init without starting the full platform
This is done by explicitely listing job dependencies. Unfortunately,
it's not yet possible to move `init` before `start` in `quickstart`,
because some services, such as discovery, depend on the LMS, which takes
a few seconds to boot up. Actually, discovery also depends on ngins, as
it points to "local.overhang.io" when referring to the lms.
2020-09-03 09:00:49 +02:00
Régis Behmo
30cd29e7ef Reduce openedx image size with static asset deduplication
This is inspired from FUN's openedx-docker project: static files are
de-duplicated with rdfind to reduce the uncompressed image size by
320Mb.
2020-09-01 19:15:58 +02:00
Régis Behmo
190f35d4d0 v10.2.1 (2020-08-27)
- [Bugfix] Upgrade all services to open-release/juniper.3
- [Bugfix] Fix upload of video transcripts to S3
- [Improvement] Memorize whether the user is running a production
platform during interactive configuration
2020-08-27 18:33:22 +02:00
Régis Behmo
0b4619ff90 Upgrade to juniper.3
Close #365.
2020-08-27 17:48:07 +02:00
Régis Behmo
e60a99c6af Fix upload of video transcripts to S3
See
https://discuss.overhang.io/t/video-transcript-upload-generates-500-error/836
2020-08-19 10:08:27 +02:00
Régis Behmo
bf0c5e3ed9 Memorize whether the user is running a production platform 2020-08-19 07:54:28 +02:00
Régis Behmo
2ac6f8e8dd v10.2.0 (2020-08-16)
- [Bugfix] Fix incorrect loading of some resources from localhost:18000
in development
- [Bugfix] Fix Samesite=None Secure=False cookie error for users
accessing the LMS with the latest release of Google Chrome
- [Security] Apply javascript security patch ([pull
request](https://github.com/edx/edx-platform/pull/24762))
- [Bugfix] Fix "FileError" on Scorm package upload in Scorm XBlock
- 💥[Improvement] Serve openedx static assets with
[whitenoise](http://whitenoise.evans.io/en/stable/) instead of nginx.
This removes the `k8s-deployments-nginx-init-containers` patch. Plugins
are encouraged to implement static asset serving with Whitenoise as
well.
- [Bugfix] Fix dependency on mysql service when mysql is not activated
- [Improvement] Improve openedx Docker image build time and size with
multi-stage build
- 💥[Feature] Get rid of outdated sysadmin dashboard in LMS at /sysadmin
2020-08-16 12:02:39 +02:00
Régis Behmo
699c589038 Mark sysadmin dashboard changelog entry as breaking 2020-08-16 12:02:35 +02:00
Régis Behmo
0d0cb0e71e Fix incorrect loading of some resources from localhost:18000
In the LMS, some resources where loaded from localhost:18000. For
instance: http://localhost:18000/static/images/logo.png

This was due to the fact that the LMS_BASE, LMS_ROOT_URL and thus
SITE_NAME settings are overwritten by the devstack settings, so we need
to define them again in development.
2020-08-16 11:56:37 +02:00
Régis Behmo
9a6439b08c Fix Samesite=None Secure=False cookie errors in Chrome
Recent releases of Chrome refuse to set csrf and session cookies for
which secure=False samesite=None. The "secure" attribute is not set by
the SameSite middleware in v0.5.1. It was introduced in v0.6.0.

Instead, the "secure" attribute is set on the sessions cookie by the
openedx.core.djangoapps.safe_sessions.middleware.SafeSessionMiddleware
middleware. For the csrf cookie, the "secure" attribute is set by
Django.

We could certainly get rid of the SafeSessionMiddleware by upgrading the
django-cookie-samesite dependency to v0.6.0. Instead, we need to define
environment-specific settings manually.

See:
https://github.com/edx/edx-platform/pull/23671
https://github.com/edx/edx-platform/pull/24593
https://discuss.overhang.io/t/users-cannot-login-csrf-cookie-not-set/815
https://discuss.openedx.org/t/lti-xblock-and-samesite/759/3
https://blog.heroku.com/chrome-changes-samesite-cookie
https://docs.djangoproject.com/en/2.2/ref/settings/#csrf-cookie-secure
https://github.com/jotes/django-cookies-samesite/issues/23
2020-08-16 11:39:00 +02:00
Régis Behmo
1d5bc05328 Apply javascript security patch
See pull request: https://github.com/edx/edx-platform/pull/24762
2020-08-15 19:18:45 +02:00
Régis Behmo
a161dbf843 Fix "FileError" on Scorm package upload in Scorm XBlock
See https://github.com/overhangio/openedx-scorm-xblock/issues/3
2020-08-15 17:24:11 +02:00
Régis Behmo
62d7fd00b9 Serve static assets with whitenoise instead of nginx
This drastically simplifies volume management, as it is no longer
necessary to manually copy static assets from the docker image to the
bind-mounted volume.

This deprecates the "k8s-deployments-nginx-init-containers" patch, as we
no longer need to init the nginx container. Plugins are encouraged to
start using whitenoise as well for serving static assets.

TODO:
- test media serving: DOES NOT WORK. Whitenoise was designed to serve a
fixed list of static files. Godammit.
- compare performances
2020-08-04 09:39:53 +02:00
Régis Behmo
895be87bca Fix dependency on mysql service when mysql is not activated
See https://github.com/overhangio/tutor/pull/359
2020-08-02 16:46:20 +02:00
Régis Behmo
398e1b96e8 Switch to multi-stage build for openedx image
This reduces the size of the final image from 3.25Gb to 2.8Gb. Also, it
should be faster to rebuild the image in most cases. For instance, we
will not have to re-install nodejs requirements after part of the
edx-platform repo was modified.
2020-07-26 00:51:26 +02:00
Régis Behmo
c46cab3df9 Revert "Enable LMS sysadmin dashboard by default"
This reverts commit 90c5842c1c.

Turns out that the sysadmin dashboard is slated for deprecation. It is
currently impossible to download the list of users in csv (see
https://discuss.overhang.io/t/tutor-10-0-11-sysadmin-feature-generates-500-error/776/4).
We should not attempt to resolve this issue.
2020-07-24 08:31:42 +02:00
Régis Behmo
ddee6f4d87 v10.1.0 (2020-07-23)
- [Security] Apply edx-platform upstream xss security fixes ([pull
request](https://github.com/edx/edx-platform/pull/24568))
- 💥[Feature] Make it possible to override the docker registry for just
a few services by setting `DOCKER_IMAGE_SERVICENAME` values.
2020-07-23 16:19:26 +02:00
Régis Behmo
8e2a06dc6f Apply edx-platform upstream xss security fixes 2020-07-23 16:18:40 +02:00
Régis Behmo
bbd92223ee Make it possible to override the docker registry for individual services
Previously, it was not possible to override the docker registry for just
one or a few services. Setting the DOCKER_REGISTRY configuration
parameter would apply to all images. This was inconvenient. To resolve
this, we include the docker registry value in the DOCKER_IMAGE_*
configuration parameters. This allows users to override the docker
registry individually by defining the DOCKER_IMAGE_SERVICENAME
configuration parameter.

See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 09:16:51 +02:00
Régis Behmo
d5659cc03b v10.0.11 (2020-07-16)
- [Feature] Upgrade all repositories to open-release/juniper.2
- [Bugfix] Fix `reload-gunicorn` command
- [Feature] Enable sysadmin dashboard in LMS at /sysadmin
2020-07-16 15:33:28 +02:00
Régis Behmo
ee0f6de499 Add changelog entry for reload-gunicorn fix
See https://github.com/overhangio/tutor/pull/354
2020-07-16 11:58:13 +02:00
Régis Behmo
ef6b1c3433 Upgrade to open-release/juniper.2 2020-07-16 11:57:35 +02:00
Régis Behmo
90c5842c1c Enable LMS sysadmin dashboard by default
The dashboard is available at /sysadmin. It's a CRUD interface for
managing users and courses.

Enabling this interface required that the DATA_DIR setting was not a
string, but a Path object.

Close #353.
2020-07-16 10:49:57 +02:00
Régis Behmo
66ab851848 v10.0.10 (2020-07-01)
- [Bugfix] Fix pycontracts installation error when building openedx
Docker image
- [Bugfix] Fix access to dicussion forum in development mode
2020-07-01 18:27:16 +02:00
Régis Behmo
e1e6a00c0f Fix pycontracts error in openedx docker image build
See
https://discuss.overhang.io/t/tutor-images-build-openedx-failed/717/2
2020-07-01 18:25:51 +02:00
Régis Behmo
a552c5af6f Fix access to dicussion forum in development mode
The devstack settings from edx-platform set the forum url to
http://edx.devstack.forum:4567 so we need to override this value.
2020-07-01 11:04:14 +02:00
Régis Behmo
521b7e8098 v10.0.9 (2020-07-07)
- [Bugfix] Share grade download settings between the LMS and the CMS
2020-07-01 10:10:38 +02:00
Régis Behmo
169a96c44f Share grade download settings between the LMS and the CMS
The GRADES_DOWNLOAD setting is shared between the LMS and the CMS, so
its definition is moved to the common settings file.

This is to address part of this issue:
https://discuss.overhang.io/t/grades-file-not-generating-juniper-version/704
An upgrade to the minio plugin is also required.
2020-07-01 10:06:53 +02:00
Régis Behmo
a9907d7eb3 v10.0.8 (2020-06-23)
- [Bugfix] Fix android user creation during init
- [Bugfix] Fix undefined settings in k8s scripts, such as `createuser`
2020-06-23 19:27:14 +02:00
Régis Behmo
f28ecca498 Fix android user creation during init
Init was crashing with error:

    django.db.utils.IntegrityError: (1062, "Duplicate entry 'android' for key 'client_id'")

See
https://discuss.overhang.io/t/reset-password-email-sent-but-activation-email-dont/690/4
2020-06-23 19:25:56 +02:00
Régis Behmo
b328e0f7d9 Fix undefined settings in k8s scripts
In k8s, creating a user is an interactive command, so it needs to run in
exec. Thus, the DJANGO_SETTINGS_MODULE needs to be defined for this
command.

Close #344
2020-06-23 19:11:35 +02:00
Régis Behmo
2fc28bef79 v10.0.7 (2020-06-22)
- [Bugfix] Fix TypeError on viewing xblock
- [Bugfix] Fix authentication in Android mobile application
2020-06-22 16:31:31 +02:00
Régis Behmo
d8676978f1 Fix TypeError due to None LEARNING_MICROFRONTEND_URL setting value
We observed the following error in production:

    lms_1             | 2020-06-22 14:03:10,220 ERROR 10 [django.request]
    [user 4] log.py:228 - Internal Server Error:
    /xblock/block-v1:edX+DemoX+Demo_Course+type@html+block@6bcccc2d7343416e9e03
    fd7325b2f232
    lms_1             | Traceback (most recent call last):
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/exception.py",
    line 34, in inner
    lms_1             |     response = get_response(request)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py",
    line 115, in _get_response
    lms_1             |     response =
    self.process_exception_by_middleware(e, request)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py",
    line 113, in _get_response
    lms_1             |     response = wrapped_callback(request,
    *callback_args, **callback_kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/views/decorators/http.py",
    line 40, in inner
    lms_1             |     return func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/edx-platform/common/djangoapps/util/views.py", line 66, in
    inner
    lms_1             |     response = view_func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/views/decorators/clickjacking.py",
    line 50, in wrapped_view
    lms_1             |     resp = view_func(*args, **kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/utils/decorators.py",
    line 142, in _wrapped_view
    lms_1             |     response = view_func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/edx-platform/lms/djangoapps/courseware/views/views.py", line
    1683, in render_xblock
    lms_1             |     'is_learning_mfe':
    request.META.get('HTTP_REFERER',
    '').startswith(settings.LEARNING_MICROFRONTEND_URL),
    lms_1             | TypeError: startswith first arg must be str or a
    tuple of str, not NoneType
2020-06-22 16:18:21 +02:00
Régis Behmo
c2e30a6855 Fix authentication in Android mobile application
Login on Android mobile app was failing with "check your username and
password".
2020-06-22 16:01:46 +02:00
Régis Behmo
92c4055b36 v10.0.6 (2020-06-22)
- [Bugfix] Fix unsent activation emails and other asynchronous tasks
2020-06-22 12:19:57 +02:00
Régis Behmo
40a76330f6 Fix unsent activation emails and other asynchronous tasks
Half of the tasks from edx.lms.core.default celery queue were being
processed by the CMS worker. Unfortunately, this CMS worker crashes on
some of those tasks. For instance, activation emails complain of a
missing "django_markup" template tag library because "xss_utils" is not
part of the installed app in the CMS.

The problem is that we need this edx.lms.core.default queue to be part
of the CELERY_QUEUES in the cms in order to send tasks from the CMS to
the LMS. The trick to resolve this situation is to ask the CMS celery
worker to not process the tasks from this queue.

To debug this issue, run in the LMS:

    from student.tasks import send_activation_email
    send_activation_email("{}")

Then watch the logs of the lms and cms workers. If the CMS workers picks
up this task (50% of the time prior to this change) then we have an
issue.

See:
https://discuss.overhang.io/t/reset-password-email-sent-but-activation-email-dont/690
2020-06-22 12:11:15 +02:00
Régis Behmo
cdf6b7e315 v10.0.5 (2020-06-21)
- [Security] Apply edx-platform upstream xss security fixes ([pull
request](https://github.com/edx/edx-platform/pull/24258))
2020-06-21 13:46:19 +02:00
Régis Behmo
1773e2a347 Apply edx-platform upstream xss security fixes 2020-06-21 11:04:46 +02:00
Régis Behmo
77a0a6f60d v10.0.4 (2020-06-19)
- [Bugfix] Fix broken `tutor ui` command in binary
2020-06-19 16:37:58 +02:00
Régis Behmo
2b0ed7daac Fix broken ui command in tutor binary
"tutor ui" was failing miserably, printing a lot of garbled characters
in the shell. In fact, a FileNotFound error was being raised and
automatically caught by the ui command. When removing the catch all,
this was the error that was raised:

    FileNotFoundError: [Errno 2] No such file or directory:
    '/tmp/_MEIimsqmq/wcwidth/version.json'     │

This is resolved on SO:
https://stackoverflow.com/questions/62155242/pyinstaller-cant-find-wcwidth-version-json-when-running-executable
2020-06-19 16:33:58 +02:00
Régis Behmo
001da4e24e v10.0.3 (2020-06-19)
- [Bugfix] Fix error on android user creation
2020-06-19 12:53:18 +02:00
Régis Behmo
2f8efbc656 Revert "Fix DoesNotExist error on android credentials creation"
This reverts commit a88d9cfc19.
2020-06-19 11:40:46 +02:00
Régis Behmo
d9aab8b314 v10.0.1 (2020-06-17)
- [Bugfix] Fix crash when viewing problem in LMS
- [Bugfix] Fix missing webpack-stats.json in openedx Docker image
2020-06-17 21:51:52 +02:00
Régis Behmo
baca27bfbd Fix crash when viewing problem in LMS
This was due to incorrectly loading the coursewarehistoryextended in the
installed applications. Also, the database router in charge of routing
requests to the student_history_module database must be disabled.
2020-06-17 11:10:37 +02:00
Régis Behmo
325c1037c0 Fix missing webpack-stats.json in openedx Docker image
In CI, the webpack-stats.json file sometimes contains just:

    {"status":"compiling"}

This was due to the fact that the `subprocess.call(...)` command in
openedx-assets did not check whether the command was killed -- for lack
of memory for instance. This is resolved by replacing "call(...)" by
"check_call(...)".
2020-06-16 06:22:12 +02:00
Régis Behmo
24035ac1fe v10.0.1 (2020-06-15)
- [Bugfix] Fix KeyError when running ``local quickstart`` for the first
time
2020-06-15 17:59:27 +02:00
Régis Behmo
06fe19fcf2 Fix KeyError when running `local quickstart` for the first time
This was due to incorrect parsing of the version number.
2020-06-15 17:57:14 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
29c6741bba Fix nginx resolver address 2020-06-14 15:47:37 +02:00
Régis Behmo
872a33414a Add --limit=myplugin option to init commands
This makes it possible to run "init" for some services only, which is
useful during debugging.
2020-06-02 14:30:06 +02:00