6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-14 09:14:14 +00:00
Commit Graph

550 Commits

Author SHA1 Message Date
Régis Behmo
28afa890bf fix: actually persist redis data
Redis data was not actually persisted, because the redis configuration file was
not mounted from the right location. In order to mount redis data in a
host-mounted directory, the working directory has to be properly set.

The problem was occurring both with docker-compose and Kubernetes.

Close #404.
2021-02-16 12:26:54 +01:00
Régis Behmo
731946cce6 v11.2.0 (2021-02-10)
- [Bugfix] Upgrade all services to open-release/koa.2
2021-02-10 00:18:00 +01:00
Régis Behmo
02f9d8db44 Upgrade to open-release/koa.2 2021-02-10 00:15:54 +01:00
Régis Behmo
1d5886baee v11.1.5 (2021-02-009)
- [Security] Apply security patch [26432](https://github.com/edx/edx-platform/pull/26432).
- [Bugfix] Print warnings to stderr
2021-02-09 09:28:39 +01:00
Régis Behmo
9c559cd30b Apply security patch 26432
See: https://github.com/edx/edx-platform/pull/26432/commits
2021-02-09 09:26:43 +01:00
Régis Behmo
73dfb5a616 Print warnings to stderr
When tutor is run as root, the `tutor config printroot` and `tutor
config printvalue` commands used to write to stdout, which was causing
issues when combined with `tutor config render` and `tutor local
settheme` commands.

See discussion:
https://discuss.overhang.io/t/error-while-install-indigo-theme/1056
2021-02-05 15:44:19 +01:00
Régis Behmo
6bed4f6d4a v11.1.4 (2021-02-04)
- [Security] Apply security patch
  [26358](https://github.com/edx/edx-platform/pull/26358).
2021-02-04 17:02:40 +01:00
Régis Behmo
dca4eb6a32 Apply security patch 26358 2021-02-04 08:55:26 +01:00
Régis Behmo
0f5feeb4eb v11.1.3 (2021-01-31)
- [Security] Apply security patch
  [26253](https://github.com/edx/edx-platform/pull/26253).
2021-01-31 10:58:46 +01:00
Régis Behmo
ed66ccdcf4 Apply security patch #26029
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-6/4162
2021-01-31 10:57:53 +01:00
Régis Behmo
56be99f854 v11.1.2 (2021-01-29)
- [Bugfix] Fix django setting value and static asset loading in openedx-dev
  image
2021-01-29 08:44:22 +01:00
Adolfo R. Brandes
d4b067483d Fix default settings on development environment
First, allow using custom Django settings on a development
environment (as documented but not implemented), setting it to the
correct value of `tutor.development`.  Prior to this, `tutor dev
runserver lms` would default to `tutor.production` when on a custom edX
branch.

Second, fix the documentation so the correct environment variable is
described, at the same time removing an option that doesn't seem to work.

See discussion: https://discuss.overhang.io/t/koa-dev-lms-doesnt-find-static-content/1250
2021-01-29 08:43:46 +01:00
Régis Behmo
6fa1db1e4f v11.1.1 (2021-01-20)
- [Feature] Add a `tutor images build --target=...` argument for [multi-stage
  Docker
builds](https://docs.docker.com/develop/develop-images/multistage-build/).
- [Feature] Create a test version of the openedx-dev Docker image for running
  edx-platform unit tests.
- [Security] Apply security patch
  [26112](https://github.com/edx/edx-platform/pull/26112)
- [Bugfix] Fix `local exec` command which crashed with a `AttributeError`.
2021-01-20 14:38:08 +01:00
Régis Behmo
8d803fb08b Get edx-platform unit tests to run
We manage to get unit tests to run in a dedicated openedx-test container. Only
35 tests are failing (out of 17k). I suspect these tests are also failing in
the devstack.
2021-01-20 14:37:10 +01:00
Régis Behmo
3ab800f2e2 Apply security patch 26112 2021-01-20 14:35:47 +01:00
Régis Behmo
df3d432518 Fix AttributeError on "local exec"
`local exec` was failing with the following stacktrace:

    Traceback (most recent call last):
      File "/home/regis/venvs/tutor/bin/tutor", line 11, in <module>
        load_entry_point('tutor-openedx', 'console_scripts', 'tutor')()
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/cli.py", line 38, in main
        cli()  # pylint: disable=no-value-for-parameter
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1259, in invoke         return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)                                                     File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)                                                                   File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 237, in ex
    ecute                                                                                                    dc_command.callback("exec", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/commands/compose.py", line 271, in dc_command
        volumes, non_volume_args = bindmounts.parse_volumes(args)                                          File "/home/regis/projets/overhang/repos/overhang/tutor/tutor/bindmounts.py", line 73, in parse_vol
    umes
        context = custom_docker_compose.make_context("custom", args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 700, in make_context    self.parse_args(ctx, args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/core.py", line 1045, in parse_args
        opts, args, param_order = parser.parse_args(args=args)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 269, in parse_args
        self._process_args_for_options(state)
      File "/home/regis/venvs/tutor/lib/python3.8/site-packages/click/parser.py", line 289, in _process_a
    rgs_for_options
        arg = state.rargs.pop(0)
    AttributeError: 'tuple' object has no attribute 'pop'
2021-01-19 11:51:37 +01:00
Régis Behmo
b648e413e6 v11.1.0 (2021-01-13)
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
  runserver` commands on Mac OS. (again, see #392)
- [Feature] Automatically bind-mount volumes from the `volumes/` directory with
  the `--volume=/...` syntax.
2021-01-13 12:32:48 +01:00
Régis Behmo
b09d2ea1ed Fix "Read-only file system" error in dev (again)
Looks like the `-perm /u=w` argument incorrectly includes files from the
read-only filesystem. This should be fixed by the `-writable` option.

Close #392
2021-01-13 12:27:34 +01:00
Régis Behmo
1c927c6e96 Automatically bind-mount volumes from volumes/
This introduces a new dev/local command:

    tutor dev bindmount CONTAINER PATH

And a new volume syntax:

    tutor dev run --volume=PATH CONTAINER

This syntax automatically bind-mounts folders from the tutorroot/volumes
directory, which is pretty nifty.
2021-01-12 22:43:06 +01:00
Régis Behmo
b015890857 v11.0.7 (2021-01-11)
- [Security] Apply security patch [26029](https://github.com/edx/edx-platform/pull/26029)
2021-01-11 14:20:42 +01:00
Régis Behmo
0435483e14 Apply security patch #26029
See: https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-4/3997
2021-01-11 14:19:50 +01:00
Régis Behmo
d2ec64fe70 v11.0.6 (2021-01-05)
- [Security] Apply security patch
[25974](https://github.com/edx/edx-platform/pull/25974)
2021-01-05 14:06:07 +01:00
Régis Behmo
813a83b029 Apply security patch #25974
See https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-3/3944
2021-01-05 14:05:31 +01:00
Régis Behmo
75af640dbf v11.0.5 (2020-12-30)
- [Bugfix] Fix "Invalid type for parameter ContentType" error on js
upload in Scorm xblock ([openedx-scorm-xblock #16](https://github.com/overhangio/openedx-scorm-xblock/issues/16))
2020-12-30 16:17:34 +01:00
Régis Behmo
3bdc038971 Fix "Invalid type for parameter ContentType" error in scorm xblock
Close https://github.com/overhangio/openedx-scorm-xblock/issues/16
See upstream PR: https://github.com/edx/edx-platform/pull/25957
See discussion: https://discuss.overhang.io/t/issue-with-scorm/1126
2020-12-30 12:21:55 +01:00
Régis Behmo
1c9e112114 v11.0.4 (2020-12-17)
- [Bugfix] Fix "Read-only file system" errors when running `tutor dev
runserver` commands on Mac OS. (#392)
2020-12-17 19:12:30 +01:00
Régis Behmo
661d238600 Fix "Read-only file system" errors
Close #392.
2020-12-17 19:10:42 +01:00
Régis Behmo
09ee675ae2 v11.0.3 (2020-12-15)
- [Bugfix] Fix upload of video transcripts to S3 (again)
2020-12-15 12:54:01 +01:00
Régis Behmo
d45c354191 Fix upload of video transcripts to S3 (again) 2020-12-15 12:53:23 +01:00
Régis Behmo
189cefa5f7 v11.0.2 (2020-12-12)
- [Bugfix] Fix missing celery tasks from edx-platform (see [upstream
PR](https://github.com/edx/edx-platform/pull/25840))
2020-12-12 11:49:09 +01:00
Régis Behmo
4d9b85a849 Fix missing celery tasks in edx-platform
See: https://github.com/edx/edx-platform/pull/25840
2020-12-12 11:48:39 +01:00
Régis Behmo
88814c5457 v11.0.1 (2020-12-10)
- [Security] Apply security patch
[25834](https://github.com/edx/edx-platform/pull/25834)
- [Bugfix] Fix Android apk directory mount path
2020-12-10 13:21:58 +01:00
Régis Behmo
1394090a48 Apply security patch
See:
https://github.com/edx/edx-platform/pull/25834/
https://discuss.openedx.org/t/security-im-patch-for-xss-fixes-2/3802
2020-12-10 13:21:25 +01:00
Régis Behmo
08e2a30e6b Fix Android apk directory mount path 2020-12-10 11:16:32 +01:00
Régis Behmo
728ef966dc v11.0.0 (2020-12-09)
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
    - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
    - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
    - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
    - The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
    - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
	- Nginx no longer listens to port 443 for https traffic
	- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
	- Configuration files for web proxies are no longer provided.
	- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
    - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
    - Replace memcached and rabbitmq by redis.
- Additional features:
    - Make it possible to disable all plugins at once with ``plugins disable all``.
    - Add ``tutor k8s wait`` command to wait for a pod to become ready
    - Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
    - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
    - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
2020-12-10 01:05:02 +01:00
Régis Behmo
d3c842c8da v10.5.3 (2020-12-09)
- [Security] Apply upstream edx-platform [security
patch](https://github.com/edx/edx-platform/pull/25782)
2020-12-09 21:50:42 +01:00
Régis Behmo
0f44d32147 Apply upstream security patch 2020-12-09 13:34:14 +01:00
Régis Behmo
0fc1358eee v10.5.2 (2020-12-07)
- [Improvement] Increase the timeout of the gunicorn worker command in
openedx Dockerfile
2020-12-07 13:23:35 +01:00
Maarten de Waard
b42beb83d9 Increase the timeout of the gunicorn worker command in openedx Dockerfile
As discussed in
https://discuss.overhang.io/t/lms-worker-timeout-issue/1057/8, the
timeout of the gunicorn worker is a bit low, causing the worker to
crashloop on slower systems. By increasing the timeout to 120 seconds,
the worker should have time to come up before it gets restarted.
2020-12-07 13:21:41 +01:00
Régis Behmo
790aa2b2d2 v10.5.1 (2020-11-30)
- [Bugfix] Fix Dockerfile parsing on Windows
- [Improvement] Add option to patch lms and cms nginx server blocks
2020-11-30 15:42:19 +01:00
Maarten de Waard
5aa78273cf add option to patch lms and cms nginx server block 2020-11-30 15:34:43 +01:00
Régis Behmo
727c204e92 Fix Dockerfile parsing on Windows
Line endings are written as "\r" by default on Windows, which makes the
Dockerfiles unreadable by Docker.

See: https://github.com/overhangio/tutor/pull/385#issuecomment-730387969
2020-11-20 16:05:56 +01:00
Régis Behmo
7b16af22b8 v10.5.0 (2020-11-19)
- 💥[Improvement] Remove `dev/local pullimages`. Instead, run `dev/local
dc pull`.
- 💥[Improvement] Add `dev dc` and `local dc` commands as thin wrappers
of the `docker-compose` CLI.
- 💥[Improvement] Remove the undocumented `local run_hook` command.
Instead, run `local init --limit=...`.
- 💥[Improvement] Remove `tutor android pullimage` command. Instead, run
`tutor images pull android`.
- [Bugfix] Fix `config save` on Windows.
- [Bugfix] Fix platform checking in user ID function
2020-11-19 11:32:17 +01:00
Régis Behmo
cacb4f4cfd Remove dev/local pullimages command
Now that we have the `dc` command this can be simplified to `dc pull`.
2020-11-16 12:50:01 +01:00
Régis Behmo
3332f9a330 Add dev dc and local dc commands
These are thin wrappers of `docker-compose` -- with all the right
arguments in place.
2020-11-16 12:46:01 +01:00
Régis Behmo
9a9ddf5497 Remove the undocumented run_hook command
This undocumented command was superseded by `local init --limit=...`.
2020-11-16 12:22:28 +01:00
Régis Behmo
df2e977b35 Remove android pullimage command
Instead, run `images pull android`.
2020-11-16 12:19:53 +01:00
Régis Behmo
b2fdaf604e Fix platform checking and env saving on Windows 2020-11-12 12:56:18 +01:00
Régis Behmo
2596e5f7d2 v10.4.1 (2020-11-11)
- [Bugfix] Fix dependency error during `pip install tutor` due to urllib3 incompatibility
- [Bugfix] Fix user ID checking under Windows.
- [Bugfix] Fix template rendering for Windows users.
- [Improvement] Switch to `bcrypt` for htpasswd password generation, for better portability on Windows.
- [Improvement] In the openedx production docker image, add some jitter to the gunicorn worker restart process to prevent all workers from restarting at the same time.
2020-11-11 10:39:58 +01:00
Régis Behmo
ad858cd699 Fix urllib3 dependency error on installation of tutor
requests depends on urllib3<1.26.0, while urllib3==1.26.0 was just
released. We need to introduce a constraint on urllib3 to avoid the
following error when running `pip install -e`:

    pkg_resources.ContextualVersionConflict: (urllib3 1.26.0
    (./venv/lib/python3.7/site-packages),
    Requirement.parse('urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1'),
    {'requests'})

 We will be able to remove this constraint once requests becomes
 compatible with urllib3==1.26.0. To test this change just run `pip
 install -e tutor` and then run a dummy command, such as `tutor local
 stop`.
2020-11-11 10:36:42 +01:00
Régis Behmo
6c4d9619fd Fix user ID checking under windows
This should make tutor portable on Windows. Well, if the `id -u` command
is supported at least...

See: https://github.com/overhangio/tutor/issues/381
2020-11-07 17:48:20 +01:00
Régis Behmo
998dec7149 Fix template name separator for Windows users
See: https://github.com/overhangio/tutor/issues/381
2020-11-07 16:37:43 +01:00
Régis Behmo
6848253b8e Switch from crypt to bcrypt for htpasswd generation
This is more portable, as described here:
https://github.com/overhangio/tutor/issues/381
2020-11-07 14:46:53 +01:00
Régis Behmo
02cc106af3 Add some jitter to the openedx gunicorn workers
This should prevent gunicorn workers from restarting all at the same
time.
2020-11-04 18:19:35 +01:00
Régis Behmo
91e647692a Add upgrade instructions for AMI users
AWS unhelpfully does not include upgrade instructions in the upgrade
email, so...
2020-11-03 10:19:04 +01:00
Régis Behmo
58ec1b9d79 v10.4.0 (2020-10-30)
- [Improvement] Mount config and setting files in read-only mode
- 💥[Improvement] Enable unit completion tracking by default.
- [Bugfix] Run `apt update` before `apt install` when installing deps in
the openedx Dockerfile
2020-10-30 18:20:19 +01:00
Régis Behmo
efee5b55cb Mount config and setting files in read-only mode
This is part of the process to get rid of as many writable mounts as
possible.
2020-10-25 18:44:54 +01:00
Régis Behmo
d8a4b26884 Enable unit completion tracking by default
See discussion:
https://discuss.overhang.io/t/course-not-starting/962/12
2020-10-22 15:24:40 +02:00
Rafael Sousa
dd0ab446a3 run apt update before installing deps 2020-10-19 17:51:13 +02:00
Régis Behmo
ed14fb1746 v10.3.1 (2020-10-16)
- [Improvement] Make all commands considerably faster
- [Improvement] Make it easier to override Mongodb connection parameters
- [Bugfix] Add support for .woff and .woff2 font files in themes (thanks @mrtndwrd!)
2020-10-16 12:45:40 +02:00
Régis Behmo
f32eb08940 Make it easier to override mongodb connection parameters
Pymongo clients can connect to Mongodb with many parameters
(https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient)
but we don't want to create a new Tutor setting for every one of them.
This makes it easy for users to override Mongodb connection parameters,
simply by modifying the `mongodb_parameters` object.
In particular, this makes it easy to add "replicaSet" connection
parameters.
See https://github.com/overhangio/tutor/pull/372
2020-10-15 14:35:41 +02:00
Régis Behmo
e17eede179 Add changelog entry for woff/woff2 change
See https://github.com/overhangio/tutor/pull/373
2020-10-15 14:29:27 +02:00
Régis Behmo
dac7da4bae v10.3.0 (2020-10-13)
- 💥[Improvement] Simplify CORS configuration
2020-10-13 14:57:05 +02:00
Régis Behmo
2f7742b099 Simplify CORS configuration
CORS headers are no longer handled by nginx, but directly by the LMS.
This makes it possible by 3rd-party apps to easily add domain names to
the CORS whitelist, even when they are not subdomains of the LMS.
2020-10-13 13:34:12 +02:00
Régis Behmo
9f94a12c7d v10.2.4 (2020-10-06)
- [Bugfix] Fix Apache proxy configuration when HTTPS is enabled (#370)
2020-10-06 12:01:25 +02:00
Régis Behmo
9291b2e1b9 Fix Apache proxy configuration when HTTPS is enabled
Close #370
2020-10-06 12:00:38 +02:00
Régis Behmo
e8e44f50b0 v10.2.3 (2020-10-02)
- [Feature] Add ``images printtag`` command
- [Improvement] Make it possible to override individual contact email
addresses with plugins
- [Bugfix] Replace "no-reply@LMS_BASE" email address by regular contact
email address
- [Bugfix] Disable learner records globally by default
- [Improvement] Upgrade to the latest release of MySQL 5.6
- [Improvement] Non-plugin settings added by "set" directives are now
automatically removed when the plugin is disabled (#241)
2020-10-02 14:18:52 +02:00
Régis Behmo
ccee481dd0 Add images printtag command 2020-10-02 00:25:03 +02:00
Régis Behmo
c8434d500f Fix no-reply email address in lms/cms 2020-10-01 13:41:28 +02:00
Régis Behmo
371b2b5c2d Disable learner records globally by default
Previously, platform administrators had to define a site configuration
to disable this feature.

See https://discuss.overhang.io/t/account-settings-is-not-working/967/3
2020-10-01 13:39:45 +02:00
Régis Behmo
718a6a5cf9 Upgrade to the latest release of MySQL 5.6
Hopefully, this should resolve some initialisation issues.
2020-09-23 15:33:22 +02:00
Régis Behmo
f5f501dad0 Remove "set" config entries on disabling plugin
Close #241
2020-09-18 13:21:15 +02:00
Régis Behmo
f0a3cc6072 v10.2.2 (2020-09-05)
- [Improvement] Add CORS basic configuration to LMS for subdomains of
the LMS
- [Feature] Add support for `images build --add-host` option (thanks
@grinderz!)
- [Bugfix] Fix podman compatibility by replacing `docker-compose rm`
command by `docker-compose stop` when stopping containers
- [Improvement] Improve plugin data deletion
- [Improvement] Introduce the `OPENEDX_COMMON_VERSION` setting
- [Bugfix] Make it possible to run init jobs without starting the entire
platform
- [Improvement] Reduce "openedx" Docker image size with static asset
de-duplication
2020-09-05 11:35:26 +02:00
Régis Behmo
055c3cad3f Add CORS config to LMS
By default, all subdomains of the LMS are allowed.

For reference:
https://enable-cors.org/server_nginx.html
https://stackoverflow.com/questions/54313216/nginx-config-to-enable-cors-with-origin-matching
2020-09-04 22:51:40 +02:00
Régis Behmo
b1c7dab03e Add support for images build --add-host
This was originally proposed here by @grinderz:
https://github.com/overhangio/tutor/pull/355
2020-09-04 12:35:44 +02:00
Régis Behmo
e9585bdc80 Switch to docker-compose stop for stopping containers
I do not understand the reason why we should use "rm" instead of "stop".

This fixes podman compatibility:
https://discuss.overhang.io/t/tutor-with-podman-not-working/905
2020-09-04 12:07:44 +02:00
Régis Behmo
fd50f3c384 Improve plugin data deletion 2020-09-04 12:03:50 +02:00
Régis Behmo
ea3839eb74 Introduce the OPENEDX_COMMON_VERSION setting 2020-09-03 17:33:25 +02:00
Régis Behmo
39d146d477 Make it possible to run init without starting the full platform
This is done by explicitely listing job dependencies. Unfortunately,
it's not yet possible to move `init` before `start` in `quickstart`,
because some services, such as discovery, depend on the LMS, which takes
a few seconds to boot up. Actually, discovery also depends on ngins, as
it points to "local.overhang.io" when referring to the lms.
2020-09-03 09:00:49 +02:00
Régis Behmo
30cd29e7ef Reduce openedx image size with static asset deduplication
This is inspired from FUN's openedx-docker project: static files are
de-duplicated with rdfind to reduce the uncompressed image size by
320Mb.
2020-09-01 19:15:58 +02:00
Régis Behmo
190f35d4d0 v10.2.1 (2020-08-27)
- [Bugfix] Upgrade all services to open-release/juniper.3
- [Bugfix] Fix upload of video transcripts to S3
- [Improvement] Memorize whether the user is running a production
platform during interactive configuration
2020-08-27 18:33:22 +02:00
Régis Behmo
0b4619ff90 Upgrade to juniper.3
Close #365.
2020-08-27 17:48:07 +02:00
Régis Behmo
e60a99c6af Fix upload of video transcripts to S3
See
https://discuss.overhang.io/t/video-transcript-upload-generates-500-error/836
2020-08-19 10:08:27 +02:00
Régis Behmo
bf0c5e3ed9 Memorize whether the user is running a production platform 2020-08-19 07:54:28 +02:00
Régis Behmo
2ac6f8e8dd v10.2.0 (2020-08-16)
- [Bugfix] Fix incorrect loading of some resources from localhost:18000
in development
- [Bugfix] Fix Samesite=None Secure=False cookie error for users
accessing the LMS with the latest release of Google Chrome
- [Security] Apply javascript security patch ([pull
request](https://github.com/edx/edx-platform/pull/24762))
- [Bugfix] Fix "FileError" on Scorm package upload in Scorm XBlock
- 💥[Improvement] Serve openedx static assets with
[whitenoise](http://whitenoise.evans.io/en/stable/) instead of nginx.
This removes the `k8s-deployments-nginx-init-containers` patch. Plugins
are encouraged to implement static asset serving with Whitenoise as
well.
- [Bugfix] Fix dependency on mysql service when mysql is not activated
- [Improvement] Improve openedx Docker image build time and size with
multi-stage build
- 💥[Feature] Get rid of outdated sysadmin dashboard in LMS at /sysadmin
2020-08-16 12:02:39 +02:00
Régis Behmo
699c589038 Mark sysadmin dashboard changelog entry as breaking 2020-08-16 12:02:35 +02:00
Régis Behmo
0d0cb0e71e Fix incorrect loading of some resources from localhost:18000
In the LMS, some resources where loaded from localhost:18000. For
instance: http://localhost:18000/static/images/logo.png

This was due to the fact that the LMS_BASE, LMS_ROOT_URL and thus
SITE_NAME settings are overwritten by the devstack settings, so we need
to define them again in development.
2020-08-16 11:56:37 +02:00
Régis Behmo
9a6439b08c Fix Samesite=None Secure=False cookie errors in Chrome
Recent releases of Chrome refuse to set csrf and session cookies for
which secure=False samesite=None. The "secure" attribute is not set by
the SameSite middleware in v0.5.1. It was introduced in v0.6.0.

Instead, the "secure" attribute is set on the sessions cookie by the
openedx.core.djangoapps.safe_sessions.middleware.SafeSessionMiddleware
middleware. For the csrf cookie, the "secure" attribute is set by
Django.

We could certainly get rid of the SafeSessionMiddleware by upgrading the
django-cookie-samesite dependency to v0.6.0. Instead, we need to define
environment-specific settings manually.

See:
https://github.com/edx/edx-platform/pull/23671
https://github.com/edx/edx-platform/pull/24593
https://discuss.overhang.io/t/users-cannot-login-csrf-cookie-not-set/815
https://discuss.openedx.org/t/lti-xblock-and-samesite/759/3
https://blog.heroku.com/chrome-changes-samesite-cookie
https://docs.djangoproject.com/en/2.2/ref/settings/#csrf-cookie-secure
https://github.com/jotes/django-cookies-samesite/issues/23
2020-08-16 11:39:00 +02:00
Régis Behmo
1d5bc05328 Apply javascript security patch
See pull request: https://github.com/edx/edx-platform/pull/24762
2020-08-15 19:18:45 +02:00
Régis Behmo
a161dbf843 Fix "FileError" on Scorm package upload in Scorm XBlock
See https://github.com/overhangio/openedx-scorm-xblock/issues/3
2020-08-15 17:24:11 +02:00
Régis Behmo
62d7fd00b9 Serve static assets with whitenoise instead of nginx
This drastically simplifies volume management, as it is no longer
necessary to manually copy static assets from the docker image to the
bind-mounted volume.

This deprecates the "k8s-deployments-nginx-init-containers" patch, as we
no longer need to init the nginx container. Plugins are encouraged to
start using whitenoise as well for serving static assets.

TODO:
- test media serving: DOES NOT WORK. Whitenoise was designed to serve a
fixed list of static files. Godammit.
- compare performances
2020-08-04 09:39:53 +02:00
Régis Behmo
895be87bca Fix dependency on mysql service when mysql is not activated
See https://github.com/overhangio/tutor/pull/359
2020-08-02 16:46:20 +02:00
Régis Behmo
398e1b96e8 Switch to multi-stage build for openedx image
This reduces the size of the final image from 3.25Gb to 2.8Gb. Also, it
should be faster to rebuild the image in most cases. For instance, we
will not have to re-install nodejs requirements after part of the
edx-platform repo was modified.
2020-07-26 00:51:26 +02:00
Régis Behmo
c46cab3df9 Revert "Enable LMS sysadmin dashboard by default"
This reverts commit 90c5842c1c.

Turns out that the sysadmin dashboard is slated for deprecation. It is
currently impossible to download the list of users in csv (see
https://discuss.overhang.io/t/tutor-10-0-11-sysadmin-feature-generates-500-error/776/4).
We should not attempt to resolve this issue.
2020-07-24 08:31:42 +02:00
Régis Behmo
ddee6f4d87 v10.1.0 (2020-07-23)
- [Security] Apply edx-platform upstream xss security fixes ([pull
request](https://github.com/edx/edx-platform/pull/24568))
- 💥[Feature] Make it possible to override the docker registry for just
a few services by setting `DOCKER_IMAGE_SERVICENAME` values.
2020-07-23 16:19:26 +02:00
Régis Behmo
8e2a06dc6f Apply edx-platform upstream xss security fixes 2020-07-23 16:18:40 +02:00
Régis Behmo
bbd92223ee Make it possible to override the docker registry for individual services
Previously, it was not possible to override the docker registry for just
one or a few services. Setting the DOCKER_REGISTRY configuration
parameter would apply to all images. This was inconvenient. To resolve
this, we include the docker registry value in the DOCKER_IMAGE_*
configuration parameters. This allows users to override the docker
registry individually by defining the DOCKER_IMAGE_SERVICENAME
configuration parameter.

See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 09:16:51 +02:00
Régis Behmo
d5659cc03b v10.0.11 (2020-07-16)
- [Feature] Upgrade all repositories to open-release/juniper.2
- [Bugfix] Fix `reload-gunicorn` command
- [Feature] Enable sysadmin dashboard in LMS at /sysadmin
2020-07-16 15:33:28 +02:00
Régis Behmo
ee0f6de499 Add changelog entry for reload-gunicorn fix
See https://github.com/overhangio/tutor/pull/354
2020-07-16 11:58:13 +02:00
Régis Behmo
ef6b1c3433 Upgrade to open-release/juniper.2 2020-07-16 11:57:35 +02:00
Régis Behmo
90c5842c1c Enable LMS sysadmin dashboard by default
The dashboard is available at /sysadmin. It's a CRUD interface for
managing users and courses.

Enabling this interface required that the DATA_DIR setting was not a
string, but a Path object.

Close #353.
2020-07-16 10:49:57 +02:00
Régis Behmo
66ab851848 v10.0.10 (2020-07-01)
- [Bugfix] Fix pycontracts installation error when building openedx
Docker image
- [Bugfix] Fix access to dicussion forum in development mode
2020-07-01 18:27:16 +02:00
Régis Behmo
e1e6a00c0f Fix pycontracts error in openedx docker image build
See
https://discuss.overhang.io/t/tutor-images-build-openedx-failed/717/2
2020-07-01 18:25:51 +02:00
Régis Behmo
a552c5af6f Fix access to dicussion forum in development mode
The devstack settings from edx-platform set the forum url to
http://edx.devstack.forum:4567 so we need to override this value.
2020-07-01 11:04:14 +02:00
Régis Behmo
521b7e8098 v10.0.9 (2020-07-07)
- [Bugfix] Share grade download settings between the LMS and the CMS
2020-07-01 10:10:38 +02:00
Régis Behmo
169a96c44f Share grade download settings between the LMS and the CMS
The GRADES_DOWNLOAD setting is shared between the LMS and the CMS, so
its definition is moved to the common settings file.

This is to address part of this issue:
https://discuss.overhang.io/t/grades-file-not-generating-juniper-version/704
An upgrade to the minio plugin is also required.
2020-07-01 10:06:53 +02:00
Régis Behmo
a9907d7eb3 v10.0.8 (2020-06-23)
- [Bugfix] Fix android user creation during init
- [Bugfix] Fix undefined settings in k8s scripts, such as `createuser`
2020-06-23 19:27:14 +02:00
Régis Behmo
f28ecca498 Fix android user creation during init
Init was crashing with error:

    django.db.utils.IntegrityError: (1062, "Duplicate entry 'android' for key 'client_id'")

See
https://discuss.overhang.io/t/reset-password-email-sent-but-activation-email-dont/690/4
2020-06-23 19:25:56 +02:00
Régis Behmo
b328e0f7d9 Fix undefined settings in k8s scripts
In k8s, creating a user is an interactive command, so it needs to run in
exec. Thus, the DJANGO_SETTINGS_MODULE needs to be defined for this
command.

Close #344
2020-06-23 19:11:35 +02:00
Régis Behmo
2fc28bef79 v10.0.7 (2020-06-22)
- [Bugfix] Fix TypeError on viewing xblock
- [Bugfix] Fix authentication in Android mobile application
2020-06-22 16:31:31 +02:00
Régis Behmo
d8676978f1 Fix TypeError due to None LEARNING_MICROFRONTEND_URL setting value
We observed the following error in production:

    lms_1             | 2020-06-22 14:03:10,220 ERROR 10 [django.request]
    [user 4] log.py:228 - Internal Server Error:
    /xblock/block-v1:edX+DemoX+Demo_Course+type@html+block@6bcccc2d7343416e9e03
    fd7325b2f232
    lms_1             | Traceback (most recent call last):
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/exception.py",
    line 34, in inner
    lms_1             |     response = get_response(request)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py",
    line 115, in _get_response
    lms_1             |     response =
    self.process_exception_by_middleware(e, request)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py",
    line 113, in _get_response
    lms_1             |     response = wrapped_callback(request,
    *callback_args, **callback_kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/views/decorators/http.py",
    line 40, in inner
    lms_1             |     return func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/edx-platform/common/djangoapps/util/views.py", line 66, in
    inner
    lms_1             |     response = view_func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/views/decorators/clickjacking.py",
    line 50, in wrapped_view
    lms_1             |     resp = view_func(*args, **kwargs)
    lms_1             |   File
    "/openedx/venv/lib/python3.5/site-packages/django/utils/decorators.py",
    line 142, in _wrapped_view
    lms_1             |     response = view_func(request, *args, **kwargs)
    lms_1             |   File
    "/openedx/edx-platform/lms/djangoapps/courseware/views/views.py", line
    1683, in render_xblock
    lms_1             |     'is_learning_mfe':
    request.META.get('HTTP_REFERER',
    '').startswith(settings.LEARNING_MICROFRONTEND_URL),
    lms_1             | TypeError: startswith first arg must be str or a
    tuple of str, not NoneType
2020-06-22 16:18:21 +02:00
Régis Behmo
c2e30a6855 Fix authentication in Android mobile application
Login on Android mobile app was failing with "check your username and
password".
2020-06-22 16:01:46 +02:00
Régis Behmo
92c4055b36 v10.0.6 (2020-06-22)
- [Bugfix] Fix unsent activation emails and other asynchronous tasks
2020-06-22 12:19:57 +02:00
Régis Behmo
40a76330f6 Fix unsent activation emails and other asynchronous tasks
Half of the tasks from edx.lms.core.default celery queue were being
processed by the CMS worker. Unfortunately, this CMS worker crashes on
some of those tasks. For instance, activation emails complain of a
missing "django_markup" template tag library because "xss_utils" is not
part of the installed app in the CMS.

The problem is that we need this edx.lms.core.default queue to be part
of the CELERY_QUEUES in the cms in order to send tasks from the CMS to
the LMS. The trick to resolve this situation is to ask the CMS celery
worker to not process the tasks from this queue.

To debug this issue, run in the LMS:

    from student.tasks import send_activation_email
    send_activation_email("{}")

Then watch the logs of the lms and cms workers. If the CMS workers picks
up this task (50% of the time prior to this change) then we have an
issue.

See:
https://discuss.overhang.io/t/reset-password-email-sent-but-activation-email-dont/690
2020-06-22 12:11:15 +02:00
Régis Behmo
cdf6b7e315 v10.0.5 (2020-06-21)
- [Security] Apply edx-platform upstream xss security fixes ([pull
request](https://github.com/edx/edx-platform/pull/24258))
2020-06-21 13:46:19 +02:00
Régis Behmo
1773e2a347 Apply edx-platform upstream xss security fixes 2020-06-21 11:04:46 +02:00
Régis Behmo
77a0a6f60d v10.0.4 (2020-06-19)
- [Bugfix] Fix broken `tutor ui` command in binary
2020-06-19 16:37:58 +02:00
Régis Behmo
2b0ed7daac Fix broken ui command in tutor binary
"tutor ui" was failing miserably, printing a lot of garbled characters
in the shell. In fact, a FileNotFound error was being raised and
automatically caught by the ui command. When removing the catch all,
this was the error that was raised:

    FileNotFoundError: [Errno 2] No such file or directory:
    '/tmp/_MEIimsqmq/wcwidth/version.json'     │

This is resolved on SO:
https://stackoverflow.com/questions/62155242/pyinstaller-cant-find-wcwidth-version-json-when-running-executable
2020-06-19 16:33:58 +02:00
Régis Behmo
001da4e24e v10.0.3 (2020-06-19)
- [Bugfix] Fix error on android user creation
2020-06-19 12:53:18 +02:00
Régis Behmo
2f8efbc656 Revert "Fix DoesNotExist error on android credentials creation"
This reverts commit a88d9cfc19.
2020-06-19 11:40:46 +02:00
Régis Behmo
d9aab8b314 v10.0.1 (2020-06-17)
- [Bugfix] Fix crash when viewing problem in LMS
- [Bugfix] Fix missing webpack-stats.json in openedx Docker image
2020-06-17 21:51:52 +02:00
Régis Behmo
baca27bfbd Fix crash when viewing problem in LMS
This was due to incorrectly loading the coursewarehistoryextended in the
installed applications. Also, the database router in charge of routing
requests to the student_history_module database must be disabled.
2020-06-17 11:10:37 +02:00
Régis Behmo
325c1037c0 Fix missing webpack-stats.json in openedx Docker image
In CI, the webpack-stats.json file sometimes contains just:

    {"status":"compiling"}

This was due to the fact that the `subprocess.call(...)` command in
openedx-assets did not check whether the command was killed -- for lack
of memory for instance. This is resolved by replacing "call(...)" by
"check_call(...)".
2020-06-16 06:22:12 +02:00
Régis Behmo
24035ac1fe v10.0.1 (2020-06-15)
- [Bugfix] Fix KeyError when running ``local quickstart`` for the first
time
2020-06-15 17:59:27 +02:00
Régis Behmo
06fe19fcf2 Fix KeyError when running `local quickstart` for the first time
This was due to incorrect parsing of the version number.
2020-06-15 17:57:14 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
29c6741bba Fix nginx resolver address 2020-06-14 15:47:37 +02:00
Régis Behmo
872a33414a Add --limit=myplugin option to init commands
This makes it possible to run "init" for some services only, which is
useful during debugging.
2020-06-02 14:30:06 +02:00
Régis Behmo
a6c9bba3bf v3.12.6 (2020-06-01)
- [Improvement] Add `dig`, `ping` utilities to openedx-dev Docker image
- [Bugfix] Resolve "Can't connect to MySQL server" on init
- [Improvement] Make it possible to customize the MySQL root username,
for connecting to external MySQL databases
2020-06-01 17:20:35 +02:00
Régis Behmo
092c3955c1 Add dig, ping utilities to openedx-dev docker image 2020-06-01 17:19:41 +02:00
Régis Behmo
47aaf63a59 Attempt to resolve "Can't connect to MySQL server" on init
Close #334 (hopefully)
2020-06-01 17:14:22 +02:00
Régis Behmo
3831e2e53d Make the mysql root username a configuration parameter
This allows users who connect to an external database to customise the
root username. Indeed, quite often the "root" username is reserved.

Close #328
2020-06-01 17:11:10 +02:00
Régis Behmo
0711dd462b v3.12.5 (2020-05-20)
- [Improvement] Upgrade Android app to v2.21.1 and enable many features,
such as downloading videos to SD card. Thanks for the help @ejklock!
- [Bugfix] Fix Android app crash when accessing course
2020-05-20 17:20:06 +02:00
Régis Behmo
a92ff42348 Fix and upgrade Android app to v2.21.1
Also, we introduce tons of cool features, including downloading videos
to SD card.
2020-05-20 17:19:13 +02:00
Régis Behmo
cce02dcdf6 v3.12.4 (2020-05-18)
- [Improvement] Add ability to rescore SCORM units
- [Bugfix] Fix scoring of graded SCORM units
- [Improvement] Increase maximum uploaded file size in the CMS from 10
to 100 Mb.
2020-05-18 11:35:31 +02:00
Régis Behmo
8035b7dfc6 Upgrade scorm to 9.2.0
This adds a bugfix (float graded submissions) and a new feature
(assignment rescoring).
2020-05-18 11:33:58 +02:00
Régis Behmo
b30617581d Increase maximum uploaded file size in the CMS
See discussion: https://discuss.overhang.io/t/increase-upload-size/556
2020-05-13 06:54:03 +02:00
Régis Behmo
95271092c5 v3.12.3 (2020-05-05)
- [Security] Apply most recent edx-platform [security
patches](https://discuss.openedx.org/t/security-patch-for-edit-chapter-xss-lint-issues/2030)
2020-05-05 18:58:01 +02:00
Régis Behmo
be38f70f5d Apply most recent edx-platform security patches
See this conversation:
https://discuss.openedx.org/t/security-patch-for-edit-chapter-xss-lint-issues/2030
2020-05-05 17:04:34 +02:00
Régis Behmo
c18ab3e7c6 v3.12.2 (2020-04-29)
- [Bugfix] Fix oauth2 authentication with 3rd-party services, such as discovery
- [Bugfix] Upgrade scorm xblock to solve caching issue
2020-04-29 16:15:10 +02:00
Régis Behmo
aef0ede0ec Fix oauth with services such as discovery
See https://discuss.overhang.io/t/problem-occured-during-checkout-please-contact-support/491/31
2020-04-29 15:53:01 +02:00
Régis Behmo
9043450de8 Upgrade scorm xblock to solve caching issue
See https://github.com/overhangio/openedx-scorm-xblock/issues/1
2020-04-29 11:23:27 +02:00
Régis Behmo
be6bc76b2c v3.12.1 (2020-04-27)
- [Improvement] Increase max upload allowed size to 250Mb in CMS
2020-04-27 00:27:17 +02:00
Régis Behmo
1815ab138e Increase allowed upload size for studio content
This is particularly useful for courses with large SCORM content.
2020-04-27 00:26:39 +02:00
Régis Behmo
47aad68a5c v3.12.0 (2020-04-26)
- 💥[Improvement] Do not deploy an ingress or SSL/TLS certificate issuer
ressource by default in Kubernetes
- [Improvement] Fix tls certificate generation in k8s
- 💥[Improvement] Radically change the way jobs are run: we no longer
"exec", but instead run a dedicated container.
- 💥[Improvement] Upgrade k8s certificate issuer to
cert-manager.io/v1alpha2
- [Feature] Add SCORM XBlock to default openedx docker image
2020-04-26 01:05:48 +02:00
Régis Behmo
a52884a311 Remove ingress/issuer from default k8s deployment
There are too many different ways to deploy an Ingress resource and to
generate SSL/TLS certificates: it's too much responsibility to make that
decision for the end user.
2020-04-25 23:12:42 +02:00
Régis Behmo
bce6432d85 Improve job running in local and k8s
Running jobs was previously done with "exec". This was because it
allowed us to avoid copying too much container specification information
from the docker-compose/deployments files to the jobs files. However,
this was limiting:

- In order to run a job, the corresponding container had to be running.
This was particularly painful in Kubernetes, where containers are
crashing as long as migrations are not correctly run.
- Containers in which we need to run jobs needed to be present in the
docker-compose/deployments files. This is unnecessary, for example when
mysql is disabled, or in the case of the certbot container.

Now, we create dedicated jobs files, both for local and k8s deployment.
This introduces a little redundancy, but not too much. Note that
dependent containers are not listed in the docker-compose.jobs.yml file,
so an actual platform is still supposed to be running when we launch the
jobs.

This also introduces a subtle change: now, jobs go through the container
entrypoint prior to running. This is probably a good thing, as it will
avoid forgetting about incorrect environment variables.

In k8s, we find ourselves interacting way too much with the kubectl
utility. Parsing output from the CLI is a pain. So we need to switch to
the native kubernetes client library.
2020-04-25 23:12:42 +02:00
Régis Behmo
091e45fe63 Fix tls certificate generation in k8s
The "Certificate" objects are no longer required. As a consequence, the
"k8s-ingress-certificates" has become useless and should be removed from
plugins.
2020-04-25 14:40:48 +02:00
Régis Behmo
e4ca99b237 Upgrade cert-manager installation in k8s
Note that the spec has changed for v1alpha2.
2020-04-25 14:40:48 +02:00
Régis Behmo
49c71f8af2 Add scorm xblock to the openedx docker image 2020-04-25 14:20:13 +02:00
Régis Behmo
0960449405 v3.11.12 (2020-04-16)
- [Feature] Make it easy to add custom translation strings to the
openedx Docker image
- [Improvement] Make it possible to rely on a different npm registry for
faster image building
2020-04-16 19:31:36 +02:00
Régis Behmo
33ab52bfeb Make it easy to add custom translation strings to edx-platform
Users can now add custom translation strings to a locale folder at build
time, very much in the same way as custom themes or requirements. This
is quite convenient, although is does require quite a bit of time to
rebuild the docker images.
2020-04-16 19:30:08 +02:00
Régis Behmo
76d1078382 Make it possible to pull nodejs packages from custom npm registry
During an incident at npmjs.org it was extremely difficult to pull
nodejs packages -- so we made it possible to pull from a custom
registry, deployed for instance with Verdaccio (https://verdaccio.org/).
2020-04-16 19:26:02 +02:00
Régis Behmo
bdc0cb04e2 v3.11.11 (2020-04-15)
- [Bugfix] Make sure all emails (including "password reset") are
properly saved to a local file in development mode (#315)
- [Improvement] Add `openedx-development-settings` patch to patch the
LMS and the CMS simultaneously in development
- [Bugfix] Fix missing celery tasks in the CMS
2020-04-15 23:53:05 +02:00
Régis Behmo
724c2c84da Make sure all emails are stored to a tmp file in development
In development, emails sent from edx-platform were using the
"file_email" channel from edx-ace ("edX's automated communication
engine"). This channel was failing because it tries to write to a file
located in the /edx folder, which does not exist in tutor containers. To
fix this, we configure edx-ace to rely on the django email backend,
which itself is configured to send emails to a file in development. It
turns out that this backend was also configured to store emails in a
file located in the /edx folder, so we had to add the standard
EMAIL_FILE_PATH django setting to our development settings.

It was easier to reconfigure the django file email backend than the
edx-ace file_email channel because the output path of the latter cannot
be modified by a setting.

Note that this causes all emails to be stored in local files instead of
being sent to actual recipients. This is the default behaviour in Open
edX, and indeed in most default django apps (in development). This is a
good thing! If, for some reason, developers would like to try out email
sending during development, they should modify the EMAIL_BACKEND
setting and set it to 'django.core.mail.backends.smtp.EmailBackend'.
This is quite easy to achieve with the help of a plugin:

    name: sendemailsindev
    version: 0.1.0
    patches:
      openedx-development-settings: |
          # actually send emails in dev
          EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"

Close #315
2020-04-15 23:52:28 +02:00
Régis Behmo
072c3a1a15 Add openedx-development-settings patch 2020-04-15 23:52:18 +02:00
Régis Behmo
54d9fe20e7 Fix missing celery tasks in the CMS
This should spare us quite a few error messages in the CMS worker.
2020-04-14 19:49:18 +02:00
Régis Behmo
68ff61a421 v3.11.10 (2020-04-14)
- [Bugfix] Fix updates to the course structure in the LMS (#302)
2020-04-14 19:24:27 +02:00
Régis Behmo
48ca6eb4b0 Fix updates to the course structure in the LMS
When we were changing unit titles in the CMS, the changes were taking a
long time to be reflected in the LMS. That's because the cache key that
corresponds to the course structure was not being updated. It was the
responsibility of an asynchronous LMS celery worker to update this cache
entry. However, this was impossible in most cases because tasks
triggered in the CMS were only processed by CMS workers. That is, unless
we are using a custom celery router:
https://celery.readthedocs.io/en/latest/userguide/routing.html#routers

This is what edx-platform does in the devstack: certain CMS tasks are
forwarded both to CMS and to LMS workers. This is achieved by defining
the ALTERNATE_WORKER_QUEUES="lms" django setting in the CMS.

Adding this setting to Tutor solves the problem in production. However,
in development mode Open edX runs without workers
(`CELERY_ALWAYS_EAGER=True`). This means that the course structure will
not be automatically updated when running `tutor dev` commands, which is
a shame. The alternative is to define the
"block_structure.invalidate_cache_on_publish" waffle switch. This can be
done from the UI (in /admin/waffle/switch/add/) or by running:

    tutor dev run lms ./manage.py lms waffle_switch block_structure.invalidate_cache_on_publish on --create

However, this flag seems to slow down access to the LMS for the first
user who tries to access the course after it has been updated.

Close #302
2020-04-14 19:16:52 +02:00
Régis Behmo
30f39233c4 v3.11.9 (2020-04-07)
- [Bugfix] Fix Android app build for domain names that include a dash
("-")
2020-04-07 15:06:21 +02:00
Régis Behmo
36e2f81845 Fix Android app build for domain names that include a dash
See
https://discuss.overhang.io/t/build-android-app-command-error/405/30

Android package names should be composed of alphanumeric or underscore
characters only
(https://developer.android.com/studio/build/application-id.html)
2020-04-07 15:04:34 +02:00
Régis Behmo
e42c20a133 v3.11.8 (2020-04-06)
- [Feature] Add `encrypt` template filter to conveniently add
htpasswd-based authentication to nginx
- [Bugfix] Fix "missing tty" during init in cron jobs
2020-04-06 10:40:28 +02:00
Régis Behmo
411327662e Add encrypt template filter
This is convenient for htpasswd-based authentication to nginx, for
instance.
2020-04-04 18:22:15 +02:00
Régis Behmo
04f672eed2 Fix "missing tty" during init in cron jobs 2020-04-02 12:36:43 +02:00
Régis Behmo
40d9058f3d v3.11.7 (2020-04-01)
- [Bugfix] Fix missing js translations
- [Bugfix] Fix tls certificate generation in cron jobs
2020-04-01 20:23:16 +02:00
Régis Behmo
1c22ade579 Add missing js translations to openedx
Client-side translations are stored in "djangojs.js" files. Supposedly,
these files were properly compiled prior to the Ironwood release -- but
this is not the case, so we need to re-generate them.

Also, we need to re-generate the djangojs.js files for the custom,
downloaded locales. The assets collection settings are also fixed to
take into account the separate locale folder.

This step needs to happen prior to static assets collection, as the
djangojs files are collected to the staticfiles/ folder.

See these conversations:
https://discuss.overhang.io/t/localization-not-works-perfect/363
https://discuss.openedx.org/t/localization-not-work-for-js-files/1671
2020-04-01 20:12:25 +02:00
Régis Behmo
e521333b75 Fix tls certificate generation in cron jobs
tls renewal and generation was failing in cron jobs because of "The
input device is not a TTY" errors. This is because the "-it" docker
option does not work when a tty is not available.
2020-03-27 10:05:40 +01:00
Régis Behmo
8a8f5abab8 Changelog typo 2020-03-25 17:10:07 +01:00
Régis Behmo
42d698d6e5 v3.11.6 (2020-03-13)
- [Bugfix] Fix "Unable to resolve dependency" error during forum
initialisation
- [Feature] Add `settheme` command to easily assign a theme to a domain
name
- [Improvement] Modify nginx access logs to include request scheme and
server name (plugin developers should use the "tutor" log format)
- [Bugfix] Fix DNS resolution of restarted service
- [Feature] Restart multiple services with `local restart`
- [Feature] Make it possible to easily reload openedx gunicorn process
with `tutor local exec lms reload-gunicorn``
- [Improvement] Rename lms/cms_worker to lms/cms-worker in local
deployment
- [Improvement] Add the management plugin to the rabbitmq container
- [Improvement] Make it possible to run an Elasticsearch service on
https
2020-03-13 19:27:01 +01:00
Régis Behmo
1a42e2d338 Fix "Unable to resolve dependency" error during forum init
When running "bundle exec rake search:initialize" in the forum, we were
getting the following error:

        /openedx/ruby/lib/ruby/site_ruby/2.4.0/rubygems/resolver.rb:235:in `search_for': Unable to resolve dependency: user requested 'did_you_mean (= 1.1.0)' (Gem::UnsatisfiableDependencyError)

It happens this error suddently happened because rubygems-update was not
pinned to a specific version. v3.0.4 was working and v3.1.2 was not.

As it happens, we don't need rubygems-update, so we simply get rid of it
entirely.
2020-03-13 19:27:01 +01:00
Régis Behmo
5c43d6ee25 Add "settheme" command
This makes it much easier to switch to a new theme.
2020-03-13 19:27:01 +01:00
Régis Behmo
08606ca1a6 Add scheme and server name to access logs
To do so, we had to create a new log_format.

Plugin developers are strongly encouraged to start using this log format
by adding the `access_log /var/log/nginx/access.log tutor;` directive to
their extra nginx configurations.

In order to load this log format early, the `tutor.conf` config file had
to be renamed to something early in the alphabet... (hence starting by
an underscore) Older users would face an error on nginx reload, so older
"tutor.conf" files are automatically removed on config save.
2020-03-12 12:01:41 +01:00
Régis Behmo
0cf8958e3a Fix nginx DNS resolution of restarted services
Previously, a common error when restarting e.g: the lms or the cms was
that nginx redirected to the wrong container. For instance:

    access studio.localhost
    tutor local restart lms cms
    access studio.localhost

In the second call to studio.localhost, we were frequently trying to
access the LMS, which resulted in a 400 error.

We solve this issue by setting a TTL of 10s on the nginx proxy name
resolution.

More docs:
http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
2020-03-12 09:53:18 +01:00
Régis Behmo
e2a3a2d912 Allow multiple services in "local restart" command 2020-03-12 09:52:22 +01:00
Régis Behmo
00ec6930ae Add simple gunicorn reload script to openedx image 2020-03-10 10:37:14 +01:00
Régis Behmo
0325e7ad95 Rename worker containers from x_worker to x-worker
This is more coherent with k8s.
2020-03-10 10:37:14 +01:00
Régis Behmo
c2628d2b9c Add the management plugin to the rabbitmq container
The management plugin exposes an http API that makes it possible to
monitor rabbitmq. By default, we do not expose the management dashboard.
As a consequence, the API is only usable by other internal containers.
2020-03-10 10:37:14 +01:00
Régis Behmo
702e8cca7d Make it possible to run an Elasticsearch service on https
This is useful when running elasticsearch from 3rd-party vendors.
2020-03-10 10:37:14 +01:00
Régis Behmo
479e9ca34a v3.11.5 (2020-02-27)
- [Improvement] Switch edx-platform from open-release/ironwood.2 tag to
the open-release/ironwood.master branch
- [Security] Upgrade django to 1.11.28
- [Improvement] Make it possible to configure the elasticsearch heap
size
- [Bugfix] Fix broken elasticsearch environment variables
- [Improvement] Restore more recent Android app version (#289).
2020-02-27 18:08:04 +01:00
Régis Behmo
872883e36f Switch edx-platform from ironwood.2 to ironwood.master
There are too many patches on top of ironwood.2, and it's not practical
to pull them all one by one. We still want to build on top of a specific
version, and not a branch, so we use a dirty hack to guarantee that the
docker image is properly rebuilt by CI when we change it.
2020-02-27 17:42:43 +01:00
Régis Behmo
5e626114de Security upgrade django 1.11.27 -> 1.11.28 2020-02-27 17:13:42 +01:00
Régis Behmo
83459d43d5 Fix broken elasticsearch configuration
Because we are running a version of elasticsearch older than Methusalem,
the docker environment variables were not properly taken into account.
For instance, the cluster name and "mlockall" settings were incorrect,
as we could see by running:

    $ tutor local run lms curl elasticsearch:9200 | grep cluster_name
    ...
      "cluster_name" : "elasticsearch",
    $ tutor local run lms curl elasticsearch:9200/_nodes/process?pretty | grep mlock
      ...
     "mlockall" : false

See
https://discuss.overhang.io/t/elastic-container-is-not-being-removed/312/3
for discussion.

This fix also introduces a new tutor configuration setting to adjust the
elasticsearch heap size.
2020-02-26 11:47:43 +01:00
Régis Behmo
ff3fea6930 Restore more recent Android app tagged release
A prior change used the ironwood.1 tag to build the Android app in an
attempt to solve #289. Turns out that this change was unnecessary. So
here we revert to a more recent release of the Android app. Instead of
building from the master branch (which might create suprises) we build
from a fixed release tag.

The source repo and version are customisable via build arguments.
2020-02-25 17:29:53 +01:00
Régis Behmo
e166a749e4 v3.11.4 (2020-02-16)
- [Bugfix] Fix auth problem in Android app (#289)
2020-02-16 16:00:35 +01:00
Régis Behmo
50238129a9 Downgrade android build version to ironwood.1
The android master branch was failing because of authentication errors.

Close #289
2020-02-16 16:00:35 +01:00
Régis Behmo
29ed111c7f v3.11.3 (2020-01-21)
- [Bugfix] Fix incorrectly parsed empty strings for `config save --set
...=...` commands
2020-01-21 18:32:34 +01:00
Régis Behmo
731de6a747 Fix incorrectly parsed empty strings in config save --set ...=...
When running `config save --set KEY=""`, KEY was incorrectly interpreted
as null.
2020-01-21 18:32:34 +01:00
Régis Behmo
6716668ff9 v3.11.2 (2020-01-17)
- [Bugfix] Make sure `docker-compose.override.yml` are loaded in dev and
local contexts
2020-01-17 12:12:47 +01:00
Régis Behmo
cdc221b20e Fix loading of docker-compose.override files
Override files are not automatically loaded when running with `-f`, so
we need to specify them manually.
2020-01-17 09:46:59 +01:00
Régis Behmo
6b083fee4e v3.11.1 (2020-01-16)
- [Feature] Add `config render` command
2020-01-16 23:58:20 +01:00
Régis Behmo
72e23f3f96 Add config render command
This is going to be useful for using custom themes with user-defined
variables.
2020-01-16 17:15:55 +01:00
Régis Behmo
3ca0c6cf46 v3.11.0 (2020-01-14)
- [Feature] Add support for simple, YAML-based plugins
- 💥[Improvement] The output of `plugins list` now includes plugin
version numbers
2020-01-14 17:35:51 +01:00
Régis Behmo
e3444d668c Add support for simple, yaml-based plugins
Those plugins are stored as yaml files in ~/.local/share/tutor-plugins
and follow the same specifications as entrypoint plugins.
2020-01-14 17:35:51 +01:00
Régis Behmo
159e24a95d Refactor plugin internals
This is for supporting json-based plugins. The great thing about this
change is that it allows us to easily print plugin version numbers in
`plugins list`.
2020-01-14 15:41:42 +01:00
Régis Behmo
e611b90441 3.10.1 (2020-01-13)
- [Improvement] Explicitely point to docker.io images, when necessary,
for [podman](https://podman.io/) compatibility.
2020-01-13 22:40:10 +01:00
Régis Behmo
079c5bc778 v3.10.0
- [Bugfix] Fix oauth authentication in dev mode
- [Improvement] Upgrade to the 3.7 docker-compose syntax
- [Improvement] The `dev runserver` command can now be run for just any
service
- 💥[Feature] `dev run/exec` commands now support generic options which
are passed to docker-compose. Consequently, defining the
`TUTOR_EDX_PLATFORM_PATH` environment variable no longer works. Instead,
users are encouraged to explicitely pass the `-v` option, define a
command alias or create a `docker-compose.override.yml` file.
2020-01-10 16:42:27 +01:00
Régis Behmo
54918647ee Clarify changelog regarding docker-compose.override file 2020-01-10 16:07:35 +01:00
Régis Behmo
c636a39af9 Fix oauth authentication in dev mode
This was causing issues with the discovery service in development mode.
2020-01-10 13:14:55 +01:00
Régis Behmo
899e4dfb9a Upgrade to v3.7 for docker-compose service definition
We were encountering issues when overriding service definitions with
"x-...". These were solved by upgrading docker-compose and switching to
3.7 service definition. As a consequence, the minimum supported docker
version is 18.06.0, as specified by
https://docs.docker.com/compose/compose-file/#compose-and-docker-compatibility-matrix
2020-01-10 11:10:54 +01:00
Régis Behmo
3540627691 Make it possible to run dev runserver for any service 2020-01-10 11:10:54 +01:00