6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-14 06:58:21 +00:00
Commit Graph

342 Commits

Author SHA1 Message Date
Régis Behmo
e2a3a2d912 Allow multiple services in "local restart" command 2020-03-12 09:52:22 +01:00
Régis Behmo
16819bd697 Rename *_PROTOCOL to *_SCHEME 2020-03-12 09:37:22 +01:00
Régis Behmo
00ec6930ae Add simple gunicorn reload script to openedx image 2020-03-10 10:37:14 +01:00
Régis Behmo
0325e7ad95 Rename worker containers from x_worker to x-worker
This is more coherent with k8s.
2020-03-10 10:37:14 +01:00
Régis Behmo
c2628d2b9c Add the management plugin to the rabbitmq container
The management plugin exposes an http API that makes it possible to
monitor rabbitmq. By default, we do not expose the management dashboard.
As a consequence, the API is only usable by other internal containers.
2020-03-10 10:37:14 +01:00
Régis Behmo
702e8cca7d Make it possible to run an Elasticsearch service on https
This is useful when running elasticsearch from 3rd-party vendors.
2020-03-10 10:37:14 +01:00
Claudio Acciaresi
a7eb9be2fe Fixing typo. 2020-02-27 21:49:12 +01:00
Régis Behmo
479e9ca34a v3.11.5 (2020-02-27)
- [Improvement] Switch edx-platform from open-release/ironwood.2 tag to
the open-release/ironwood.master branch
- [Security] Upgrade django to 1.11.28
- [Improvement] Make it possible to configure the elasticsearch heap
size
- [Bugfix] Fix broken elasticsearch environment variables
- [Improvement] Restore more recent Android app version (#289).
2020-02-27 18:08:04 +01:00
Régis Behmo
872883e36f Switch edx-platform from ironwood.2 to ironwood.master
There are too many patches on top of ironwood.2, and it's not practical
to pull them all one by one. We still want to build on top of a specific
version, and not a branch, so we use a dirty hack to guarantee that the
docker image is properly rebuilt by CI when we change it.
2020-02-27 17:42:43 +01:00
Régis Behmo
7eb99a3811 Minor code refactoring, for naming clarity 2020-02-27 17:14:00 +01:00
Régis Behmo
5e626114de Security upgrade django 1.11.27 -> 1.11.28 2020-02-27 17:13:42 +01:00
Régis Behmo
83459d43d5 Fix broken elasticsearch configuration
Because we are running a version of elasticsearch older than Methusalem,
the docker environment variables were not properly taken into account.
For instance, the cluster name and "mlockall" settings were incorrect,
as we could see by running:

    $ tutor local run lms curl elasticsearch:9200 | grep cluster_name
    ...
      "cluster_name" : "elasticsearch",
    $ tutor local run lms curl elasticsearch:9200/_nodes/process?pretty | grep mlock
      ...
     "mlockall" : false

See
https://discuss.overhang.io/t/elastic-container-is-not-being-removed/312/3
for discussion.

This fix also introduces a new tutor configuration setting to adjust the
elasticsearch heap size.
2020-02-26 11:47:43 +01:00
Régis Behmo
ff3fea6930 Restore more recent Android app tagged release
A prior change used the ironwood.1 tag to build the Android app in an
attempt to solve #289. Turns out that this change was unnecessary. So
here we revert to a more recent release of the Android app. Instead of
building from the master branch (which might create suprises) we build
from a fixed release tag.

The source repo and version are customisable via build arguments.
2020-02-25 17:29:53 +01:00
Régis Behmo
e166a749e4 v3.11.4 (2020-02-16)
- [Bugfix] Fix auth problem in Android app (#289)
2020-02-16 16:00:35 +01:00
Régis Behmo
50238129a9 Downgrade android build version to ironwood.1
The android master branch was failing because of authentication errors.

Close #289
2020-02-16 16:00:35 +01:00
Régis Behmo
29ed111c7f v3.11.3 (2020-01-21)
- [Bugfix] Fix incorrectly parsed empty strings for `config save --set
...=...` commands
2020-01-21 18:32:34 +01:00
Régis Behmo
731de6a747 Fix incorrectly parsed empty strings in config save --set ...=...
When running `config save --set KEY=""`, KEY was incorrectly interpreted
as null.
2020-01-21 18:32:34 +01:00
Régis Behmo
eeb5f43239 Replace @ by == in "pluginname==version" 2020-01-17 15:06:07 +01:00
Régis Behmo
6716668ff9 v3.11.2 (2020-01-17)
- [Bugfix] Make sure `docker-compose.override.yml` are loaded in dev and
local contexts
2020-01-17 12:12:47 +01:00
Régis Behmo
cdc221b20e Fix loading of docker-compose.override files
Override files are not automatically loaded when running with `-f`, so
we need to specify them manually.
2020-01-17 09:46:59 +01:00
Régis Behmo
6b083fee4e v3.11.1 (2020-01-16)
- [Feature] Add `config render` command
2020-01-16 23:58:20 +01:00
Régis Behmo
2159226b4b Fix missing partials folder in config render
"partials" folders were not rendered at all when using `config render`.
2020-01-16 23:58:20 +01:00
Régis Behmo
72e23f3f96 Add config render command
This is going to be useful for using custom themes with user-defined
variables.
2020-01-16 17:15:55 +01:00
Régis Behmo
c4c12b0ab8 env.py refactoring
Clarify a few variable names, make code more modular. Also, the Renderer
class now makes more sense as a singleton. We took the opportunity to
delete quite a lot of code.
2020-01-16 15:07:35 +01:00
Régis Behmo
3ca0c6cf46 v3.11.0 (2020-01-14)
- [Feature] Add support for simple, YAML-based plugins
- 💥[Improvement] The output of `plugins list` now includes plugin
version numbers
2020-01-14 17:35:51 +01:00
Régis Behmo
e3444d668c Add support for simple, yaml-based plugins
Those plugins are stored as yaml files in ~/.local/share/tutor-plugins
and follow the same specifications as entrypoint plugins.
2020-01-14 17:35:51 +01:00
Régis Behmo
159e24a95d Refactor plugin internals
This is for supporting json-based plugins. The great thing about this
change is that it allows us to easily print plugin version numbers in
`plugins list`.
2020-01-14 15:41:42 +01:00
Régis Behmo
c1e47cec82 Minor internal function renaming 2020-01-14 12:01:06 +01:00
Régis Behmo
e611b90441 3.10.1 (2020-01-13)
- [Improvement] Explicitely point to docker.io images, when necessary,
for [podman](https://podman.io/) compatibility.
2020-01-13 22:40:10 +01:00
Régis Behmo
e8248b26d6 Tutor docker images should point to docker.io, too 2020-01-10 17:22:01 +01:00
Régis Behmo
c03e86f43b Explicitely point to docker.io images
This is useful for tools like podman which do not take it for granted
that base images come from hub.docker.com.
2020-01-10 17:15:48 +01:00
Régis Behmo
079c5bc778 v3.10.0
- [Bugfix] Fix oauth authentication in dev mode
- [Improvement] Upgrade to the 3.7 docker-compose syntax
- [Improvement] The `dev runserver` command can now be run for just any
service
- 💥[Feature] `dev run/exec` commands now support generic options which
are passed to docker-compose. Consequently, defining the
`TUTOR_EDX_PLATFORM_PATH` environment variable no longer works. Instead,
users are encouraged to explicitely pass the `-v` option, define a
command alias or create a `docker-compose.override.yml` file.
2020-01-10 16:42:27 +01:00
Régis Behmo
c636a39af9 Fix oauth authentication in dev mode
This was causing issues with the discovery service in development mode.
2020-01-10 13:14:55 +01:00
Régis Behmo
899e4dfb9a Upgrade to v3.7 for docker-compose service definition
We were encountering issues when overriding service definitions with
"x-...". These were solved by upgrading docker-compose and switching to
3.7 service definition. As a consequence, the minimum supported docker
version is 18.06.0, as specified by
https://docs.docker.com/compose/compose-file/#compose-and-docker-compatibility-matrix
2020-01-10 11:10:54 +01:00
Régis Behmo
3540627691 Make it possible to run dev runserver for any service 2020-01-10 11:10:54 +01:00
Régis Behmo
d17fdaa658 More feature-complete dev/local docker-compose commands
By de-duplicating the code between dev.py and local.py, we are able to
support more docker-compose run/up/stop options passed from tutor. To do
so, we had to disable some features, such as automatically mounting the
edx-platform repo when the TUTOR_EDX_PLATFORM_PATH environment variable
was defined.
2020-01-10 11:10:54 +01:00
Régis Behmo
fcf2dd0d6f v3.9.1
- [Improvement] Make it possible to override the project name in development mode
- [Bugfix] Fix user switching in development mode
- [Bugfix] Fix "k8s quickstart" crash
2020-01-08 20:09:33 +01:00
Régis Behmo
f82cbd84f1 Fix crash on "k8s quickstart"
This was due to the recent context refactoring.

See https://discuss.overhang.io/t/error-with-kubernetes-deployment/240
2020-01-08 20:07:25 +01:00
Régis Behmo
f5c087cada Fix user switching in development mode
This incorrect user switching was (probably) causing permission issues,
as well as incorrect settings.
2020-01-08 19:35:57 +01:00
Régis Behmo
cc649e7378 Make it possible to override dev project name 2020-01-08 14:36:51 +01:00
Régis Behmo
49a7946d67 Simplify patch internals 2020-01-08 12:20:24 +01:00
Régis Behmo
b5ef89abed Fix --edx-platform-settings cli doc 2020-01-08 12:07:47 +01:00
Régis Behmo
5179931c0e Fix autocompletion of "dev exec" command 2020-01-07 14:33:55 +01:00
Régis Behmo
41c9bfc337 v3.9.0
- [Security] Apply django 1.11.21 -> 1.11.27 security patches
- [Bugfix] Fix running dev image with `sudo`
- [Improvement] Add `cms/lms-env-features` patches (#276)
- [Feature] Add plugin subcommands
- 💥[Improvement] Move ``-r/--root`` option to parent command level
- [Bugfix] Fix course about page visibility
- [Improvement] Print gunicorn access logs in the console
- 💥[Improvement] Get rid of the `indexcourses` and `portainer` command
(#269)
2019-12-21 16:54:57 +01:00
Régis Behmo
2ba57fb8ba Apply django 1.11.21 -> 1.11.27 security patches 2019-12-21 15:18:29 +01:00
Régis Behmo
25f98250cb Fix running dev openedx image with sudo
When running with `sudo`, the USERID image ARG is 0. The docker
entrypoint needed to take this case into account.

See:
https://discuss.overhang.io/t/openedx-is-not-the-name-of-a-known-user/224/5
2019-12-16 09:33:43 +01:00
Régis Behmo
0b1cac5134 Make it possible to run plugin subcommands 2019-12-15 18:52:58 +01:00
Régis Behmo
13de3c8adc Move "-r/--root" option to parent command level 2019-12-15 18:52:58 +01:00
Claudio Acciaresi
399114ad5b Adding the ability to specify CMS or LMS specific features. 2019-12-13 10:00:59 +01:00
Régis Behmo
494838956a Fix course about page visibility
"Course visibility In Catalog" advanced CMS setting was being ignored
because the default the COURSE_CATALOG_VISIBILITY_PERMISSION and
COURSE_ABOUT_VISIBILITY_PERMISSION settings were incorrectly set to the
legacy default, which "see_exists".

See
https://discuss.overhang.io/t/catalog-visibility-in-tutor-deployed-open-edx/214
2019-12-09 10:18:54 +01:00
Régis Behmo
2e9967879d Fix minor scripts.py formatting 2019-12-05 12:42:41 +01:00
Régis Behmo
a028bad636 Print gunicorn access logs in the console
See docs: https://docs.gunicorn.org/en/stable/settings.html#accesslog
2019-12-05 12:41:00 +01:00
Régis Behmo
afab59e17d Fix non-ascii platform names
There was an issue with non ascii platform names, such as "Курсы". This
is fixed by switching to utf8-encode settings file. That is, until we
switch to python3.

Close #270
2019-12-05 12:04:27 +01:00
Régis Behmo
302e1fc986 Get rid of the local portainer command
It makes more sense to document this command instead of adding it to the
`local` commands. If need be, in the future we should be able to re-add
it as a plugin.
2019-12-05 11:54:44 +01:00
Régis Behmo
0fb9bfe008 Get rid of the indexcourses command
This command adds a burden on the `local` and `k8s` command. It does not
make sense to provide this command out of the box, and not other
administration commands. Instead, we should better document how to run
regular `manage.py` commands from tutor.

Close #269.
2019-12-05 11:49:53 +01:00
Régis Behmo
74b3baab47 v3.8.0
- [Improvement] Add `k8s-deployments-nginx-volume-mounts` patch
- [Bugfix] Fix running forum locally when both elasticsearch and mongodb
are not activated (#266)
- [Bugfix] Fix MongoDb url in forum when running separate service (#267)
- 💥[Improvement] Better `dev` commands, with dedicated development
docker image. One of the consequences is that the `dev watchthemes`
command is replaced by `dev run lms watchthemes`.
- [Improvement] `images` commands now accept multiple `image` arguments
2019-11-22 13:01:42 +01:00
Régis Behmo
07aa0e47f2 Add k8s-deployments-nginx-volume-mounts plugin patch
This is for mounting discovery/ecommerce static asset folders in nginx.
2019-11-22 11:54:49 +01:00
Régis Behmo
fbafd1c301 Fix incorrect django settings during CMS init 2019-11-22 09:32:50 +01:00
Régis Behmo
c89a5d3c3a Minor black formatting issue 2019-11-22 09:23:59 +01:00
Régis Behmo
993694909a Fix forum depends_on when data services are disabled
When both mongodb and elasticsearch were not activated, there was a
syntax error in the local docker-compose.yml file.

Close #266
2019-11-22 09:21:59 +01:00
Régis Behmo
475df37729 Fix Mongodb url in forum when running separate service
MONGOHQ_URL was not properly set when MONGODB_HOST/PORT was customised.
That's because the environment variable was being defined in the
Dockerfile, and not at runtime.

Close #267
2019-11-22 08:56:30 +01:00
Régis Behmo
d19bd53b2b Fix missing bundle static assets in dev mode
The static assets need to be properly collected in the development image
in order to have a working webpack-stats.json file.

The /openedx/staticfiles folder is still bind-mounted in the docker
image in order to have a working `watchthemes` command.

This is for issue #235, again.
2019-11-14 12:06:17 +01:00
Régis Behmo
1f08006e90 Dynamic openedx settings configuration
`tutor.production` should not be hardcoded in the init scripts. Instead,
we load the `$SERVICE_VARIANT.envs.$SETTINGS` settings.

This was inspired by a conversation on slack:
https://openedx.slack.com/archives/CGE253B7V/p1573561290003700?thread_ts=1573478499.002800&cid=CGE253B7V

Thanks @cacciaresi!
2019-11-14 09:27:28 +01:00
Régis Behmo
df46f8579b Clarify dev setting variable name 2019-11-07 09:29:37 +01:00
Régis Behmo
a2cb012e65 Fix dev user creation when running root on host 2019-10-25 10:38:15 +02:00
Régis Behmo
5e9d405a91 Fix dev/images formatting 2019-10-24 21:34:14 +02:00
Régis Behmo
b01f4d9c0e Better dev environment
The `dev` commands now rely on a different openedx-dev docker image.
This gives us multiple improvements:

- no more chown in base image
- faster chown in development
- mounted requirements volume in development
- fix static assets issues
- bundled ipdb/vim/... packages, which are convenient for development

Close #235
2019-10-24 20:03:36 +02:00
Régis Behmo
6a6750d04b Beautify openedx-assets script 2019-10-22 19:39:23 +02:00
Régis Behmo
a3f39b3526 v3.7.4 (2019-10-19)
- [Bugfix] Fix missing requirements file in pypi package (#261)
- [Improvement] Add missing cms/lms production/development setting
patches
- [Improvement] Allow SigV4 authentication for video upload to S3
- [Bugfix] Fix cms development settings
2019-10-19 19:13:57 +02:00
Régis Behmo
ed0cf8d99f Fix incorrect lms settings patch 2019-10-18 18:19:38 +02:00
Régis Behmo
38ed7d4985 Add missing cms/lms production/development setting patches 2019-10-18 18:19:38 +02:00
Régis Behmo
9cc4074f53 Support SigV4 authentication for video upload 2019-10-18 18:19:38 +02:00
Régis Behmo
16e2e8fcb9 Do not render partials/common directory
Subdirectories of partials/ folder are rendered, so we need to move the
common settings files to the partials/ folder.
2019-10-18 14:55:36 +02:00
Régis Behmo
11f7385276 Fix cms development settings
LMS settings were being incorrectly included in the cms dev settings.
2019-10-18 11:36:07 +02:00
Régis Behmo
1c9111198f 3.7.3 (2019-10-09)
- [Improvement] Upgrade openedx locales to Ironwood
2019-10-17 17:23:16 +02:00
Régis Behmo
e539f31504 Simplify common openedx settings management
Use "{% include %}" jinja directive instead of python "exec", which was
a bit weird.

Also, this sets the technical email for the studio, which was previous
set at "technical@example.com".
2019-10-17 10:22:04 +02:00
Régis Behmo
eec60dd248 Upgrade openedx locales to Ironwood 2019-10-09 15:50:32 +02:00
Régis Behmo
8918724021 v3.7.2 (2019-10-09)
- [Improvement] Properly set studio title (#246)
- [Improvement] Enable Mysql strict mode
2019-10-09 13:40:18 +02:00
Régis Behmo
73277059bc Properly set studio title
This title is visible in the browser title bar and the main page of the
CMS.

Close #246.
2019-10-09 12:04:15 +02:00
Régis Behmo
81dafa44ea k8s support has long been out of beta 2019-10-08 22:25:31 +02:00
Régis Behmo
cb0e2c509d De-duplicate asset settings 2019-10-07 16:39:24 +02:00
Régis Behmo
56850683b3 Get rid of mysql strict trans mode warnings
Here, we adopt the same behaviour as edX.org.

See the following links for documentation:
https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-sql-mode
https://dev.mysql.com/doc/refman/en/sql-mode.html#sql-mode-strict
2019-10-07 16:39:24 +02:00
Régis Behmo
1d4826dc3d v3.7.1
- [Feature] Introduce tutor docker image
- [Feature] Introduce `local hook` command for plugin development.
- [Bugfix] Persist `private.txt` file between two runs of `config save`.
(#247)
- [Improvement] Added configuration values to limit the number of
gunicorn workers for the LMS and CMS.
2019-10-07 14:40:49 +02:00
Régis Behmo
762e7984c2 Introduce local hook command 2019-10-04 11:44:24 +02:00
Régis Behmo
74d50e52c3 Persist private.txt between two runs of "config save"
Close #247.
2019-09-23 11:32:10 +02:00
Régis Behmo
8659b6e7ac Added config values for #gunicorn workers 2019-09-19 15:39:18 +02:00
Régis Behmo
a2f072faa9 v3.7.0 (2019-09-03)
- 💥[Improvement] Get rid of mysql-client container
- [Improvement] Add "local-docker-compose-lms/cms-dependencies" plugin
patches
- [Improvement] Use "exec" instead of "run" to initialise local platform
2019-09-03 11:12:10 +02:00
Régis Behmo
0c0db78310 Get rid of mysql-client container
This has an impact on plugin hooks. Plugin hooks that needed to run
inside mysql-client now need to run inside mysql container. This
simplifies the deployment, as we no longer have an empty mysql-client
container sitting around.

When mysql is not enabled (ACTIVATE_MYSQL=False) the mysql container is
simply a mysql client.
2019-09-03 10:36:10 +02:00
Régis Behmo
14dd9e54a9 Add lms/cms dependency patches
This makes it possible to declare that the lms/cms depend on minio.
2019-09-03 09:35:55 +02:00
Régis Behmo
f5c16f81b0 Nicer sleep commands for mysql-client container 2019-09-03 09:34:44 +02:00
Régis Behmo
8548e9e347 Simpler forum init script 2019-09-03 09:34:13 +02:00
Régis Behmo
5a24056b8e Switch from "run" to "exec" for local initialisation
We ran into an issue when trying to run migrations when the MinIO plugin
is activated. As seen in issues #243 and #244, the
certificates.0003_data__default_modes migration requires access to
MinIO. To do so, the MinIO host must be reached. That means that SSL
certificates must be in place (if https is enabled) and that the nginx
server must be booted. However, it does not make sense to require that
the minio container depends on the nginx container. So, in effect, we
need a fully working platform to run migrations.

In a sense, this is better as it harmonises the init task with k8s: in
k8s, init was already run with exec.

Next step is to get rid of these ugly mysql-client/minio-client
containers that must be up at all times. It would be much simpler to
just exec the commands inside the mysql/minio containers.
2019-09-03 09:29:38 +02:00
Régis Behmo
88d94bcc24 v3.6.3 (2019-08-31)
- [Security] Fix CustomTagModule mako template injection
- [Improvement] Move all plugins outside of the tutor repo
- [Bugfix/Improvement] Add all plugins (with data) into binary bundle
(#242)
2019-08-31 13:41:11 +02:00
Régis Behmo
bcf1ffe556 Fix CustomTagModule mako template injection
See announcement: https://groups.google.com/forum/#!topic/openedx-ops/aVHomKimstU
2019-08-31 12:13:19 +02:00
Régis Behmo
c9adf68ba3 Avoid reliance on __file__
"__file__" does not work with some bundle compilers, such as pyoxidizer.
Also, it's not the recommended way to proceed. See
https://pyoxidizer.readthedocs.io/en/latest/packaging_pitfalls.html#reliance-on-file
2019-08-20 17:04:18 +02:00
Régis Behmo
4d7ec486f3 v3.6.2 (2019-08-07)
- [Bugfix] Fix missing templates in bundled plugins
- [Bugfix] Enable html certificate view
2019-08-07 14:27:31 +02:00
Régis Behmo
91e357b460 Enable HTML certificates by default
See this discussion
https://discuss.overhang.io/t/how-to-generate-a-certificate-for-students/100/3
2019-07-31 13:11:22 +02:00
Régis Behmo
2a883603c8 v3.6.1
- [Bugfix] Fix missing patches from minio plugin (thanks @Wejie!)
2019-07-26 23:22:46 +02:00
Régis Behmo
f93f62a3a9 v3.6.0 (2019-07-11)
- [Feature] Modify ``createuser`` commands to define a password from the
command line
- [Improvement] Better yaml value parsing from command line
- [Feature] Add `dev exec` command
- [Bugfix] Fix incorrect notes settings definition
- [Improvement] Make it possible to start/stop/reboot a selection of
services
- [Improvement] Add `local/k8s reboot` commands
- [Improvement] Add `-U/--unset` option to `config save`
- [Bugfix] Fix insecure static asset loading when web proxy is enabled
- [Improvement] Rename `SECRET_KEY` configuration parameter to
`OPENEDX_SECRET_KEY`
- [Improvement] Add support for SSL and TLS in external SMTP server
(#231)
- [Bugfix] Fix missing video transcripts in LMS (#229)
- [Improvement] Make it possible to enable/disable multiple plugins at
once
- [Improvement] Add a few local and k8s patches for plugins
2019-07-11 14:26:51 +08:00
Régis Behmo
d9a18790d3 Define a password from the CLI on user creation
Add a `-p/--password` option to `createuser` commands.
2019-07-11 11:56:18 +08:00