6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-10 23:30:59 +00:00
Commit Graph

1185 Commits

Author SHA1 Message Date
Régis Behmo
72cf5fe30e fix: mongodb upgrade during koa -> lilac migration
When upgrading mongodb, the mongodb container takes a little while to become
ready. Running the "exec" command thus triggers an error:

    docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setF
    eatureCompatibilityVersion: "4.0" })
    MongoDB shell version v4.0.24
    connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
    2021-06-14T10:53:21.510+0000 E QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused:
    connect@src/mongo/shell/mongo.js:356:17
    @(connect):2:6
    exception: connect failed
    Error: Command failed with status 1: docker-compose -f /path/to/env/local/docker-compose.yml -f /path/to/env/local/docker-compose.prod.yml --project-name tutor_local exec mongodb mongo --eval db.adminCommand({ setFeatureCompatibilityVersion: "4.0" })

We add a "sleep" statement to the upgrade process to ensure that the mongodb
container is available.
2021-09-21 09:18:10 +02:00
Régis Behmo
9754719277 v12.1.2 2021-09-18 10:27:03 +02:00
Régis Behmo
bf5d2b80d8 fix: forum connection to mongodb (again)
Turns out, the authentication mechanism should only be defined if there
is an actual authentication. For now, because of the urgency, we
hardcode this auth_mech to ":scram". We'll add a way to override it if
necessary, in the future.
2021-09-18 10:19:02 +02:00
Régis Behmo
9263add948 v12.1.1 (2021-09-17)
- [Bugfix] Fix forum starting issue: "NoMethodError: undefined method
`encode' for nil:NilClass".
2021-09-17 16:52:34 +02:00
Régis Behmo
22d42cea45 fix: Revert "Remove MONGOID_AUTH_MECH setting from forum Dockerfile."
This reverts commit 750bdca04d.

The MONGOID_AUTH_MECH environment variable is actually necessary and
required by the ruby gem. Thus, we should not remove it, but keep it to
its default value.

See: https://github.com/overhangio/tutor/pull/493#issuecomment-921758422
https://discuss.overhang.io/t/edx-forum-mongodb-authentication/927/18
2021-09-17 16:51:24 +02:00
Régis Behmo
922c8bb373 v12.1.0 (2021-09-17)
- [Improvement] Make it easier to run edx-platform unit tests.
- [Bugfix] Fix segmentation fault during `tutor config save` on Mac OS M1 (#473). Thanks @ghassanmas!
- [Bugfix] Fix a bug that prevented connecting to external MongoDB instances.
- [Improvement] Make sure that the logo included in email notifications (including discussion responses) is the same as the site logo.
- [Bugfix] Install IPython directly from pypi instead of installing it from source (the reason it was installed from source is no longer relevant). The effect of this shall speed up the process of building the openedx-dev Docker image.
- [Feature] Add "openedx-dockerfile-post-git-checkout" patch.
- [Improvement] In the "openedx" Docker images, convert git patches to cherry-picks for a cleaner source tree.
- 💥[Feature] Make it possible to override local job configuration. This deprecates the older model for running jobs which dates back from a long time ago.
2021-09-17 10:52:15 +02:00
Régis Behmo
010279a0d9 feat: make it easier to run edx-platform unit tests
It should be unnecessary to build a custom openedx-dev Docker image. All tests
can run from within the dev Docker image, with a couple additional environment
variables.
2021-09-17 10:48:00 +02:00
Régis Behmo
b651ebc3c4 fix: segmentation fault on Mac OS M1 (Big Sur)
On Mac OS M1 (ARM processor), when the GMP library is available, pycryptodome
crashes with a segmentation fault during the generation of the RSA keys.

Upstream issue: https://github.com/Legrandin/pycryptodome/issues/506
Upstream fix: https://github.com/Legrandin/pycryptodome/pull/541

Close #473.
2021-09-17 10:44:52 +02:00
Régis Behmo
eef3c15c17 improvement: correctly set email logo
Previously, the logo included in emails was loaded from edX' CDN. Here, we make
sure that the logo is actually the same as the site logo. Because the logo may
be theme-specific, we need to compute the logo url at runtime, and use a
lazily-evaluated string.

Close #447.
2021-09-17 10:39:34 +02:00
Andrés González
750bdca04d Remove MONGOID_AUTH_MECH setting from forum Dockerfile. 2021-09-16 22:06:09 +02:00
Ghassan Maslamani
b1e91e3626 BugFix: fixes slow installation close #488
Change the source of installation Ipyton, install it from PyPy instead of source, which shall speed up the process of building the docker image for openedx-dev
2021-09-16 08:16:18 +02:00
Régis Behmo
6d18ccc632 feat: add "openedx-dockerfile-post-git-checkout" patch
This will be convenient for plugins which need to patch edx-platform.
2021-09-14 17:49:02 +02:00
Régis Behmo
ca3aabedce feat: cleaner git tree in openedx Docker image
With "git patch", the resulting source tree was dirty, showing uncommitted
changes. Here, we replace "git patch" with "git cherry-pick". We avoid pulling
the entire remote repo by fetching individual commits. To do that, we need to
assign an identity to the git user.
2021-09-14 17:49:02 +02:00
Régis Behmo
d3bfc4fb87 feat: make it possible to override local docker-compose job
Previously, it was possible to override docker-compose services, but not jobs. This requirement has appeared because some people need to override project-wide MTU settings.

See: https://discuss.overhang.io/t/problem-fetching-saml-idp-metadata/1330/23
2021-09-09 09:03:37 +02:00
Régis Behmo
1411e5a0ff docs: using docker-compose.override.yml
This feature was previously undocumented.

See: https://discuss.overhang.io/t/override-and-docker-compose/1857
2021-09-09 09:03:37 +02:00
Michael Wheeler
f5650d8a29 Fix third-party auth example 2021-09-07 10:53:29 +02:00
Régis Behmo
4812bff8bd docs: fix links kubectl docs
Previous links resulted in 404.

Close #479.
2021-09-06 12:40:50 +02:00
Régis Behmo
ecf26fccda v12.0.4 (2021-08-12)
- [Security] Apply security patch [28442](https://github.com/edx/edx-platform/pull/28442).
2021-08-12 16:47:57 +02:00
Régis Behmo
c33bd7c892 Apply security patch #28442
See: https://discuss.openedx.org/t/security-patch-for-xxe-vulnerability-in-course-import/5575
2021-08-12 16:47:12 +02:00
Régis Behmo
eeb08f63d9 v12.0.3 (2021-08-10)
- [Improvement] Upgrade all services to open-release/lilac.2.
- [Bugfix] Fix "`sh` is not a recognized command" in some plugins, including minio.
- [Improvement] Set the default contact mailing email address
- [Bugfix] Fix minio initialisation in Kubernetes.
2021-08-10 12:30:06 +02:00
Régis Behmo
41b65b80d5 Upgrade to open-release/lilac.2 2021-08-10 12:26:30 +02:00
Sofiane Bebert
4bf61d6d35 fix: set default CONTACT_MAILING_ADDRESS 2021-08-10 10:46:38 +02:00
Régis Behmo
f1773c0ba9 Revert "fix: empty entrypoints in docker-compose=2.0.0.beta4"
This reverts commit 6f04223d01.

It turns out that we cannot ignore "command: []" statements in k8s manifests.
That's because there is no way to clear entrypoint in k8s manifests, but
sometimes we do need to bypass the entrypoints. For instance, the minio client
container sets "mc" as the default entrypoint, which does not work with our job
logic.

As a consequence, Tutor becomes incompatible with docker-compose 2.0.0.beta4.
The "entrypoint must be a string" error is actually an upstream bug:
https://github.com/docker/compose-cli/issues/1848

See:

The corresponding minio issue: https://github.com/overhangio/tutor-minio/issues/9
The previous conversation about empty entrypoints: https://discuss.overhang.io/t/undefined-entrypoint-throws-error-in-docker-compose-2-0-0-beta-4/1716
2021-08-10 09:16:55 +02:00
Régis Behmo
c678638ea2 v12.0.2 (2021-07-06)
- [Bugfix] Fix "Invalid command argument" during upgrade from Koa to Lilac.
- [Bugfix] Fix mysql initialisation in docker-compose==2.0.0beta4.
- [Improvement] Tutor is now published on pypi as "tutor".
2021-07-06 09:50:11 +02:00
Régis Behmo
8be574aac8 fix: make sure that tutor-openedx is an empty package
Previously, the tutor-openedx package was loading tons of template data from
the MANIFEST.in. Turns out, we cannot ignore the MANIFEST.in file with
setuptools. So we need to move tutor-openedx to a separate, dedicated folder.
To auto-discover the package version, we copy it at runtime (in the make
command).
2021-07-06 09:42:57 +02:00
Crist Ye
db5852e9c4 fix: mongodb upgrade error when upgrading from koa
ref: https://discuss.overhang.io/t/mongo-db-error-warning-on-koa-lilac-upgrade/1744

Close #469
2021-07-06 08:42:13 +02:00
Régis Behmo
4f8f0fe006 fix: remote sync 2021-07-05 10:26:18 +02:00
Régis Behmo
6f04223d01 fix: empty entrypoints in docker-compose=2.0.0.beta4
An issue with the latest release of docker-compose was reported here:
https://discuss.overhang.io/t/undefined-entrypoint-throws-error-in-docker-compose-2-0-0-beta-4/1716

The mysql-job definition had an empty entrypoint (`[]`). This was causing the following error:

    the initiation of mysql fails with “services.mysql-job.entrypoint must be a string …
    Error: Command failed with status 15”

I can't remember at all why we had to define an empty entrypoint. It probably
has to do with the fact that we could not run `sh -e -c "..."` commands in
mysql jobs. Similarly, the k8s job definition sets `command: []`. I tested both
local and k8s deployments without these definitions and they work just fine. So
I guess we can get rid of them.
2021-07-05 10:01:37 +02:00
Régis Behmo
665905037c fix: docs instructions to enable shell completion
Instructions actually changed after the upgrade to Click 0.8.

See:
https://click.palletsprojects.com/en/8.0.x/changes/?highlight=completion#version-8-0-0
https://click.palletsprojects.com/en/8.0.x/shell-completion/

Close #457.
2021-07-03 11:08:43 +02:00
Régis Behmo
8cb402e0fb fix: incompatible deprecated click versions
Tutor breaks on very old (2014) versions of click.

See: https://discuss.overhang.io/t/prompt-got-an-unexpected-keyword-argument-show-choices/1725
2021-07-03 11:08:27 +02:00
Régis Behmo
14d8276529 chore: upgrade pinned requirements
The dev requirements had to be updated to take into account the fact that some
type annotations have been moved to dedicated packages.

See the related packages:
https://pypi.org/project/types-setuptools/
https://pypi.org/project/types-PyYAML/

And the corresponding parent project:
https://github.com/python/typeshed
2021-07-03 11:08:27 +02:00
Régis Behmo
c15eec53dc feat: switch the package name from "tutor-openedx" to "tutor"
The package maintainer of the "tutor" package was kind enough to
transfer ownership of the project to us. This is great, because we no
longer have to use the "openedx" suffix, which is trademarked.

For the time being, we keep maintaining the "tutor-openedx" package
which has a 1-to-1 dependency on the "tutor" package. In the future, we
expect that we will no longer push upgrades to tutor-openedx.
2021-07-03 11:07:37 +02:00
Régis Behmo
b6aefdab7d docs: advertise Cairn and the maintainers group
Here we add to the docs a few shameless plugs about Cairn -- because
it's really awesome!
We also add a few improvements to the wording, here and there.
2021-07-03 11:07:37 +02:00
Régis Behmo
b8729d4478 fix: don't refer to obsolete proxy config in docs
nginx/openedx.conf was a configuration file provided to configure
proxies on the host. This file no longer ships with Tutor since v11.0.0.

See: https://discuss.overhang.io/t/local-deployment-webproxy/1688
2021-07-03 11:07:22 +02:00
ChrisChV
b4ee6760eb docs: Forum config section added in Configuration and customisation 2021-06-23 00:51:10 +02:00
Régis Behmo
5cc64bf9ee v12.0.1 (2021-06-22)
- [Bugfix] Fix double pulling mongodb image when upgrading from Koa to Lilac.
- [Improvement] Better logging during `plugins disable`.
- [Bugfix] Fix "upstream sent too big header" error during login of existing users after a Koa to Lilac upgrade.
- [Feature] Added the ability to skip `config.yml` file modification while running `tutor config save` command with `-e` or `--env-only` flag.
- [Feature] Add new config value `FORUM_MONGODB_DATABASE` to set the forum database name
2021-06-22 12:47:18 +02:00
ChrisChV
d6af5ef55b feat: FORUM_MONGODB_DATABASE config val added to fix the hardcoded forum database name
In config.yml the new value FORUM_MONGO_DB_DATABASE was added with `cs_comments_service` as default value.
In docker-entrypoint.sh of forum I changed the hardcoded `cs_commecnts_service` with the new config value.
Multiple .yml files changed to handle the new config value.
2021-06-22 12:38:23 +02:00
Régis Behmo
ef189e7f67 fix: better logging during plugins disable
When disable a plugin that set config entried, such as the minio plugin, tutor was logging the following:

    Disabling plugin minio...
        Removed config entry OPENEDX_AWS_ACCESS_KEY=openedx
        Removed config entry OPENEDX_AWS_SECRET_ACCESS_KEY={{ MINIO_AWS_SECRET_ACCESS_KEY }}
        Plugin disabled

The config values were not rendered during printing, which is a shame, because
the whole point of this log line is to warn users of passwords/secrets that are
being removed. Here, we make sure that the config values are properly rendered.
The new logs are now:

    Disabling plugin minio...
        Removing config entry OPENEDX_AWS_ACCESS_KEY=openedx
        Removing config entry OPENEDX_AWS_SECRET_ACCESS_KEY=64vpCVLxhDxBuNjakSrX4CQg
        Plugin disabled
2021-06-22 12:28:58 +02:00
Crist
d49d166383 Fix: double pulling mongodb image when upgrading from Koa to Lilac. ref: https://discuss.overhang.io/t/double-pulling-mongo-images-when-upgrading-from-koa-to-lilac/1625 2021-06-22 12:28:26 +02:00
Shimul Chowdhury
31a41f81c3 feat: env only mode for config generation
chore: added -e flag to the changelog
2021-06-22 12:25:29 +02:00
Régis Behmo
e7d3e722ce fix: "upstream sent too big header" nginx errors
It was reported that existing users could not sign-in after a Koa -> Lilac
upgrade. This is due to the fact that cookies exceed the 4kb limit, which is
rejected by nginx. The fix consists in increasing this limit to 8kb:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size

This issue was reported here: https://discuss.overhang.io/t/lilac-upgrade-login-issues/1617
Note that I was not able to reproduce the issue. The fix was tested by the
issue reporter.
2021-06-22 08:58:50 +02:00
Régis Behmo
775a20b156 ci: fix sync with private repository 2021-06-14 08:16:40 +02:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
915551268c v11.3.1 (2021-06-08)
- [Improvement] Avoid permission issues in Kubernetes/Openshift for users who do not have the rights to edit their namespace.
- [Improvement] Better Kubernetes object creation.
2021-06-08 20:17:44 +02:00
Régis Behmo
4a451844e6 fix: avoid namespace edition for k8s users without access rights
In most cases, it makes very little sense to edit the namespace that an
application is running in. Quite often, users are granted access to just one
namespace and don't have the necessary rights to edit the namespace -- and for
good security reasons. In such cases, the k8s namespace object already exists
and there is no need for the user to edit or create it. Here, what we do is
that we create the namespace only if it does not exist. This should solve quite
a few permission issues, notably for Openshift users.
2021-06-08 19:54:09 +02:00
Régis Behmo
9b6b770dd7 feat: better object creation in "k8s start"
When running "tutor k8s start", it's unnecessary to attempt to create volumes
and namespace multiple times.
2021-06-08 19:54:09 +02:00
Régis Behmo
900572fdae v11.3.0 (2021-05-18)
- 💥[Security] Disable python-evaluated input by default as we don't run codejail.
- [Bugfix] Fix missing discussion notifications.
- [Improvement] Better error logging when loading a plugin from an incompatible version.
2021-06-06 14:40:24 +02:00
Régis Behmo
e990291d16 feat: upgrade pinned requirements to click 8+
We were forced to pin click to < v8 because of missing dependencies. In
particular, click_repl was broken. This is no longer the case, as click_repl
0.20 was published. Also, Jinja2 now includes type annotations, which allows us
to get rid of a few "# type: ignore" statements.

We take the opportunity to upgrade all requirements, which allows us resolve a
security issue on urllib3<1.26.0.
2021-06-06 14:38:59 +02:00
Régis Behmo
a79d194bbc security: disable python-evaluated input by default
As described in issue #284, tutor does not come with codejail enabled out of
the box. Actually, we don't even have a working plugin, yet. To prevent users
from running unsafe code, we explicitely disable python-evaluated input by
disabling the "python" interpreter. This might break some courses; thus, this
is a non-backward compatible change.
2021-06-06 14:37:46 +02:00
liupengcheng
58c77ad3fb fix: discussion email notification not work.
Discussion email notification relies on edx_ace and celery routing
key for ace is not set properly by default, which causes the send
mail task is never consumed.

Close #439.
2021-05-25 10:00:45 +02:00