6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-12 22:27:47 +00:00
Commit Graph

1334 Commits

Author SHA1 Message Date
Régis Behmo
85a44213b7 docs: improve changelog to highlight 3rd party contributions
With this change, we want to better highlight the contributions of
developers to Tutor. We want to publicly acknowledge the positive impact
that individuals and companies have on the development of the platform.
to that end, each changelog entry can now be suffixed with the name of
the author (individual or company) who authored the change. These names
will find their way to the release notes for every release. Eventually,
we also want to spread these release notes more widely. For instance, we
could post new releases to the forum to notify the community of
important changes.

If you have contributed to Tutor in the past, feel free to open a PR and
append your name to the changes that you made. We will not be able to
update the release notes for every release out there, but your
contributions will be acknowledged from the changelog.
2022-05-06 13:58:59 +01:00
Régis Behmo
3290def1ef fix: broken file upload in studio
see discussion: https://discuss.overhang.io/t/missing-js-css-files-missing-from-openedx-docker-image-in-studio/2629
see pull request: https://github.com/openedx/edx-platform/pull/30309
(thanks @uetuluk!)
2022-05-06 13:51:07 +01:00
Régis Behmo
1a4c904d7f fix: formatting 2022-04-26 14:21:18 +01:00
Régis Behmo
514e3fce22 fix: compose error on "dev --mount=learning:..."
When mounting a directory in a dev-only container, such as the
"learning" mfe, docker-compose is failing because it is attempting to
run "docker-compose stop" in the local context -- which knows nothing
about the learning container.

To resolve this, we store tmp volumes either in the local or dev
docker-compose.yml, and load either one depending on the context.
2022-04-26 14:11:26 +01:00
Régis Behmo
75ed5b4609 fix: cms settings in development
See:
https://discuss.overhang.io/t/lms-and-cms-containers-both-run-lms-in-v13-2-0-dev/2693/2
2022-04-25 12:50:05 +02:00
Régis Behmo
5eec11f70b feat: minor warning formatting change 2022-04-25 08:06:01 +02:00
Régis Behmo
db455a1379 v13.2.0 (2022-04-24)
- [Improvement] Add the `COMPOSE_PROJECT_STARTED` action and run `dev
  stop` on `local start` (and vice versa).
- [Feature] Introduce `local/dev copyfrom` command to copy contents from
  a container.
- [Bugfix] Fix a race condition that could prevent a newly provisioned
  LMS container from starting due to a `FileExistsError` when creating
data folders.
- [Deprecation] Mark `tutor dev runserver` as deprecated in favor of
  `tutor dev start`. Since `start` now supports bind-mounting and
breakpoint debugging, `runserver` is redundant and will be removed in a
future release.
- [Improvement] Allow breakpoint debugging when attached to a service
  via `tutor dev start SERVICE`.
- [Security] Apply rate limiting security fix (see
  [commit](b5723e416e)).
- [Feature] Introduce the ``-m/--mount`` option in ``local`` and ``dev``
  commands to auto-magically bind-mount folders from the host.
- [Feature] Add `tutor dev quickstart` command, which is similar to
  `tutor local quickstart`, except that it uses dev containers instead
of local production ones and includes some other small differences for
the convience of Open edX developers. This should remove some friction
from the Open edX development setup process, which previously required
that users provision using local producation containers (`tutor local
quickstart`) but then stop them and switch to dev containers (`tutor
local stop && tutor dev start -d`).
- 💥[Improvement] Make it possible to run `tutor k8s exec <command with
  multiple arguments>` (#636). As a consequence, it is no longer
possible to run quoted commands: `tutor k8s exec "<some command>"`.
Instead, you should remove the quotes: `tutor k8s exec <some command>`.
- 💥[Deprecation] Drop support for the `TUTOR_EDX_PLATFORM_SETTINGS`
  environment variable. It is now recommended to create a plugin
instead.
- 💥[Improvement] Complete overhaul of the plugin extension mechanism.
  Tutor now has a hook-based Python API: actions can be triggered at
different points of the application life cycle and data can be modified
thanks to custom filters. The v0 plugin API is still supported, for
backward compatibility, but plugin developers are encouraged to migrate
their plugins to the new API. See the new plugin tutorial for more
information.
- [Improvement] Improved the output of `tutor plugins list`.
- [Feature] Add `tutor [dev|local|k8s] status` command, which provides
  basic information about the platform's status.
2022-04-24 15:09:16 +02:00
Régis Behmo
d3e079320b ci: fix release description format 2022-04-24 15:09:16 +02:00
Régis Behmo
51e93941e0 fix: v1 plugins in binary bundle 2022-04-24 15:09:16 +02:00
Régis Behmo
e15c107570 ci: attempt to fix github release CI
Github release CI was running on ubuntu 18.04 withh python 3.6.
Installing tomli==2.0.1, which is required in dev, triggers a failure in
python 3.6 because it is no longer available.
2022-04-24 15:09:16 +02:00
Régis Behmo
b6ec87d17c fix: copyfrom unit test on python 3.7
`call_args.args` was only introduced in Python 3.8. This is breaking CI.
2022-04-24 10:39:51 +02:00
Régis Behmo
7d20329894 feat: local stop on dev start (and vice versa)
Running `local start` while a dev platform is still running is a common sourse
of mistakes. Here we introduce a new action to automatically stop local and dev
projects whenever a project with a different name is started.
2022-04-24 09:56:12 +02:00
Régis Behmo
27449f4068 feat: add dev/local copyfrom commands
`copyfrom` copies data from a container to the local filesystem. It's similar
to bindmount, but less clunky, and more intuitive. Also, it plays along great
with `--mount`. Eventually we'll just get rid of the `bindmount` command and
the `--volume` option.
2022-04-24 09:51:46 +02:00
Braden MacDonald
fde20f0e8a fix: race condition could cause mkdirs() to fail with "dir exists" 2022-04-23 18:11:32 +02:00
Kyle McCormick
99a3da809e fix: in runserver, only recommend --mount once per invocation
Previously, `tutor dev runserver --volume=x --volume=y`
would log:

  'runserver' is deprecated and will be removed in a future release.
  Use 'start' instead. Bind-mounts can be specified using '-m/--mount'.
  Bind-mounts can be specified using '-m/--mount'.
2022-04-22 10:14:11 +02:00
Kyle McCormick
126c6062e5 fix: --mount should accept service names with hyphens
(no changelog entry, as this fixes a feature that hasn't
 yet been released)
2022-04-22 10:14:11 +02:00
Kyle McCormick
5a81b6a6cb feat: deprecate runserver in favor of start
`tutor dev runserver` will be removed in a future release.
Developers are encouraged to use `tutor dev start` instead,
which is more flexible and provides a consistent interface
with `tutor local start`.

As part of this deprecation, we enable the `tty` and
`stdin_open` options on development docker-compose
services. This will allow developers to use `start`
for breakpoint debugging, which was previously only
availble via `runserver`. Several parallel PRs have
been merged in order to make the same change in the
development services of the official plugins.

Although `start` does not support the `--volume` option,
it supports a more-powerful `--mount` option. So, where
developers previously used:

    tutor dev runserver --volume ...

to bind-mount host directories, they should now use:

    tutor dev start --mount ...

Resolves https://github.com/overhangio/2u-tutor-adoption/issues/61
2022-04-21 15:00:54 +02:00
Régis Behmo
82c9dbc1eb fix: TypeError on local quickstart 2022-04-20 20:12:31 +02:00
Régis Behmo
658d6957db sec: apply rate limiting security fix 2022-04-20 19:48:34 +02:00
Régis Behmo
51d93cd54c docs: list template filters in the v1 docs 2022-04-20 19:48:21 +02:00
Régis Behmo
34c8eeeaec fix: unit tests with v1 plugin installed
When a v1 plugin was installed, several things were happening regarding tests:

1. v1 plugin loading was happening despite the TUTOR_IGNORE_ENTRYPOINT_PLUGINS
   environment variable.
2. the CORE_READY event was not triggered because it was happening just once at
   import time.

This was causing some tests to incorrectly load the MFE plugin.
2022-04-20 19:48:21 +02:00
Régis Behmo
d9486018a2 feat: add --mount option to local/dev
The `--mount` option is available both with `tutor local`
and `tutor dev` commands. It allows users to easily bind-mount containers from
the host to containers. Yes, I know, we already provide that possibility with
the `bindmount` command and the `--volume=/path/` option. But these suffer from
the following drawbacks:

- They are difficult to understand.
- The "bindmount" command name does not make much sense.
- It's not convenient to mount an arbitrary folder from the host to multiple
  containers, such as the many lms/cms containers (web apps, celery workers and
  job runners).

To address this situation, we now recommend to make use of --mount:

1. `--mount=service1[,service2,...]:/host/path:/container/path`: manually mount
   `/host/path` to `/container/path` in container "service1" (and "service2").
2. `--mount=/host/path`: use the new v1 plugin API to discover plugins that
   will detect this option and select the right containers in which to bind-mount
   volumes. This is really nifty...

Close https://github.com/overhangio/2u-tutor-adoption/issues/43
2022-04-20 19:33:17 +02:00
Kyle McCormick
df0e26c58e feat: introduce tutor dev quickstart
Add `tutor dev quickstart` command, which is equivalent to
`tutor local quickstart`, but uses dev containers instead
of local production ones and includes some other small
differences for the convience of Open edX developers.
This should remove some friction
from the Open edX development setup process, which previously
required that users provision using local producation
containers but then stop them and switch to dev containers:
 * tutor local quickstart
 * tutor local stop
 * tutor dev start -d

Document the command and its improved workflow in
./docs/tutorials/nightly.rst

Fixes overhangio/2u-tutor-adoption#58
2022-04-19 16:53:57 +02:00
Kyle McCormick
07047288fc docs: explain command tree in conceptual intro 2022-04-19 16:53:57 +02:00
Régis Behmo
a6f136fd2c fix: push "all" images
Command `tutor images push all` was failing with:

    Error: Image 'all' could not be found
2022-04-15 17:48:13 +02:00
Régis Behmo
d5a790d5d0 refactor: get rid of the openedx Docker entrypoint
The entrypoint in the "openedx" Docker image was used only to define the
DJANGO_SETTINGS_MODULE environment variable, based on SERVICE_VARIANT and
SETTINGS. We ditch SETTINGS in favour of defining explicitely
DJANGO_SETTINGS_MODULE.

The problem with the Docker entrypoint is that it was bypassed whenever we ran
`tutor local exec` or `tutor k8s exec`. By removing it we make it simpler for
end-users to run manage.py commands in kubernetes.
2022-04-15 15:37:56 +02:00
Régis Behmo
5de7bec029 depr: drop the possibility of overriding edx-platform settings
Previously, it was possible to override settings by defining the
TUTOR_EDX_PLATFORM_SETTINGS environment variable. But let's face it:

- It was not very well supported.
- It was poorly explained.
- It was not very useful.
- It causes unnecessary code complexity.

For these reasons, we drop that feature.
2022-04-15 15:37:56 +02:00
Régis Behmo
15b219e235 feat: migrate to plugins.v1 with filters & actions
This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.

Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.

This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.

The plugin v0 API remains supported for backward compatibility of
existing plugins.

Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
  the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority

Close #499.
2022-04-15 15:30:54 +02:00
Gábor Boros
649244d6c4 style: update template formatting 2022-04-15 10:43:28 +02:00
Gábor Boros
060a0c7d32 refactor: split user and password creation 2022-04-15 10:43:28 +02:00
Gábor Boros
b71ae61f09 fix: remove quotes 2022-04-15 10:43:28 +02:00
Gábor Boros
bc17d47ebc fix: ensure mysql8 compatibility
MySQL 8 drop the support for creating users by executing `GRANT ALL`. This commit splits the user creation and permission granting, therefore the newer MySQL versions are supported too.

MySQL 8 is supported by edx-platform: 1cdb0347c5/playbooks/roles/mysql/tasks/mysql.yml (L93-L98)
2022-04-15 10:43:28 +02:00
Kyle McCormick
b6999824a7 feat: add tutor [dev|local|k8s] status command 2022-04-15 09:29:02 +02:00
Florian Haas
407a8566df fix: Correct EDX_PLATFORM_VERSION default for local edx-platform forks
PR #619 set the EDX_PLATFORM_VERSION build arg's default to
OPENEDX_COMMON_VERSION. While this works fine for setting a
non-default branch to run edx code from (say, "master"), it may break
if the user sets OPENEDX_COMMON_VERSION to a branch or tag name that
does not exist upstream in repositories *other than*
EDX_PLATFORM_REPOSITORY.

Thus, introduce a separate configuration parameter,
EDX_PLATFORM_VERSION, to match the build arg of the same name. Set its
default to OPENEDX_COMMON_VERSION.

This way, the user can deploy an arbitrarily-named fork of
edx-platform, while retaining the default OPENEDX_COMMON_VERSION
(like, for example "open-release/maple.3") for everything else.
2022-04-15 09:24:15 +02:00
Régis Behmo
cac9d14e01 feat: run tutor k8s exec ... commands with multiple args
Previously, the `k8s exec` command did not support unknown "--options". This
made it impossible to launch, say, a django shell in the lms container.

While implementing this feature we saw an opportunity to simplify the way jobs
are handled in the k8s commands.

Close #636.
Another related issue is: https://github.com/overhangio/2u-tutor-adoption/issues/52
2022-04-15 09:21:36 +02:00
Kyle McCormick
5f4159c030 docs: wrap tutor[full] in quotes for zsh compatibility
The docs recommend commands like:

    pip install tutor[full]
    pip install -e ./tutor[full]

for installing Tutor. These work fine in bash. For zsh,
though, which is now the default on macOS, quotes are
needed, otherwise zsh will interpret the brackets as
special syntax:

    pip install "tutor[full]"
    pip install -e "./tutor[full]"

Caveat: I have not tested this myself since I don't
own a Mac, but I've read several issue reports to this
effect, such as:
https://github.com/pypa/pipenv/issues/2830#issuecomment-419593199
2022-04-15 09:20:21 +02:00
Kyle McCormick
3ca4522f68 docs: update Nightly docs to reference [full] installation
The full installation will include all the plugins that
come bundled with Tutor stable. This is made possible by
a recent change to Tutor Nightly
(https://github.com/overhangio/tutor/pull/626).
2022-04-15 09:20:21 +02:00
Régis Behmo
7f7138f050 v13.1.11 (2022-04-12)
- [Security] Apply SAML security fix.
- [Improvement] In addition to the Docker build arguments
  `EDX_PLATFORM_REPOSITORY` and `NPM_REGISTRY`, also support two corresponding
  and identically-named `config.yml` values serving the same purpose.
2022-04-12 17:37:55 +02:00
Régis Behmo
16b2378165 sec: fix SAML vulnerability 2022-04-12 17:36:59 +02:00
Florian Haas
eaa578eabd feat: Make the platform repository and NPM registry configurable from config.yml
Previously, the only way for Tutor users to use a fork of edx-platform
or a custom NPM registry was to use build args during the image build.

This is suboptimal in the case of automatically building images from
CI pipelines, which may want to auto-detect when an image needs to be
rebuilt based on config.yml changes.

In addition, the EDX_PLATFORM_VERSION build argument can already be
set via a corresponding config.yml parameter (OPENEDX_COMMON_VERSION),
so it's reasonable to follow that precedent and also introduce
config.yml parameters to correspond with the EDX_PLATFORM_REPOSITORY
and NPM_REGISTRY build arguments.

Thus, introduce two new configuration parameters:

- EDX_PLATFORM_REPOSITORY
- NPM_REGISTRY

These parameters can now optionally be used instead of the
aforementioned build args.
2022-04-12 15:46:50 +02:00
Régis Behmo
11bfa96971 v13.1.10 (2022-04-11)
- [Feature] Upgrade all applications to open-release/maple.3.
2022-04-11 18:45:28 +02:00
Régis Behmo
0b86568ad5 feat: upgrade all apps to open-release/maple.3
Fresh out of the oven!
2022-04-11 18:44:15 +02:00
Rebecca Graber
7822f4e217 v13.1.9 (2022-04-06) 2022-04-06 22:56:40 +02:00
Florian Haas
baf7d03cf8 fix: Fix typo ("MongDB")
An user-facing echo_info message in the upgrade commands rendered
MongoDB as "MongDB". Trivial fix.
2022-04-01 11:29:14 +02:00
Régis Behmo
5aa1d53167 fix: remove duplicate LOCAL_PROJECT_NAME config entry 2022-03-31 16:31:03 +02:00
Régis Behmo
0500bfe9a6 fix: package version in nightly
The string returned by "make version" should not include the "-nightly"
suffix, otherwise tests fail.
2022-03-31 15:36:30 +02:00
Régis Behmo
fa32269e91 fix: nightly package version
The version of the nightly python package should not include the "-nightly"
suffix. That's because when we `pip install -e` tutor plugins, pip also
installs the latest tutor release, as part of the requirements. This overrides
the local (nightly) installation of tutor.

See: https://app.slack.com/client/T02SNA1T6/C02V3GHE3UP
2022-03-29 08:49:54 +02:00
Emad Rad
a25ae73031 minor typos fixed 2022-03-29 08:44:33 +02:00
Régis Behmo
43c5177187 v13.1.8 (2022-03-18)
- [Bugfix] Fix "evalsymlink failure" during `k8s quickstart` (#611).
- [Bugfix] Fix "TypeError: upgrade() got an unexpected keyword argument 'non_interactive'" during `local upgrade`.
2022-03-24 09:41:41 +01:00
Régis Behmo
6e2cef0db5 fix: "evalsymlink failure" during k8s namespace creation
Close #611 (again).
2022-03-24 08:25:56 +01:00