6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-11-17 18:45:15 +00:00
Commit Graph

170 Commits

Author SHA1 Message Date
Régis Behmo
8f2df6a842 ci: better github release management with scriv
In scriv 1.1.0 the GitHub release description can be templated:
https://github.com/nedbat/scriv/issues/61
https://github.com/nedbat/scriv/releases/tag/1.1.0

This means that we can finally get rid of our ugly scripts to generate the
release description \o/
2023-01-17 12:12:10 +01:00
Overhang.IO
a8039dc076 Merge remote-tracking branch 'origin/master' into nightly 2022-11-29 15:58:06 +00:00
Régis Behmo
08a14c80db ci: simplify release process
Previously, tags had to be created locally before they were pushed upstream.
Now they are automatically created in CI when __version__ is bumped.
2022-11-29 16:46:06 +01:00
Overhang.IO
d9cdf1731f Merge remote-tracking branch 'origin/master' into nightly 2022-11-29 14:50:31 +00:00
Régis Behmo
88d882b5b8 ci: modernize github release script
We address the following issues in CI:

- Node 12 deprecation: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
  To do that we upgrade actions/checkout and actions/setup-python to v3.
- Deprecated set-output command: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
- Get rid of the actions/upload-files-to-a-github-release action to use the gh
  script which ships by default in GitHub hosted runners: https://github.com/cli/cli

Unfortunately we could not get rid of the ugly `sed` in `make
release-description`. I wish we could use `envsubst` but it's not available on
GitHub action runners.
2022-11-29 15:11:47 +01:00
Régis Behmo
c06ac5b020 Merge branch 'master' into nightly 2022-11-24 17:27:47 +01:00
Régis Behmo
c886b72ae7 docs: migrate to scriv to manage changelog
Changelog management was starting to be a hassle:

- there were conflicts every time a PR was merged
- there were conflicts every time we merged the nightly branch in the new
  release branch, or vice versa.

Now, all changelog entries are stored as separate files in changelog.d,
including nightly. Nightly entries will be collected for every major release.
2022-11-24 16:15:15 +01:00
Régis Behmo
d7c667835a Merge branch 'master' into nightly 2022-11-15 16:59:19 +01:00
Régis Behmo
020f6d1137 depr: richie plugin is no longer official
The richie plugin was transferred to the openfun GitHub org. As a consequence,
it is no longer part of the list of official plugins.

See: https://github.com/overhangio/tutor-contrib-richie/pull/5
2022-09-30 12:09:31 +02:00
Régis Behmo
15b219e235 feat: migrate to plugins.v1 with filters & actions
This is a very large refactoring which aims at making Tutor both more
extendable and more generic. Historically, the Tutor plugin system was
designed as an ad-hoc solution to allow developers to modify their own
Open edX platforms without having to fork Tutor. The plugin API was
simple, but limited, because of its ad-hoc nature. As a consequence,
there were many things that plugin developers could not do, such as
extending different parts of the CLI or adding custom template filters.

Here, we refactor the whole codebase to make use of a generic plugin
system. This system was inspired by the Wordpress plugin API and the
Open edX "hooks and filters" API. The various components are added to a
small core thanks to a set of actions and filters. Actions are callback
functions that can be triggered at different points of the application
lifecycle. Filters are functions that modify some data. Both actions and
filters are collectively named as "hooks". Hooks can optionally be
created within a certain context, which makes it easier to keep track of
which application created which callback.

This new hooks system allows us to provide a Python API that developers
can use to extend their applications. The API reference is added to the
documentation, along with a new plugin development tutorial.

The plugin v0 API remains supported for backward compatibility of
existing plugins.

Done:
- Do not load commands from plugins which are not enabled.
- Load enabled plugins once on start.
- Implement contexts for actions and filters, which allow us to keep track of
  the source of every hook.
- Migrate patches
- Migrate commands
- Migrate plugin detection
- Migrate templates_root
- Migrate config
- Migrate template environment globals and filters
- Migrate hooks to tasks
- Generate hook documentation
- Generate patch reference documentation
- Add the concept of action priority

Close #499.
2022-04-15 15:30:54 +02:00
Régis Behmo
0500bfe9a6 fix: package version in nightly
The string returned by "make version" should not include the "-nightly"
suffix, otherwise tests fail.
2022-03-31 15:36:30 +02:00
Régis Behmo
2d20a04363 feat: add a convenient tutor k8s apply command
This is convenient to check k8s template validity.
2022-02-25 14:50:36 +01:00
Ned Batchelder
f2260006bd build: 'make help' works nicer on Mac
sed on Mac doesn't seem to understand \n.

While I was in here, I fixed some typos.
2022-01-06 10:44:53 +01:00
alex.soh
72843c06f9 refactor: add code coverage, cover CLI commands with tests 2022-01-04 13:40:33 +01:00
Régis Behmo
b18c9dc4f8 fix: lint unused arguments in code base 2021-12-20 21:19:10 +01:00
Régis Behmo
b8ab829c11 feat: move all forum-related code to a dedicated plugin
Forum is an optional feature, and as such it deserves its own plugin. Starting
from Maple, users will be able to install the forum from
https://github.com/overhangio/tutor-forum/

Close #450.
2021-12-20 21:19:10 +01:00
Régis Behmo
9964315abe chore: get rid of tutor-openedx
In the past, tutor was installed with "pip install tutor-openedx". For
some time (since v12.0.2), "tutor" was installed as a dependency of
"tutor-openedx". Now is the time to get rid of that old package.
The standard way of installing tutor is now with "pip install tutor".
2021-12-20 21:19:10 +01:00
Régis Behmo
0a4a2bbac2 feat: add richie to bundled plugins
For more information, checkout the plugin docs:
https://github.com/overhangio/tutor-richie
2021-11-09 11:48:32 +01:00
Régis Behmo
8be574aac8 fix: make sure that tutor-openedx is an empty package
Previously, the tutor-openedx package was loading tons of template data from
the MANIFEST.in. Turns out, we cannot ignore the MANIFEST.in file with
setuptools. So we need to move tutor-openedx to a separate, dedicated folder.
To auto-discover the package version, we copy it at runtime (in the make
command).
2021-07-06 09:42:57 +02:00
Régis Behmo
c15eec53dc feat: switch the package name from "tutor-openedx" to "tutor"
The package maintainer of the "tutor" package was kind enough to
transfer ownership of the project to us. This is great, because we no
longer have to use the "openedx" suffix, which is trademarked.

For the time being, we keep maintaining the "tutor-openedx" package
which has a 1-to-1 dependency on the "tutor" package. In the future, we
expect that we will no longer push upgrades to tutor-openedx.
2021-07-03 11:07:37 +02:00
Régis Behmo
ceddc11c29 feat: upgrade to open-release/lilac.master
One of the breaking changes of this release is the removal of the webui and
android features; these are moved to dedicated plugins. This causes a breaking
change: the renaming of the DOCKER_IMAGE_ANDROID
config variable to ANDROID_DOCKER_IMAGE.

See this TEP for reference: https://discuss.overhang.io/t/separate-webui-and-android-from-tutor-core-and-move-to-dedicated-plugins/1473
2021-06-08 23:29:12 +02:00
Régis Behmo
6af6390fb1 refactor: migrate from Travis.ci to GitHub Actions
GitHub Actions now performs the following tasks:

- run tests on every PR
- sync with git.overhang.io on push
- build binary releases on tags

Travis.CI was completely removed from this repo.
2021-05-17 21:29:52 +02:00
Régis Behmo
887ba31e09 code: lint unused imports
Unused imports are only warnings in pylint; here we make sure that they
are processed as errors.
2021-04-18 18:02:02 +02:00
Régis Behmo
f688a279dc Actually remove the images build --pull option
This option is mostly useless to us in CI, as it attempts (and fails) to
update the base image when building the dev image. For good reason: the
base image of the dev image is the latest openedx, which has not been
pushed to docker.io at the time of building -_-
2021-03-17 18:21:16 +01:00
Régis Behmo
27a7893c45 Clean some minor stuff from Makefile 2021-03-17 18:21:16 +01:00
Régis Behmo
b557cf9b43 Simplify the release process
Instead of pushing to multiple remotes, push to just one. It will then
sync changes to the other.
2021-03-17 17:33:06 +01:00
Régis Behmo
0a670d7ead refactor: add type annotations
Annotations were generated with pyannotate:
https://github.com/dropbox/pyannotate

We are running in strict mode, which is awesome!

This affects a large part of the code base, which might be an issue for
people running a fork of Tutor. Nonetheless, the behavior should not be
affected. If anything, this process has helped find and resolve a few
type-related bugs. Thus, this is not considered as a breaking change.
2021-03-15 21:46:55 +01:00
Régis Behmo
9afdb3f42b Upgrade github-release to fix "no such host" error
See failing job: https://travis-ci.org/github/overhangio/tutor/jobs/757515583

This is almost certainly due to a change in the github API response:
https://github.com/meterup/github-release/issues/1

See discussion:
https://discuss.overhang.io/t/no-packages-for-release-11-1-4/1278/2
2021-02-04 17:02:40 +01:00
Régis Behmo
728ef966dc v11.0.0 (2020-12-09)
- 💥[Improvement] Upgrade Open edX to Koa
- 💥 Setting changes:
    - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
    - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
    - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
    - The ``NGINX_HTTPS_PORT`` setting is deprecated.
- Architectural changes:
    - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
	- Nginx no longer listens to port 443 for https traffic
	- The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
	- Configuration files for web proxies are no longer provided.
	- Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
    - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
    - Replace memcached and rabbitmq by redis.
- Additional features:
    - Make it possible to disable all plugins at once with ``plugins disable all``.
    - Add ``tutor k8s wait`` command to wait for a pod to become ready
    - Faster, more reliable static assets with local memory caching
- Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
- Removed plugin `{{ patch (...) }}` statements:
    - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
    - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
2020-12-10 01:05:02 +01:00
Régis Behmo
2ca62be52e Force running tests on creating release
This will help us detect early issues with the releases.
2020-10-15 14:28:18 +02:00
Régis Behmo
9d0f2d9178 Simplify CI
This comes after a big refactoring of our CI pipeline.
2020-10-02 14:17:29 +02:00
Régis Behmo
443d7fa46c Add CI command to view size of bundled tutor binary 2020-07-01 09:32:08 +02:00
Régis Behmo
77150c0424 Push to repo as part of "make release" 2020-06-21 13:47:14 +02:00
Régis Behmo
4d6de0138a v10.0.0 Upgrade to Juniper (2020-06-15)
Here, we upgrade the Open edX platform from Ironwood to Juniper. This
upgrade does not come with many feature changes, but there are many
technical improvements under the hood:

- Upgrade from Python 2.7 to 3.5
- Upgrade from Mongodb v3.2 to v3.6
- Upgrade Ruby to 2.5.7

We took the opportunity to completely rething the way locally running
platforms should be accessed for testing purposes. It is no longer
possible to access a running platform from http://localhost and
http://studio.localhost. Instead, users should access
http://local.overhang.io and https://studio.local.overhang.io. This
drastically simplifies internal communication between Docker containers.

To upgrade, users should simply run:

    tutor local quickstart

For Kubernetes platform, the upgrade process is outlined when running:

    tutor k8s upgrade --from=ironwood
2020-06-15 10:19:07 +02:00
Régis Behmo
103b78a78a Fix github release download in travis CI 2020-04-26 01:05:48 +02:00
Régis Behmo
19d85e621b Fix "distribution was not found and is required by kubernetes" in CI
When running "python setup.py install" in CI, we were getting
"requests distribution was not found and is required by kubernetes"
errors. I can reproduce this issue locally. The error disappears after
the same command is run a second time.

This is a similar issue: https://github.com/pypa/setuptools/issues/498
2020-04-26 00:36:17 +02:00
Régis Behmo
fac336e6d7 Upgrade script for pushing github releases
The github-release is no longer maintained by aktau. We need to upgrade
because of Github authentication warnings. We also add a custom python
script for uploading assets to github. We thought it would be a simple
script, but instead we need to deal with deletion of existing assets and
releases, so we decide instead to keep relying on that 3rd-party script.
2020-04-20 06:23:57 +02:00
Régis Behmo
159e24a95d Refactor plugin internals
This is for supporting json-based plugins. The great thing about this
change is that it allows us to easily print plugin version numbers in
`plugins list`.
2020-01-14 15:41:42 +01:00
Régis Behmo
e2aba3bc93 Fix missing openssl-dev library in CI
This was causing an error during the install of the cryptography module
in CI (alpine linux).
2020-01-13 22:40:10 +01:00
Régis Behmo
237d9d76ee Attempt to pin setuptools/pip to fix CI
We are facing the following issues in CI:

    $ ./dist/tutor --version
    [5201] Failed to execute script pyi_rth_pkgres
    Traceback (most recent call last):
      File "site-packages/PyInstaller/loader/rthooks/pyi_rth_pkgres.py", line 13, in <module>
      File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 623, in exec_module
          exec(bytecode, module.__dict__)
      File "site-packages/pkg_resources/__init__.py", line 86, in <module>
      ModuleNotFoundError: No module named 'pkg_resources.py2_warn'
    make: *** [ci-bundle] Error 255

And:

    $ make ci-install-python-requirements
    ...
    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
2020-01-13 22:40:10 +01:00
Régis Behmo
2e3ebaf084 Upgrade all requirements
Also, add a `upgrade-requirements` target.
2020-01-10 15:38:17 +01:00
Régis Behmo
42359b8f02 Add "docs" target to root makefile 2020-01-10 15:30:09 +01:00
Régis Behmo
9620447a4d Add the lts plugin to the tutor binary 2019-12-25 01:09:53 +01:00
Régis Behmo
8127546af8 Improve make test-packages
`make test-packages` now tests a single package.
2019-11-22 09:38:53 +01:00
Régis Behmo
11f8c87f4a Make sure tutor-openedx is installed as part of requirements
Otherwise, tutor-openedx will be pulled from pypi during plugin install.
This was failing because the latest pypi release was not installing
correctly (see #261).
2019-10-19 19:13:03 +02:00
Régis Behmo
816f32e610 Fix alpine requirements in CI 2019-10-09 13:40:18 +02:00
Régis Behmo
1224313e7c Make sure twine & pip are up-to-date in CI
We were affected by the following bug when installing pyinstaller in CI:

https://github.com/pyinstaller/pyinstaller/issues/4006
https://github.com/pypa/pip/issues/6163

As a consequence, pip is upgraded in CI. Also, twine is now part of dev
requirements.
2019-10-09 13:22:00 +02:00
Régis Behmo
a94cbc2914 Move plugins to dedicated repositories
minio, notes and xqueue plugins are moved to tutor-* github repos.
2019-08-21 18:32:39 +02:00
Régis Behmo
7790028cf7 Move tutor-minio to dedicated plugin repo 2019-08-20 17:46:53 +02:00
Régis Behmo
be1ff08917 Add all plugins (with data) to binary bundle
All existing plugins are added to the binary bundle, in their latest
version, so that users don't need to pip install tutor.

Also, the tutor MANIFEST.in file was removed to simplify the management
of package data.

Close #242.
2019-08-20 17:03:46 +02:00