6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-13 22:48:20 +00:00
Commit Graph

1882 Commits

Author SHA1 Message Date
Régis Behmo
947b37524f feat: auto build "openedx-dev" on "dev launch"
To achieve that, we introduce a new IMAGES_BUILD_REQUIRED filter.
2023-06-14 21:08:49 +02:00
Régis Behmo
17f66fb467 refactor: simplify image tag management 2023-06-14 21:08:49 +02:00
Régis Behmo
2a21b2adf3 feat: images build openedx-dev
We no longer run `docker-compose up --build`. Instead, users are
encouraged to build the "openedx-dev" Docker image.
2023-06-14 21:08:49 +02:00
Régis Behmo
a5dd3017d7 depr: remove obsolete task actions 2023-06-14 21:08:49 +02:00
Régis Behmo
3ab0dcb9e6 depr: templated hooks
Templated hooks we almost completely useless, so we get rid of them.
This allows us to get rid entirely of hook names and hook indexes, which
makes the whole implementation much simpler. Hook removal (with
`clear_all`) is achieved thanks to weak references.
2023-06-14 21:08:49 +02:00
Régis Behmo
cbe32cbc15 docs: improve all available values from ENV_TEMPLATE_VARIABLES 2023-06-14 21:08:49 +02:00
Régis Behmo
18ce1f2fe4 feat: persistent bind-mounts
This is an important change, where we get remove the previous `--mount`
option, and instead opt for persistent bind-mounts.

Persistent bind mounts have several advantages:
- They make it easier to remember which folders need to be bind-mounted.
- Code is *much* less clunky, as we no longer need to generate temporary
  docker-compose files.
- They allow us to bind-mount host directories *at build time* using the
  buildx `--build-context` option.
- The transition from development to production becomes much easier, as
  images will automatically be built using the host repo.

The only drawback is that persistent bind-mounts are slightly less
portable: when a config.yml file is moved to a different folder, many
things will break if the repo is not checked out in the same path.

For instance, this is how to start working on a local fork of
edx-platform:

    tutor config save --append MOUNTS=/path/to/edx-platform

And that's all there is to it. No, this fork will be used whenever we
run:

    tutor images build openedx
    tutor local start
    tutor dev start

This change is made possible by huge improvements in the build time
performance. These improvements make it convenient to re-build Docker
images often.

Related issues:
https://github.com/openedx/wg-developer-experience/issues/71
https://github.com/openedx/wg-developer-experience/issues/66
https://github.com/openedx/wg-developer-experience/issues/166
2023-06-14 21:08:49 +02:00
Régis Behmo
7972a75915 feat: leverage RUN --mount for faster image building
We make use of the Docker build cache to install python and nodejs
requirements faster in the case of repeated builds.

This feature is only possible for users of BuildKit, so we detect
whether `docker buildx` is available at runtime.

We do not make use of `COPY --link` because the `--link` option is
incompatible with `--chown=app:app`:
https://github.com/docker/buildx/issues/1408

For reference, see:

https://www.docker.com/blog/dockerfiles-now-support-multiple-build-contexts/
https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
2023-06-14 21:08:49 +02:00
Régis Behmo
220b8296a9 feat: config save --append/--remove KEY=VAL options
This paves the way for persisting bind-mounts across runs, since this
setting will be a list.
2023-06-14 19:40:58 +02:00
Régis Behmo
c2265c3f11 feat: simplify docker-compose permissions
It was useless to create a *-permissions job for every application.
Instead, we create a single "permissions" service. It can be extended
via the "docker-compose-permissions-command" patch.
2023-06-14 19:40:58 +02:00
Régis Behmo
ee8de62770 depr: RUN_LMS, RUN_CMS settings
These tutor settings are mostly useless and make templates much more
difficult to work with.
2023-06-14 19:40:58 +02:00
Régis Behmo
b2a22a6993 fix: remove useless "privileged: false" statements
These values are by default anyway.
2023-06-14 19:40:58 +02:00
Régis Behmo
39a8b46854 depr: drop support for docker-compose
Instead, the compose plugin must be installed.
We deprecate docker-compose because v1 is not supported starting from
the end of June 2023.

See "evolution of compose": https://docs.docker.com/compose/compose-v2/
2023-06-14 19:40:58 +02:00
Régis Behmo
11b2091e32 depr: halt compatibility with python 3.7 2023-06-14 19:40:58 +02:00
Régis Behmo
2d6b9c1767 feat: hide TOS link during registration
Registering a user was causing a 400 error because the LMS expected the
TOS checkbox to be checked, but it's not displayed in the frontend. So
we just disable it.

Close https://github.com/openedx/wg-build-test-release/issues/262
2023-06-14 19:40:58 +02:00
Régis Behmo
6dbb5baf81 feat: importdemocourse --repo-dir=... option
This allows us to run:

    tutor local do importdemocourse --repo=https://github.com/openedx/openedx-test-course --version=open-release/palm.master --repo-dir=test-course/course
2023-06-14 19:40:58 +02:00
Régis Behmo
b3c3c4a2cc feat: upgrade to Palm
Among other changes: ORA2 file uploads were stored in a folder named
"SET-ME-PLEASE (ex.  bucket-name)" (sigh). With this change, the folder
should be automatically renamed to "openedxuploads". This issue has been
occuring since June 2019... (sigh²)

Close #707
2023-06-14 19:40:58 +02:00
Régis Behmo
253b69ffe3 feat: faster builds with registry cache
Automatically pull Docker build cache from remote registry. This
considerably improves build performances, as discovered here:
https://github.com/overhangio/test-docker-build/
2023-06-14 19:40:58 +02:00
Overhang.IO
016502aad0 Merge remote-tracking branch 'origin/master' into nightly 2023-06-14 09:20:51 +00:00
Régis Behmo
5c426257fb ci: run on macOS-11 because there no more github runners for macOS 10
Jobs are no longer being picked up:
https://github.com/overhangio/tutor/actions/runs/5259868936/jobs/9515592636
2023-06-14 09:41:44 +02:00
Régis Behmo
b8747996c9 fix: scriv error during github release
See:
https://github.com/overhangio/tutor/actions/runs/5260213022/jobs/9506811909#step:9:33

`scriv github-release --repo=overhangio/tutor` causes the following
error:

	Traceback (most recent call last):
	  File "/opt/hostedtoolcache/Python/3.7.16/x64/bin/scriv", line 8, in
	<module>
	    sys.exit(cli())
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/click/core.py",
	line 1130, in __call__
	    return self.main(*args, **kwargs)
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/click/core.py",
	line 1055, in main
	    rv = self.invoke(ctx)
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/click/core.py",
	line 1657, in invoke
	    return _process_result(sub_ctx.command.invoke(sub_ctx))
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/click/core.py",
	line 1404, in invoke
	    return ctx.invoke(self.callback, **ctx.params)
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/click/core.py",
	line 760, in invoke
	    return __callback(*args, **kwargs)
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/scriv/ghrel.py",
	line 99, in github_release
	    config=scriv.config,
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/jinja2/environment.py",
	line 1301, in render
	    self.environment.handle_exception()
	  File
	"/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/jinja2/environment.py",
	line 936, in handle_exception
	    raise rewrite_traceback_stack(source=source)
	  File "<template>", line 3, in top-level template code
	TypeError: 'Version' object is not subscriptable
2023-06-14 09:38:07 +02:00
Overhang.IO
bcc4f6bf1a Merge remote-tracking branch 'origin/master' into nightly 2023-06-13 08:41:13 +00:00
Régis Behmo
9d2c1f134c v15.3.7 2023-06-13 10:21:55 +02:00
Florian Haas
30d3ba3ba8 chore: Bump MongoDB default to version 4.2.24
Bump the default image reference for MongoDB from 4.2.17 to 4.2.24, to
address a critical issue present in versions 4.2.0 through 4.2.23.

References:
https://www.mongodb.com/docs/manual/release-notes/4.2/#patch-releases
https://jira.mongodb.org/browse/WT-10461

Fixes #854.
2023-06-13 10:10:53 +02:00
Overhang.IO
1fdbd0b326 Merge remote-tracking branch 'origin/master' into nightly 2023-06-06 13:46:58 +00:00
Régis Behmo
fb15f21bf5 chore: upgrade requirements
Resolve vulnerability detected in this automated PR:
https://github.com/overhangio/tutor/pull/840
2023-06-06 15:30:37 +02:00
Emad Rad
9fa04be2a3 doc: changelog entry created 2023-06-06 15:15:23 +02:00
Emad Rad
86f846aba3 feat: Add support for loading in-cluster config when running inside a pod
in some cases, tutor might run inside a pod, which that pod has access to a cluster via role binding and a service account. this way, there's no ./kube/config file, but kubectl commands run with no issue.

Close #843
2023-06-06 15:14:52 +02:00
Overhang.IO
e73e13a6e9 Merge remote-tracking branch 'origin/master' into nightly 2023-06-01 14:10:29 +00:00
Emad Rad
2d9a09f792 doc: changelog entry created 2023-06-01 15:44:27 +02:00
Emad Rad
1d3a215c00 feat: Add support for loading in-cluster config when running inside a pod
in some cases, tutor might run inside a pod, which that pod has access to a cluster via role binding and a service account. this way, there's no ./kube/config file, but kubectl commands run with no issue.

Close #843
2023-06-01 15:44:27 +02:00
Overhang.IO
af997dbc51 Merge remote-tracking branch 'origin/master' into nightly 2023-05-29 14:19:19 +00:00
Johan Castiblanco
4bb8671f26 fix!: use correct case for authsource mongo
Update `authSource`  keyword argument to  `authsource`

Authsource was configurated in camelcase.
https://github.com/overhangio/tutor/blob/master/tutor/templates/apps/openedx/settings/partials/common_all.py#L16
Something I think is ok because pymongo uses it in camelcase.
https://pymongo.readthedocs.io/en/3.10.1/api/pymongo/mongo_client.html

However, the error was presented because edx-platform use another variable when credentials are provided **(LOWERCASE)**.
https://github.com/openedx/edx-platform/blob/master/xmodule/mongo_utils.py#L72

Looking more ,  the `auth_source `is configured with another key name:  `authsource` **(LOWERCASE)** from the kwargs.
https://github.com/openedx/edx-platform/blob/master/xmodule/mongo_utils.py#L34

This won't change the **kwargs behavior of the camelCase in pymongo `MongoClient` because pymongo has a caseInsentive method.
https://github.com/mongodb/mongo-python-driver/blob/3.12.3/pymongo/mongo_client.py#L651
 ``it's  OK to pass `authsource` instead of authSource as a keyword argument.``
2023-05-29 15:53:36 +02:00
Overhang.IO
6d62660d26 Merge remote-tracking branch 'origin/master' into nightly 2023-05-29 09:40:27 +00:00
Régis Behmo
93688e18a1 ci: improve compatibility with main & nightly 2023-05-29 11:18:13 +02:00
Overhang.IO
d6ae056c94 Merge remote-tracking branch 'origin/master' into nightly 2023-05-26 14:28:57 +00:00
Brian Mesick
87ae658773 docs: Update Google Analytics example to GA4 2023-05-26 16:04:04 +02:00
Régis Behmo
35743eb905 Merge branch 'master' into nightly 2023-05-23 08:16:43 +02:00
Régis Behmo
185a13a2e1 v15.3.6 2023-05-22 18:17:34 +02:00
Régis Behmo
2233c3422a feat: upgrade to open-release/olive.4 2023-05-22 18:16:39 +02:00
Overhang.IO
e7f7d5b394 Merge remote-tracking branch 'origin/master' into nightly 2023-05-17 09:49:12 +00:00
Régis Behmo
de0093aa8d fix: build-time warning
Installing from source triggers a warning on pip 23.0.1 if
pyproject.toml is not present. Building does not require any special
dependencies, so we just add a simple pyproject.toml file.

Close #836
2023-05-17 10:29:32 +02:00
Overhang.IO
91bdcb860c Merge remote-tracking branch 'origin/master' into nightly 2023-05-16 10:28:38 +00:00
Régis Behmo
f042ca9b90 docs: fix ENV_PATCHES example 2023-05-16 12:03:38 +02:00
Overhang.IO
ca69eb31d3 Merge remote-tracking branch 'origin/master' into nightly 2023-05-03 16:46:39 +00:00
Kyle McCormick
2e276cbb09 docs: explain why docker-compose.jobs.yml exists
Based on:
https://github.com/overhangio/tutor/issues/642#issuecomment-1483970113
2023-05-03 17:51:13 +02:00
Overhang.IO
cadc8c0cc1 Merge remote-tracking branch 'origin/master' into nightly 2023-04-28 10:15:14 +00:00
Régis Behmo
6771732414 ci: fix dependency caching 2023-04-28 11:15:01 +02:00
Régis Behmo
5c5cd266f2 ci: allow manual trigger of "release" action
This is useful when the generation of a macOS binary has failed, for
instance.

In addition, we restore checking for the `gh` utility. This is necessary
when running CI locally with `act`.

In addition, we cache pip dependencies on github release and test runs.
2023-04-28 11:11:49 +02:00
Overhang.IO
c8aff67064 Merge remote-tracking branch 'origin/master' into nightly 2023-04-28 07:41:28 +00:00