6
0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-12-12 14:17:46 +00:00

Merge remote-tracking branch 'origin/master' into nightly

This commit is contained in:
Overhang.IO 2022-04-15 09:14:51 +00:00
commit b70bbddce3
13 changed files with 93 additions and 44 deletions

View File

@ -4,6 +4,9 @@ Note: Breaking changes between versions are indicated by "💥".
## Unreleased
- [Feature] Add `tutor [dev|local|k8s] status` command, which provides basic information about the platform's status.
- 💥[Improvement] Make it possible to run `tutor k8s exec <command with multiple arguments>` (#636). As a consequence, it is no longer possible to run quoted commands: `tutor k8s exec "<some command>"`. Instead, you should remove the quotes: `tutor k8s exec <some command>`.
## v13.1.11 (2022-04-12)
- [Security] Apply SAML security fix.

View File

@ -1,6 +1,6 @@
Install this version from pip with::
pip install tutor[full]==TUTOR_VERSION
pip install "tutor[full]==TUTOR_VERSION"
Or download the compiled binaries::

View File

@ -87,6 +87,10 @@ This defines the git repository from which you install Open edX platform code. I
This defines the default version that will be pulled from all Open edX git repositories.
- ``EDX_PLATFORM_VERSION`` (default: the value of ``OPENEDX_COMMON_VERSION``)
This defines the version that will be pulled from just the Open edX platform git repositories. You may also override this configuration parameter at build time, by providing a ``--build-arg`` option.
- ``OPENEDX_CMS_UWSGI_WORKERS`` (default: ``2``)
- ``OPENEDX_LMS_UWSGI_WORKERS`` (default: ``2``)
@ -222,7 +226,7 @@ openedx Docker Image build arguments
When building the "openedx" Docker image, it is possible to specify a few `arguments <https://docs.docker.com/engine/reference/builder/#arg>`__:
- ``EDX_PLATFORM_REPOSITORY`` (default: ``"{{ EDX_PLATFORM_REPOSITORY }}"``)
- ``EDX_PLATFORM_VERSION`` (default: ``"{{ OPENEDX_COMMON_VERSION }}"``)
- ``EDX_PLATFORM_VERSION`` (default: ``"{{ EDX_PLATFORM_VERSION }}"``, which if unset defaults to ``{{ OPENEDX_COMMON_VERSION }}``)
- ``NPM_REGISTRY`` (default: ``"{{ NPM_REGISTRY }}"``)
These arguments can be specified from the command line, `very much like Docker <https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg>`__. For instance::

View File

@ -1,3 +1,3 @@
.. parsed-literal::
pip install tutor[full]
pip install "tutor[full]"

View File

@ -89,7 +89,7 @@ Upgrading
To upgrade Open edX or benefit from the latest features and bug fixes, you should simply upgrade Tutor. Start by upgrading the "tutor" package and its dependencies::
pip install --upgrade tutor[full]
pip install --upgrade "tutor[full]"
Then run the ``quickstart`` command again. Depending on your deployment target, run either::

View File

@ -64,6 +64,10 @@ On Minikube, the dashboard is already installed. To access the dashboard, run::
minikube dashboard
Lastly, Tutor itself provides a rudimentary listing of your cluster's nodes, workloads, and services::
tutor k8s status
Technical details
-----------------

View File

@ -98,6 +98,14 @@ Finally, tracking logs that store `user events <https://edx.readthedocs.io/proje
$(tutor config printroot)/data/lms/logs/tracking.log
$(tutor config printroot)/data/cms/logs/tracking.log
Status
~~~~~~
You can view your platform's containers::
tutor local status
Notice the **State** column in the output. It will tell you whether each container is starting, restarting, running (``Up``), cleanly stopped (``Exit 0``), or stopped on error (``Exit N``, where N ≠ 0).
Common tasks
------------

View File

@ -11,16 +11,16 @@ Installing Tutor Nightly
Running Tutor Nightly requires more than setting a few configuration variables: because there are so many Open edX settings, version numbers, etc. which may change between the latest release and the current master branch, Tutor Nightly is actually maintained as a separate branch of the Tutor repository. To install Tutor Nightly, you should install Tutor from the "nightly" branch of the source repository. To do so, run::
git clone --branch=nightly https://github.com/overhangio/tutor.git
pip install -e ./tutor
pip install -e "./tutor[full]"
As usual, it is strongly recommended to run the command above in a `Python virtual environment <https://docs.python.org/3/tutorial/venv.html>`__.
All Tutor plugins that you wish to use should likewise be installed from the "nightly branch". For instance, the `MFE plugin <https://github.com/overhangio/tutor-mfe>`__::
In addition to installing Tutor Nightly itself, this will install automatically the nightly versions of all official Tutor plugins (which are enumerated in `plugins.txt <https://github.com/overhangio/tutor/tree/nightly/requirements/plugins.txt>`_). Alternatively, if you wish to hack on an official plugin or install a custom plugin, you can clone that plugin's repository and install it. For instance::
git clone --branch=nightly https://github.com/overhangio/tutor-mfe.git
pip install -e ./tutor-mfe
git clone --branch=nightly https://github.com/myorganization/tutor-contrib-myplugin.git
pip install -e ./tutor-contrib-myplugin
You can then run the usual ``tutor`` commands::
Once Tutor Nightly is installed, you can run the usual ``tutor`` commands::
tutor local quickstart
tutor local stop

View File

@ -263,6 +263,12 @@ def logs(context: click.Context, follow: bool, tail: bool, service: str) -> None
context.invoke(dc_command, command="logs", args=args)
@click.command(help="Print status information for containers")
@click.pass_context
def status(context: click.Context) -> None:
context.invoke(dc_command, command="ps")
@click.command(
short_help="Direct interface to docker-compose.",
help=(
@ -307,3 +313,4 @@ def add_commands(command_group: click.Group) -> None:
command_group.add_command(bindmount_command)
command_group.add_command(execute)
command_group.add_command(logs)
command_group.add_command(status)

View File

@ -8,7 +8,7 @@ from tutor import config as tutor_config
from tutor import env as tutor_env
from tutor import exceptions, fmt, jobs, serialize, utils
from tutor.commands.config import save as config_save_command
from tutor.commands.context import Context
from tutor.commands.context import BaseJobContext
from tutor.commands.upgrade.k8s import upgrade_from
from tutor.types import Config, get_typed
@ -147,9 +147,15 @@ class K8sJobRunner(jobs.BaseJobRunner):
return 0
class K8sContext(BaseJobContext):
def job_runner(self, config: Config) -> K8sJobRunner:
return K8sJobRunner(self.root, config)
@click.group(help="Run Open edX on Kubernetes")
def k8s() -> None:
pass
@click.pass_context
def k8s(context: click.Context) -> None:
context.obj = K8sContext(context.obj.root)
@click.command(help="Configure and run Open edX from scratch")
@ -213,7 +219,7 @@ Press enter when you are ready to continue"""
)
@click.argument("names", metavar="name", nargs=-1)
@click.pass_obj
def start(context: Context, names: List[str]) -> None:
def start(context: K8sContext, names: List[str]) -> None:
config = tutor_config.load(context.root)
# Create namespace, if necessary
# Note that this step should not be run for some users, in particular those
@ -263,7 +269,7 @@ def start(context: Context, names: List[str]) -> None:
)
@click.argument("names", metavar="name", nargs=-1)
@click.pass_obj
def stop(context: Context, names: List[str]) -> None:
def stop(context: K8sContext, names: List[str]) -> None:
config = tutor_config.load(context.root)
names = names or ["all"]
for name in names:
@ -301,7 +307,7 @@ def reboot(context: click.Context) -> None:
@click.command(help="Completely delete an existing platform")
@click.option("-y", "--yes", is_flag=True, help="Do not ask for confirmation")
@click.pass_obj
def delete(context: Context, yes: bool) -> None:
def delete(context: K8sContext, yes: bool) -> None:
if not yes:
click.confirm(
"Are you sure you want to delete the platform? All data will be removed.",
@ -319,9 +325,9 @@ def delete(context: Context, yes: bool) -> None:
@click.command(help="Initialise all applications")
@click.option("-l", "--limit", help="Limit initialisation to this service or plugin")
@click.pass_obj
def init(context: Context, limit: Optional[str]) -> None:
def init(context: K8sContext, limit: Optional[str]) -> None:
config = tutor_config.load(context.root)
runner = K8sJobRunner(context.root, config)
runner = context.job_runner(config)
wait_for_pod_ready(config, "caddy")
for name in ["elasticsearch", "mysql", "mongodb"]:
if tutor_config.is_service_activated(config, name):
@ -333,7 +339,7 @@ def init(context: Context, limit: Optional[str]) -> None:
@click.argument("deployment")
@click.argument("replicas", type=int)
@click.pass_obj
def scale(context: Context, deployment: str, replicas: int) -> None:
def scale(context: K8sContext, deployment: str, replicas: int) -> None:
config = tutor_config.load(context.root)
utils.kubectl(
"scale",
@ -354,25 +360,32 @@ def scale(context: Context, deployment: str, replicas: int) -> None:
"-p",
"--password",
help="Specify password from the command line. If undefined, you will be prompted to input a password",
prompt=True,
hide_input=True,
)
@click.argument("name")
@click.argument("email")
@click.pass_obj
def createuser(
context: Context, superuser: str, staff: bool, password: str, name: str, email: str
context: K8sContext,
superuser: str,
staff: bool,
password: str,
name: str,
email: str,
) -> None:
config = tutor_config.load(context.root)
command = jobs.create_user_command(superuser, staff, name, email, password=password)
# This needs to be interactive in case the user needs to type a password
kubectl_exec(config, "lms", command, attach=True)
runner = context.job_runner(config)
runner.run_job("lms", command)
@click.command(help="Import the demo course")
@click.pass_obj
def importdemocourse(context: Context) -> None:
def importdemocourse(context: K8sContext) -> None:
fmt.echo_info("Importing demo course")
config = tutor_config.load(context.root)
runner = K8sJobRunner(context.root, config)
runner = context.job_runner(config)
jobs.import_demo_course(runner)
@ -391,20 +404,24 @@ def importdemocourse(context: Context) -> None:
)
@click.argument("theme_name")
@click.pass_obj
def settheme(context: Context, domains: List[str], theme_name: str) -> None:
def settheme(context: K8sContext, domains: List[str], theme_name: str) -> None:
config = tutor_config.load(context.root)
runner = K8sJobRunner(context.root, config)
runner = context.job_runner(config)
domains = domains or jobs.get_all_openedx_domains(config)
jobs.set_theme(theme_name, domains, runner)
@click.command(name="exec", help="Execute a command in a pod of the given application")
@click.command(
name="exec",
help="Execute a command in a pod of the given application",
context_settings={"ignore_unknown_options": True},
)
@click.argument("service")
@click.argument("command")
@click.argument("args", nargs=-1, required=True)
@click.pass_obj
def exec_command(context: Context, service: str, command: str) -> None:
def exec_command(context: K8sContext, service: str, args: List[str]) -> None:
config = tutor_config.load(context.root)
kubectl_exec(config, service, command, attach=True)
kubectl_exec(config, service, args)
@click.command(help="View output from containers")
@ -414,7 +431,7 @@ def exec_command(context: Context, service: str, command: str) -> None:
@click.argument("service")
@click.pass_obj
def logs(
context: Context, container: str, follow: bool, tail: bool, service: str
context: K8sContext, container: str, follow: bool, tail: bool, service: str
) -> None:
config = tutor_config.load(context.root)
@ -435,7 +452,7 @@ def logs(
@click.command(help="Wait for a pod to become ready")
@click.argument("name")
@click.pass_obj
def wait(context: Context, name: str) -> None:
def wait(context: K8sContext, name: str) -> None:
config = tutor_config.load(context.root)
wait_for_pod_ready(config, name)
@ -476,7 +493,7 @@ def upgrade(context: click.Context, from_release: Optional[str]) -> None:
)
@click.argument("args", nargs=-1)
@click.pass_obj
def apply_command(context: Context, args: List[str]) -> None:
def apply_command(context: K8sContext, args: List[str]) -> None:
kubectl_apply(context.root, *args)
@ -484,9 +501,14 @@ def kubectl_apply(root: str, *args: str) -> None:
utils.kubectl("apply", "--kustomize", tutor_env.pathjoin(root), *args)
def kubectl_exec(
config: Config, service: str, command: str, attach: bool = False
) -> int:
@click.command(help="Print status information for all k8s resources")
@click.pass_obj
def status(context: K8sContext) -> int:
config = tutor_config.load(context.root)
return utils.kubectl("get", "all", *resource_namespace_selector(config))
def kubectl_exec(config: Config, service: str, command: List[str]) -> int:
selector = f"app.kubernetes.io/name={service}"
pods = K8sClients.instance().core_api.list_namespaced_pod(
namespace=k8s_namespace(config), label_selector=selector
@ -498,18 +520,15 @@ def kubectl_exec(
pod_name = pods.items[0].metadata.name
# Run command
attach_opts = ["-i", "-t"] if attach else []
return utils.kubectl(
"exec",
*attach_opts,
"--stdin",
"--tty",
"--namespace",
k8s_namespace(config),
pod_name,
"--",
"sh",
"-e",
"-c",
command,
*command,
)
@ -561,3 +580,4 @@ k8s.add_command(logs)
k8s.add_command(wait)
k8s.add_command(upgrade)
k8s.add_command(apply_command)
k8s.add_command(status)

View File

@ -33,7 +33,7 @@ RUN dockerize_url="https://github.com/powerman/dockerize/releases/download/$DOCK
###### Checkout edx-platform code
FROM minimal as code
ARG EDX_PLATFORM_REPOSITORY={{ EDX_PLATFORM_REPOSITORY }}
ARG EDX_PLATFORM_VERSION={{ OPENEDX_COMMON_VERSION }}
ARG EDX_PLATFORM_VERSION={{ EDX_PLATFORM_VERSION }}
RUN mkdir -p /openedx/edx-platform && \
git clone $EDX_PLATFORM_REPOSITORY --branch $EDX_PLATFORM_VERSION --depth 1 /openedx/edx-platform
WORKDIR /openedx/edx-platform

View File

@ -19,6 +19,7 @@ DOCKER_IMAGE_PERMISSIONS: "{{ DOCKER_REGISTRY }}overhangio/openedx-permissions:{
DOCKER_IMAGE_REDIS: "docker.io/redis:6.2.6"
DOCKER_IMAGE_SMTP: "docker.io/devture/exim-relay:4.95-r0-2"
EDX_PLATFORM_REPOSITORY: "https://github.com/openedx/edx-platform.git"
EDX_PLATFORM_VERSION: "{{ OPENEDX_COMMON_VERSION }}"
ELASTICSEARCH_HOST: "elasticsearch"
ELASTICSEARCH_PORT: 9200
ELASTICSEARCH_SCHEME: "http"

View File

@ -15,5 +15,7 @@ done
echo "MySQL is up and running"
# edx-platform database
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e 'CREATE DATABASE IF NOT EXISTS {{ OPENEDX_MYSQL_DATABASE }};'
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e 'GRANT ALL ON {{ OPENEDX_MYSQL_DATABASE }}.* TO "{{ OPENEDX_MYSQL_USERNAME }}"@"%" IDENTIFIED BY "{{ OPENEDX_MYSQL_PASSWORD }}";'
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e "CREATE DATABASE IF NOT EXISTS {{ OPENEDX_MYSQL_DATABASE }};"
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e "CREATE USER IF NOT EXISTS '{{ OPENEDX_MYSQL_USERNAME }}';"
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e "ALTER USER '{{ OPENEDX_MYSQL_USERNAME }}'@'%' IDENTIFIED BY '{{ OPENEDX_MYSQL_PASSWORD }}';"
mysql -u {{ MYSQL_ROOT_USERNAME }} --password="{{ MYSQL_ROOT_PASSWORD }}" --host "{{ MYSQL_HOST }}" --port {{ MYSQL_PORT }} -e "GRANT ALL ON {{ OPENEDX_MYSQL_DATABASE }}.* TO '{{ OPENEDX_MYSQL_USERNAME }}'@'%';"