christianlight/tutor
7
0
Fork 0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-06-26 16:53:28 +00:00
Code Issues Releases Activity
d3c842c8da
tutor/tutor/templates/local/docker-compose.jobs.yml

41 lines
1.6 KiB
YAML
Raw Normal View History

Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
version: "3.7"
services:
mysql-job:
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
image: {{ DOCKER_IMAGE_MYSQL }}
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
entrypoint: []
command: ["echo", "done"]
Attempt to resolve "Can't connect to MySQL server" on init Close #334 (hopefully)
2020-06-01 15:14:22 +00:00
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
lms-job:
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
image: {{ DOCKER_IMAGE_OPENEDX }}
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
environment:
SERVICE_VARIANT: lms
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
volumes:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro
- ../apps/openedx/settings/cms/:/openedx/edx-platform/cms/envs/tutor/:ro
- ../apps/openedx/config/:/openedx/config/:ro
Make it possible to run init without starting the full platform This is done by explicitely listing job dependencies. Unfortunately, it's not yet possible to move `init` before `start` in `quickstart`, because some services, such as discovery, depend on the LMS, which takes a few seconds to boot up. Actually, discovery also depends on ngins, as it points to "local.overhang.io" when referring to the lms.
2020-09-03 06:58:28 +00:00
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
cms-job:
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
image: {{ DOCKER_IMAGE_OPENEDX }}
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
environment:
SERVICE_VARIANT: cms
SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
volumes:
- ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/:ro
- ../apps/openedx/settings/cms/:/openedx/edx-platform/cms/envs/tutor/:ro
- ../apps/openedx/config/:/openedx/config/:ro
Make it possible to run init without starting the full platform This is done by explicitely listing job dependencies. Unfortunately, it's not yet possible to move `init` before `start` in `quickstart`, because some services, such as discovery, depend on the LMS, which takes a few seconds to boot up. Actually, discovery also depends on ngins, as it points to "local.overhang.io" when referring to the lms.
2020-09-03 06:58:28 +00:00
depends_on: {{ [("mysql", ACTIVATE_MYSQL)]|list_if }}
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
forum-job:
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
image: {{ DOCKER_IMAGE_FORUM }}
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
environment:
SEARCH_SERVER: "{{ ELASTICSEARCH_SCHEME }}://{{ ELASTICSEARCH_HOST }}:{{ ELASTICSEARCH_PORT }}"
MONGODB_AUTH: "{% if MONGODB_USERNAME and MONGODB_PASSWORD %}{{ MONGODB_USERNAME}}:{{ MONGODB_PASSWORD }}@{% endif %}"
MONGODB_HOST: "{{ MONGODB_HOST }}"
MONGODB_PORT: "{{ MONGODB_PORT }}"
Make it possible to run init without starting the full platform This is done by explicitely listing job dependencies. Unfortunately, it's not yet possible to move `init` before `start` in `quickstart`, because some services, such as discovery, depend on the LMS, which takes a few seconds to boot up. Actually, discovery also depends on ngins, as it points to "local.overhang.io" when referring to the lms.
2020-09-03 06:58:28 +00:00
depends_on: {{ [("elasticsearch", ACTIVATE_ELASTICSEARCH), ("mongodb", ACTIVATE_MONGODB)]|list_if }}
Make it possible to override the docker registry for individual services Previously, it was not possible to override the docker registry for just one or a few services. Setting the DOCKER_REGISTRY configuration parameter would apply to all images. This was inconvenient. To resolve this, we include the docker registry value in the DOCKER_IMAGE_* configuration parameters. This allows users to override the docker registry individually by defining the DOCKER_IMAGE_SERVICENAME configuration parameter. See https://discuss.overhang.io/t/kubernetes-ci-cd-pipeline/765/3
2020-07-21 07:13:00 +00:00
Improve job running in local and k8s Running jobs was previously done with "exec". This was because it allowed us to avoid copying too much container specification information from the docker-compose/deployments files to the jobs files. However, this was limiting: - In order to run a job, the corresponding container had to be running. This was particularly painful in Kubernetes, where containers are crashing as long as migrations are not correctly run. - Containers in which we need to run jobs needed to be present in the docker-compose/deployments files. This is unnecessary, for example when mysql is disabled, or in the case of the certbot container. Now, we create dedicated jobs files, both for local and k8s deployment. This introduces a little redundancy, but not too much. Note that dependent containers are not listed in the docker-compose.jobs.yml file, so an actual platform is still supposed to be running when we launch the jobs. This also introduces a subtle change: now, jobs go through the container entrypoint prior to running. This is probably a good thing, as it will avoid forgetting about incorrect environment variables. In k8s, we find ourselves interacting way too much with the kubectl utility. Parsing output from the CLI is a pain. So we need to switch to the native kubernetes client library.
2020-03-25 17:47:36 +00:00
{{ patch("local-docker-compose-jobs-services")|indent(4) }}
Copy Permalink