christianlight/tutor
7
0
Fork 0
mirror of https://github.com/ChristianLight/tutor.git synced 2024-06-07 00:20:49 +00:00
Code Issues Releases Activity

Merge remote-tracking branch 'origin/master' into nightly

Browse Source
This commit is contained in:
Overhang.IO 2021-10-25 18:26:44 +00:00
commit da7d95dea5
7 changed files with 115 additions and 108 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -7,6 +7,7 @@ Note: Breaking changes between versions are indicated by "💥".
## v12.1.5 (2021-10-25)
- 💥[Improvement] Change the `settheme` command such that, by default, a custom theme is assigned to the LMS and the CMS, both in production and development mode.
- [Bugfix] Change the `local quickstart` command to check for adequate Docker memory allocation on macOS (#463)
## v12.1.4 (2021-10-11)

4
docs/troubleshooting.rst
View File

@ -62,7 +62,9 @@ If the above command does not work, you should fix your Docker installation. Som
"Running migrations... Killed!" / "Command failed with status 137: docker-compose"
----------------------------------------------------------------------------------
Open edX requires at least 4 GB RAM, in particular to run the SQL migrations. If the ``tutor local quickstart`` command dies after displaying "Running migrations", you most probably need to buy more memory or add swap to your machine. On Docker for Mac OS, by default, containers are allocated at most 2 GB of RAM. You should follow `these instructions from the official Docker documentation <https://docs.docker.com/docker-for-mac/#advanced>`__ to allocate at least 4-5 Gb to the Docker daemon.
Open edX requires at least 4 GB RAM, in particular to run the SQL migrations. If the ``tutor local quickstart`` command dies after displaying "Running migrations", you most probably need to buy more memory or add swap to your machine.
On macOS, by default, Docker allocates at most 2 GB of RAM to containers. ``quickstart`` tries to check your current allocation and outputs a warning if it can't find a value of at least 4 GB. You should follow `these instructions from the official Docker documentation <https://docs.docker.com/docker-for-mac/#advanced>`__ to allocate at least 4-5 GB to the Docker daemon.
If migrations were killed halfway, there is a good chance that the MySQL database is in a state that is hard to recover from. The easiest way to recover is simply to delete all the MySQL data and restart the quickstart process. After you have allocated more memory to the Docker daemon, run::

31
requirements/base.txt
View File

@ -1,28 +1,26 @@
#
# This file is autogenerated by pip-compile with python 3.7
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
# pip-compile requirements/base.in
#
appdirs==1.4.4
# via -r requirements/base.in
cachetools==4.2.2
cachetools==4.2.4
# via google-auth
certifi==2021.5.30
certifi==2021.10.8
# via
# kubernetes
# requests
charset-normalizer==2.0.6
charset-normalizer==2.0.7
# via requests
click==8.0.1
click==8.0.3
# via -r requirements/base.in
google-auth==2.1.0
google-auth==2.3.1
# via kubernetes
idna==3.2
idna==3.3
# via requests
importlib-metadata==4.8.1
# via click
jinja2==3.0.1
jinja2==3.0.2
# via -r requirements/base.in
kubernetes==18.20.0
# via -r requirements/base.in
@ -40,11 +38,11 @@ pyasn1==0.4.8
# rsa
pyasn1-modules==0.2.8
# via google-auth
pycryptodome==3.10.1
pycryptodome==3.11.0
# via -r requirements/base.in
python-dateutil==2.8.2
# via kubernetes
pyyaml==5.4.1
pyyaml==6.0
# via
# -r requirements/base.in
# kubernetes
@ -58,24 +56,19 @@ rsa==4.7.2
# via google-auth
six==1.16.0
# via
# google-auth
# kubernetes
# python-dateutil
toml==0.10.2
# via mypy
typed-ast==1.4.3
# via mypy
typing-extensions==3.10.0.2
# via
# importlib-metadata
# mypy
# via mypy
urllib3==1.26.7
# via
# kubernetes
# requests
websocket-client==1.2.1
# via kubernetes
zipp==3.6.0
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
# setuptools

64
requirements/dev.txt
View File

@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile with python 3.7
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
# pip-compile requirements/dev.in
@ -8,53 +8,49 @@ altgraph==0.17.2
# via pyinstaller
appdirs==1.4.4
# via -r requirements/base.txt
astroid==2.8.0
astroid==2.8.3
# via pylint
black==21.9b0
# via -r requirements/dev.in
bleach==4.1.0
# via readme-renderer
cachetools==4.2.2
cachetools==4.2.4
# via
# -r requirements/base.txt
# google-auth
certifi==2021.5.30
certifi==2021.10.8
# via
# -r requirements/base.txt
# kubernetes
# requests
cffi==1.14.6
cffi==1.15.0
# via cryptography
charset-normalizer==2.0.6
charset-normalizer==2.0.7
# via
# -r requirements/base.txt
# requests
click==8.0.1
click==8.0.3
# via
# -r requirements/base.txt
# black
# pip-tools
colorama==0.4.4
# via twine
cryptography==3.4.8
cryptography==35.0.0
# via secretstorage
docutils==0.17.1
# via readme-renderer
google-auth==2.1.0
google-auth==2.3.1
# via
# -r requirements/base.txt
# kubernetes
idna==3.2
idna==3.3
# via
# -r requirements/base.txt
# requests
importlib-metadata==4.8.1
# via
# -r requirements/base.txt
# click
# keyring
# pep517
# pyinstaller
# twine
isort==5.9.3
# via pylint
@ -62,7 +58,7 @@ jeepney==0.7.1
# via
# keyring
# secretstorage
jinja2==3.0.1
jinja2==3.0.2
# via -r requirements/base.txt
keyring==23.2.1
# via twine
@ -91,13 +87,13 @@ packaging==21.0
# via bleach
pathspec==0.9.0
# via black
pep517==0.11.0
pep517==0.12.0
# via pip-tools
pip-tools==6.3.0
pip-tools==6.4.0
# via -r requirements/dev.in
pkginfo==1.7.1
# via twine
platformdirs==2.3.0
platformdirs==2.4.0
# via
# black
# pylint
@ -112,7 +108,7 @@ pyasn1-modules==0.2.8
# google-auth
pycparser==2.20
# via cffi
pycryptodome==3.10.1
pycryptodome==3.11.0
# via -r requirements/base.txt
pygments==2.10.0
# via readme-renderer
@ -122,19 +118,19 @@ pyinstaller-hooks-contrib==2021.3
# via pyinstaller
pylint==2.11.1
# via -r requirements/dev.in
pyparsing==2.4.7
pyparsing==3.0.1
# via packaging
python-dateutil==2.8.2
# via
# -r requirements/base.txt
# kubernetes
pyyaml==5.4.1
pyyaml==6.0
# via
# -r requirements/base.txt
# kubernetes
readme-renderer==29.0
readme-renderer==30.0
# via twine
regex==2021.8.28
regex==2021.10.23
# via black
requests==2.26.0
# via
@ -161,15 +157,15 @@ six==1.16.0
# via
# -r requirements/base.txt
# bleach
# google-auth
# kubernetes
# python-dateutil
# readme-renderer
toml==0.10.2
# via
# -r requirements/base.txt
# mypy
# pylint
tomli==1.2.1
tomli==1.2.2
# via
# black
# pep517
@ -177,22 +173,15 @@ tqdm==4.62.3
# via twine
twine==3.4.2
# via -r requirements/dev.in
typed-ast==1.4.3
# via
# -r requirements/base.txt
# astroid
# black
# mypy
types-pyyaml==5.4.10
types-pyyaml==6.0.0
# via -r requirements/dev.in
types-setuptools==57.4.0
types-setuptools==57.4.2
# via -r requirements/dev.in
typing-extensions==3.10.0.2
# via
# -r requirements/base.txt
# astroid
# black
# importlib-metadata
# mypy
# pylint
urllib3==1.26.7
@ -208,13 +197,10 @@ websocket-client==1.2.1
# kubernetes
wheel==0.37.0
# via pip-tools
wrapt==1.12.1
wrapt==1.13.2
# via astroid
zipp==3.6.0
# via
# -r requirements/base.txt
# importlib-metadata
# pep517
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
# pip

38
requirements/docs.txt
View File

@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile with python 3.7
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
# pip-compile requirements/docs.in
@ -10,20 +10,20 @@ appdirs==1.4.4
# via -r requirements/base.txt
babel==2.9.1
# via sphinx
cachetools==4.2.2
cachetools==4.2.4
# via
# -r requirements/base.txt
# google-auth
certifi==2021.5.30
certifi==2021.10.8
# via
# -r requirements/base.txt
# kubernetes
# requests
charset-normalizer==2.0.6
charset-normalizer==2.0.7
# via
# -r requirements/base.txt
# requests
click==8.0.1
click==8.0.3
# via
# -r requirements/base.txt
# sphinx-click
@ -32,21 +32,17 @@ docutils==0.17.1
# sphinx
# sphinx-click
# sphinx-rtd-theme
google-auth==2.1.0
google-auth==2.3.1
# via
# -r requirements/base.txt
# kubernetes
idna==3.2
idna==3.3
# via
# -r requirements/base.txt
# requests
imagesize==1.2.0
# via sphinx
importlib-metadata==4.8.1
# via
# -r requirements/base.txt
# click
jinja2==3.0.1
jinja2==3.0.2
# via
# -r requirements/base.txt
# sphinx
@ -77,19 +73,19 @@ pyasn1-modules==0.2.8
# via
# -r requirements/base.txt
# google-auth
pycryptodome==3.10.1
pycryptodome==3.11.0
# via -r requirements/base.txt
pygments==2.10.0
# via sphinx
pyparsing==2.4.7
pyparsing==3.0.1
# via packaging
python-dateutil==2.8.2
# via
# -r requirements/base.txt
# kubernetes
pytz==2021.1
pytz==2021.3
# via babel
pyyaml==5.4.1
pyyaml==6.0
# via
# -r requirements/base.txt
# kubernetes
@ -110,6 +106,7 @@ rsa==4.7.2
six==1.16.0
# via
# -r requirements/base.txt
# google-auth
# kubernetes
# python-dateutil
snowballstemmer==2.1.0
@ -139,14 +136,9 @@ toml==0.10.2
# via
# -r requirements/base.txt
# mypy
typed-ast==1.4.3
# via
# -r requirements/base.txt
# mypy
typing-extensions==3.10.0.2
# via
# -r requirements/base.txt
# importlib-metadata
# mypy
urllib3==1.26.7
# via
@ -157,10 +149,6 @@ websocket-client==1.2.1
# via
# -r requirements/base.txt
# kubernetes
zipp==3.6.0
# via
# -r requirements/base.txt
# importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
# setuptools

14
tutor/commands/local.py
View File

@ -8,6 +8,7 @@ from .. import env as tutor_env
from .. import fmt
from ..types import get_typed, Config
from .. import utils
from .. import exceptions
from . import compose
from .config import save as config_save_command
from .context import Context
@ -44,6 +45,19 @@ def local(context: Context) -> None:
@click.option("-p", "--pullimages", is_flag=True, help="Update docker images")
@click.pass_context
def quickstart(context: click.Context, non_interactive: bool, pullimages: bool) -> None:
try:
utils.check_macos_memory()
except exceptions.TutorError as e:
fmt.echo_alert(
"""Could not verify sufficient RAM allocation in Docker:
{}
Tutor may not work if Docker is configured with < 4 GB RAM. Please follow instructions from:
https://docs.tutor.overhang.io/install.html
""".format(
str(e)
)
)
if tutor_env.needs_major_upgrade(context.obj.root):
click.echo(fmt.title("Upgrading from an older release"))
context.invoke(

71
tutor/utils.py
View File

@ -11,30 +11,6 @@ from typing import List, Tuple
import click
# We monkey-patch the pycryptodome library to address a segmentation fault issue
# that affects a specific architecture
# We can get rid of this ugly patch when the following issue is resolved:
# https://github.com/Legrandin/pycryptodome/issues/506
# When running Mac OS ...
if sys.platform == "darwin":
import platform
# ... with ARM processor (M1)...
if platform.machine() == "arm64":
# ... if the GMP module is installed...
try:
import Crypto.Math._IntegerGMP
# ... monkey-patch the library to not use GMP
import Crypto.Math.Numbers
import Crypto.Math._IntegerNative
Crypto.Math.Numbers.Integer = Crypto.Math._IntegerNative.IntegerNative
Crypto.Math.Numbers._implementation = {}
except (ImportError, OSError, AttributeError):
pass
from Crypto.Protocol.KDF import bcrypt, bcrypt_check
from Crypto.PublicKey import RSA
from Crypto.PublicKey.RSA import RsaKey
@ -244,3 +220,50 @@ def check_output(*command: str) -> bytes:
raise exceptions.TutorError(
"Command failed: {}".format(" ".join(command))
) from e
def check_macos_memory() -> None:
"""
Try to check that the RAM allocated to the Docker VM on macOS is at least 4 GB.
"""
if sys.platform != "darwin":
return
settings_path = os.path.expanduser(
"~/Library/Group Containers/group.com.docker/settings.json"
)
try:
with open(settings_path) as fp:
data = json.load(fp)
memory_mib = int(data["memoryMiB"])
except OSError as e:
raise exceptions.TutorError(
"Error accessing {}: [{}] {}".format(settings_path, e.errno, e.strerror)
) from e
except json.JSONDecodeError as e:
raise exceptions.TutorError(
"Error reading {}: invalid JSON: {} [{}:{}]".format(
settings_path, e.msg, e.lineno, e.colno
)
) from e
except (ValueError, TypeError, OverflowError) as e:
# ValueError from open() indicates an encoding error
raise exceptions.TutorError(
"Text encoding error or unexpected JSON data: in {}: {}".format(
settings_path, str(e)
)
) from e
except KeyError as e:
# Value is absent (Docker creates the file with the default setting of 2048 explicitly
# written in, so we shouldn't need to assume a default value here.)
raise exceptions.TutorError(
"key 'memoryMiB' not found in {}".format(settings_path)
) from e
if memory_mib < 4096:
raise exceptions.TutorError(
"Docker is configured to allocate {} MiB RAM, less than the recommended {} MiB".format(
memory_mib, 4096
)
)