Make it possible to persist lms/cms logs to files

By symlinking /dev/stdout to a .log file, we make it possible for users
to manually persist lms/cms logs to a file.

docs/local.rst

@@ -20,7 +20,7 @@ All-in-one command
 A fully-functional platform can be configured and run in one command::
 
     tutor local quickstart
-    
+
 But you may want to run commands one at a time: it's faster when you need to run only part of the local deployment process, and it helps you understand how your platform works. In the following we decompose the ``quickstart`` command.
 
 Configuration
@@ -187,6 +187,28 @@ The default settings module loaded by ``edx-platform`` is ``tutor.production``.
 
 Of course, your settings should be compatible with the docker install. You can get some inspiration from the ``production.py`` settings modules generated by Tutor, or just import it as a base by adding ``from .production import *`` at the top of ``mysettings.py``.
 
+
+Saving logs to files instead of sending them to Docker
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default tracking logs and app logs are sent to docker, and are visible with ``tutor local logs``. Another option is to persist logs in a directory alongside the database data.
+To direct logs from an openedx component (lms, cms or workers) edit the docker-compose file at ``$(tutor config printroot)/env/local/docker-compose.yml`` and add the followinf line to the lms/cms and worker containers, replaceing SERVICE_NAME with the relevant service name (lms, cms, lms-worker, cms-worker):
+
+    volumes:
+      - ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/
+      - ../apps/openedx/settings/cms/:/openedx/edx-platform/cms/envs/tutor/
+      ...
+      - ../../data/logs/SERVICE_NAME:/openedx/logs # <-- Add this line
+
+If you also want logs from nginx to be saved to disk instead of sent to docker, add the following to the nginx container in ``$(tutor config printroot)/env/local/docker-compose.yml``:
+
+    volumes:
+      - ../apps/nginx:/etc/nginx/conf.d/
+      - ../../data/openedx:/var/www/openedx:ro
+      ...
+      - ../../data/logs/lms:/var/log/nginx # <-- Add this line
+
+
 Upgrading from earlier versions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

tutor/templates/apps/openedx/settings/cms/common.py

@@ -9,10 +9,16 @@ VIDEO_IMAGE_SETTINGS["STORAGE_KWARGS"]["location"] = MEDIA_ROOT
 VIDEO_TRANSCRIPTS_SETTINGS["STORAGE_KWARGS"]["location"] = MEDIA_ROOT
 
 # Change syslog-based loggers which don't work inside docker containers
-LOGGING["handlers"]["local"] = {"class": "logging.NullHandler"}
+LOGGING["handlers"]["local"] = {
+    "class": "logging.handlers.WatchedFileHandler",
+    "filename": "/openedx/logs/edx.log",
+    "formatter": "standard",
+}
+
 LOGGING["handlers"]["tracking"] = {
     "level": "DEBUG",
-    "class": "logging.StreamHandler",
+    "class": "logging.handlers.WatchedFileHandler",
+    "filename": "/openedx/logs/tracking.log",
     "formatter": "standard",
 }
 

tutor/templates/apps/openedx/settings/lms/common.py

@@ -11,10 +11,16 @@ VIDEO_IMAGE_SETTINGS["STORAGE_KWARGS"]["location"] = MEDIA_ROOT
 VIDEO_TRANSCRIPTS_SETTINGS["STORAGE_KWARGS"]["location"] = MEDIA_ROOT
 
 # Change syslog-based loggers which don't work inside docker containers
-LOGGING["handlers"]["local"] = {"class": "logging.NullHandler"}
+LOGGING["handlers"]["local"] = {
+    "class": "logging.handlers.WatchedFileHandler",
+    "filename": "/openedx/logs/edx.log",
+    "formatter": "standard",
+}
+
 LOGGING["handlers"]["tracking"] = {
     "level": "DEBUG",
-    "class": "logging.StreamHandler",
+    "class": "logging.handlers.WatchedFileHandler",
+    "filename": "/openedx/logs/tracking.log",
     "formatter": "standard",
 }
 

tutor/templates/build/openedx/Dockerfile

@@ -93,6 +93,11 @@ ENV SETTINGS tutor.production
 # Entrypoint will fix permissions of all files and run commands as openedx
 ENTRYPOINT ["docker-entrypoint.sh"]
 
+# Configure logging
+RUN mkdir -p /openedx/logs \
+    && ln -s /dev/stdout /openedx/logs/tracking.log \
+    && ln -s /dev/stderr /openedx/logs/edx.log
+
 # Run server
 COPY gunicorn_conf.py /openedx/gunicorn_conf.py
 EXPOSE 8000

tutor/templates/local/docker-compose.yml

@@ -150,7 +150,7 @@ services:
       SERVICE_VARIANT: lms
       SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
       C_FORCE_ROOT: "1" # run celery tasks as root #nofear
-    command: ./manage.py lms celery worker --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild 100
+    command: ./manage.py lms celery worker --logfile=/openedx/logs/edx.log --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild 100
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/
@@ -168,7 +168,7 @@ services:
       SERVICE_VARIANT: cms
       SETTINGS: ${EDX_PLATFORM_SETTINGS:-tutor.production}
       C_FORCE_ROOT: "1" # run celery tasks as root #nofear
-    command: ./manage.py cms celery worker --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100
+    command: ./manage.py cms celery worker --logfile=/openedx/logs/edx.log --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms/:/openedx/edx-platform/lms/envs/tutor/