fix: infinite cache growth

See the discussion here: https://github.com/overhangio/tutor/pull/984
And the upstream PR here: https://github.com/openedx/edx-platform/pull/34210

The tl;dr is that the Redis course structure cache was growing without
bounds. While the upstream fix should resolve that issue, we decided
that Tutor should have a maxmemory limit and an eviction policy set for
operational safety.

Thus, Redis now has a 4gb maxmemory. If you need more memory on your
instance, you should implement the "redis-conf" patch.

To manually expire existing keys, run:

    tutor local run cms ./manage.py cms shell -c "from django.core.cache import caches; c = caches['course_structure_cache']; [c.expire(key, 604800) for key in c.keys('*')]"

changelog.d/20240208_111236_regis_infinite_course_cache.md

@@ -0,0 +1,4 @@
+- 💥[Bugfix] Prevent infinite growth of course structure cache in Redis. (by @regisb)
+  - Redis is now configured with a maximum memory size of 4GB. If this is too low for your platform, you should increase this value using the new "redis-conf" patch.
+  - Make sure that course structure cache keys have an actual timeout.
+- [Feature] Introduce the "redis-conf" patch. (by @regisb)

docs/reference/patches.rst

@@ -376,6 +376,13 @@ File: ``apps/openedx/settings/lms/production.py``
 
 Python-formatted LMS settings in production. Values defined here override the values from :patch:`openedx-lms-common-settings`.
 
+``redis-conf``
+==============
+
+File: ``apps/redis/redis.conf``
+
+Implement this patch to override hard-coded Redis configuration values. See the `Redis configuration reference <https://redis.io/docs/management/config-file/>`__`.
+
 ``uwsgi-config``
 ================
 

requirements/dev.txt

@@ -42,7 +42,7 @@ click-log==0.4.0
     # via scriv
 coverage==7.3.2
     # via -r requirements/dev.in
-cryptography==41.0.7
+cryptography==42.0.3
     # via secretstorage
 dill==0.3.7
     # via pylint

tutor/templates/apps/openedx/settings/partials/common_all.py

@@ -73,7 +73,7 @@ CACHES = {
     },
     "course_structure_cache": {
         "KEY_PREFIX": "course_structure",
-        "TIMEOUT": 7200,
+        "TIMEOUT": 604800, # 1 week
         "BACKEND": "django_redis.cache.RedisCache",
         "LOCATION": "redis://{% if REDIS_USERNAME and REDIS_PASSWORD %}{{ REDIS_USERNAME }}:{{ REDIS_PASSWORD }}{% endif %}@{{ REDIS_HOST }}:{{ REDIS_PORT }}/{{ OPENEDX_CACHE_REDIS_DB }}",
     },

tutor/templates/apps/redis/redis.conf

@@ -39,3 +39,10 @@ auto-aof-rewrite-percentage 100
 auto-aof-rewrite-min-size 64mb
 aof-load-truncated yes
 aof-use-rdb-preamble yes
+
+############################## MEMORY MANAGEMENT ################################
+
+maxmemory 4gb
+maxmemory-policy allkeys-lru
+
+{{ patch("redis-conf") }}

tutor/templates/build/openedx/Dockerfile

@@ -51,6 +51,9 @@ RUN git config --global user.email "tutor@overhang.io" \
 {{ patch("openedx-dockerfile-git-patches-default") }}
 {%- else %}
 # Patch edx-platform
+# Prevent course structure cache infinite growth
+# https://github.com/openedx/edx-platform/pull/34210
+RUN curl -fsSL https://github.com/openedx/edx-platform/commit/ad201cd664b6c722cbefcbda23ae390c06daf621.patch | git am
 {%- endif %}
 
 {# Example: RUN curl -fsSL https://github.com/openedx/edx-platform/commit/<GITSHA1>.patch | git am #}