feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update them at least monthly. The instance now does it alone: a `dictionaries` service in compose, built from the curator (halkyon_learn_curator/ Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate update --every 7`, downloading EDRDG's files of the day once its files are a week old and replacing them in a volume the server mounts read-only. The curator never enters the server's image or process (decided 2026-10-09: a container of its own, weekly). The server rereads the folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed, and loads now run one at a time. redeploy.sh builds both images. An operator who'd rather supply the files removes the service and mounts deploy/dictionaries/ into the server, as compose.yaml says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
82b38fadd8
commit
25e01ad128
3 changed files with 211 additions and 32 deletions
|
|
@ -1,4 +1,5 @@
|
|||
# The Halkyon Learn instance: Caddy in front, the Serverpod server, PostgreSQL.
|
||||
# The Halkyon Learn instance: Caddy in front, the Serverpod server, PostgreSQL, and
|
||||
# the dictionaries kept current beside them.
|
||||
#
|
||||
# cd deploy
|
||||
# cp .env.example .env && chmod 600 .env # then fill in the secrets
|
||||
|
|
@ -57,15 +58,36 @@ services:
|
|||
FILE_LIMIT_ANONYMOUS_MB: ${FILE_LIMIT_ANONYMOUS_MB:-}
|
||||
FILE_LIMIT_IDENTIFIED_MB: ${FILE_LIMIT_IDENTIFIED_MB:-}
|
||||
FILE_LIMIT_INSTANCE_MB: ${FILE_LIMIT_INSTANCE_MB:-}
|
||||
# The dictionary files it loads at start, those that changed since (see the
|
||||
# server's README); read only, and empty unless the operator adds some.
|
||||
# The dictionary files it loads at start and rereads daily, those that changed
|
||||
# (see the server's README): those the dictionaries service keeps current, read
|
||||
# only. To supply files yourself instead, remove that service and mount
|
||||
# ./dictionaries here (`- ./dictionaries:/app/dictionaries:ro`).
|
||||
volumes:
|
||||
- ./dictionaries:/app/dictionaries:ro
|
||||
- dictionaries:/app/dictionaries:ro
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
logging: *logging
|
||||
|
||||
# Keeps the dictionary files current from EDRDG's files of the day, once they are a
|
||||
# week old (EDRDG's licence asks it of servers showing its dictionaries at least
|
||||
# monthly; the curator's README): in a container of its own, the curator never in
|
||||
# the server's image or process. It only downloads (some 25 MB a week) and writes
|
||||
# into the volume; the files are never backed up, as they can be had again.
|
||||
dictionaries:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: halkyon_learn_curator/Dockerfile
|
||||
image: halkyon-learn-dictionaries
|
||||
restart: unless-stopped
|
||||
# Docker's stop signal reaches the curator through a small init.
|
||||
init: true
|
||||
# A conversion takes some 400 MB for a minute a week.
|
||||
mem_limit: 1g
|
||||
volumes:
|
||||
- dictionaries:/dictionaries
|
||||
logging: *logging
|
||||
|
||||
postgres:
|
||||
image: postgres:16
|
||||
restart: unless-stopped
|
||||
|
|
@ -88,4 +110,5 @@ services:
|
|||
volumes:
|
||||
caddy_data:
|
||||
caddy_config:
|
||||
dictionaries:
|
||||
postgres_data:
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# Dictionary files
|
||||
|
||||
Dictionary files (`*.jsonl`, `*.jsonl.gz`) for the server to load at start; see
|
||||
the server's README. They are downloads turned into files by `halkyon_learn_curator`,
|
||||
not kept in git.
|
||||
For an instance that supplies its dictionary files itself, rather than the
|
||||
`dictionaries` service keeping them current (`compose.yaml`): put the files
|
||||
(`*.jsonl`, `*.jsonl.gz`, made by `halkyon_learn_curator`) here and mount this
|
||||
folder into the server as the comment there says. Mind EDRDG's licence: a server
|
||||
showing its dictionaries must update them at least once a month. Not kept in git.
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ main() {
|
|||
trap 'git reset -q --hard "$deployed"; echo "The checkout is back at $deployed." >&2' EXIT
|
||||
cd deploy
|
||||
# Build while the running server keeps serving; then dump, migrate, switch.
|
||||
docker compose build -q server
|
||||
docker compose build -q server dictionaries
|
||||
dump
|
||||
# Migrations as a run of their own, before the switch: Serverpod's maintenance role
|
||||
# applies them and exits with 0 only if that worked. Each migration is one
|
||||
|
|
|
|||
Loading…
Reference in a new issue