halkyon-days/halkyon
Watch
1
0
Fork
You've already forked halkyon
 0
Code Issues Pull requests Projects Releases 37 Packages Wiki Activity Actions

feat(deploy): instances keep their dictionaries current, weekly, by themselves

Browse source 
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.

An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-09 16:52:37 +02:00
commit 25e01ad128
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
1 changed files with 211 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

27
halkyon_learn_server/README.md
View file

@ -14,16 +14,25 @@ When you are finished, you can shut down the running server with `Q`.
The dictionaries people look words up in (JMdict, …) are reference data the
server holds for everyone. They come as dictionary files, which
`halkyon_learn_curator` makes from the sources' downloads (its README says how).
At every start, the server loads the files in its `dictionaries` folder that
changed since they were last loaded (by their SHA-256): in the background, a
source at a time, and all at once, so searches meanwhile still find the old
entries. A file it can't read is logged and left out.
At every start, and daily at 04:07, the server loads the files in its
`dictionaries` folder that changed since they were last loaded (by their
SHA-256): in the background, a source at a time, and all at once, so searches
meanwhile still find the old entries. A file it can't read is logged and left
out. `DICTIONARIES_DIR` names another folder.
- **Development:** put the file into `halkyon_learn_server/dictionaries/` and
restart `serverpod start`.
- **An instance:** put it into `deploy/dictionaries/` beside `compose.yaml`
(mounted read-only as `/app/dictionaries`) and restart the server container.
`DICTIONARIES_DIR` names another folder.
EDRDG's licence asks a server showing its dictionaries to update them at least
once a month:
- **An instance** keeps them current by itself: the `dictionaries` service in
`deploy/compose.yaml` (the curator, in a container of its own) downloads
EDRDG's files of the day once its files are a week old, converts them, and
replaces them in a volume the server reads (`curate update`, the curator's
README); the server takes them up the next morning. To supply files yourself
instead, remove the service and mount `deploy/dictionaries/` into the server
(`compose.yaml` says how), and update them monthly.
- **Development:** `dart run halkyon_learn_curator:curate update --into
../halkyon_learn_server/dictionaries` in the curator's folder (about a
minute), then restart `serverpod start`, or wait for the morning.
JMdict in English, with the JLPT levels and Tatoeba's example sentences, is
about 20 MB gzipped and takes the server half a minute to load (219,000