feat(deploy): instances keep their dictionaries current, weekly, by themselves

EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.

An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>

halkyon_learn_server/README.md

@@ -14,16 +14,25 @@ When you are finished, you can shut down the running server with `Q`.
 The dictionaries people look words up in (JMdict, …) are reference data the
 server holds for everyone. They come as dictionary files, which
 `halkyon_learn_curator` makes from the sources' downloads (its README says how).
-At every start, the server loads the files in its `dictionaries` folder that
-changed since they were last loaded (by their SHA-256): in the background, a
-source at a time, and all at once, so searches meanwhile still find the old
-entries. A file it can't read is logged and left out.
+At every start, and daily at 04:07, the server loads the files in its
+`dictionaries` folder that changed since they were last loaded (by their
+SHA-256): in the background, a source at a time, and all at once, so searches
+meanwhile still find the old entries. A file it can't read is logged and left
+out. `DICTIONARIES_DIR` names another folder.
 
-- **Development:** put the file into `halkyon_learn_server/dictionaries/` and
-  restart `serverpod start`.
-- **An instance:** put it into `deploy/dictionaries/` beside `compose.yaml`
-  (mounted read-only as `/app/dictionaries`) and restart the server container.
-  `DICTIONARIES_DIR` names another folder.
+EDRDG's licence asks a server showing its dictionaries to update them at least
+once a month:
+
+- **An instance** keeps them current by itself: the `dictionaries` service in
+  `deploy/compose.yaml` (the curator, in a container of its own) downloads
+  EDRDG's files of the day once its files are a week old, converts them, and
+  replaces them in a volume the server reads (`curate update`, the curator's
+  README); the server takes them up the next morning. To supply files yourself
+  instead, remove the service and mount `deploy/dictionaries/` into the server
+  (`compose.yaml` says how), and update them monthly.
+- **Development:** `dart run halkyon_learn_curator:curate update --into
+  ../halkyon_learn_server/dictionaries` in the curator's folder (about a
+  minute), then restart `serverpod start`, or wait for the morning.
 
 JMdict in English, with the JLPT levels and Tatoeba's example sentences, is
 about 20 MB gzipped and takes the server half a minute to load (219,000

halkyon_learn_server/lib/server.dart

@@ -42,22 +42,25 @@ Future<void> run(List<String> args) async {
       .filePruning
       .pruneFiles();
 
-  // The dictionaries the operator put into the server's folder, those that
-  // changed since; in the background, as loading takes a minute or two.
+  // The dictionaries in the server's folder, those that changed since; in
+  // the background, as loading takes a minute or two. Then again daily, as
+  // the dictionaries service replaces them weekly.
   unawaited(_loadDictionaries(pod));
+  await pod.futureCalls.cancel(_dictionaryReload);
+  await pod.futureCalls
+      .callRecurring(identifier: _dictionaryReload)
+      .cron('7 4 * * *')
+      .dictionaryReload
+      .reloadDictionaries();
 }
 
-/// Loads the dictionaries the operator put into the server's folder (or
-/// `DICTIONARIES_DIR`). [loadDictionaries] throws nothing; a session that
-/// can't be had is reported here, as an error in the background would end
-/// the server.
+/// Loads the dictionaries in the server's folder ([dictionaryFolder]).
+/// [loadDictionaries] throws nothing; a session that can't be had is
+/// reported here, as an error in the background would end the server.
 Future<void> _loadDictionaries(Serverpod pod) async {
   try {
     await pod.withSession(
-      (session) => loadDictionaries(
-        session,
-        Directory(Platform.environment['DICTIONARIES_DIR'] ?? 'dictionaries'),
-      ),
+      (session) => loadDictionaries(session, dictionaryFolder()),
     );
   } on Object catch (error) {
     stderr.writeln('Loading the dictionaries failed: $error');
@@ -69,3 +72,6 @@ const _pruning = 'prune-abandoned-anonymous-people';
 
 /// The identifier of the daily pruning of files.
 const _filePruning = 'prune-unreferenced-files';
+
+/// The identifier of the daily rereading of the dictionaries.
+const _dictionaryReload = 'reload-dictionaries';

halkyon_learn_server/lib/src/dictionary/dictionary_loader.dart

@@ -15,6 +15,16 @@ const dictionaryFormatVersion = 1;
 /// How many entries go into the database at once.
 const _batch = 1000;
 
+/// The folder of dictionary files: `DICTIONARIES_DIR`, else `dictionaries`
+/// in the server's folder.
+Directory dictionaryFolder() =>
+    Directory(Platform.environment['DICTIONARIES_DIR'] ?? 'dictionaries');
+
+/// The loads asked for, one after another: two at once would both replace
+/// a changed source, the second failing on the first's rows (and logging
+/// it) for nothing.
+Future<void> _loads = Future.value();
+
 /// Loads every dictionary file in [directory] (`*.jsonl`, or gzipped
 /// `*.jsonl.gz`) that changed since it was loaded: the operator drops a
 /// file the curator wrote into the server's `dictionaries` folder, and the
@@ -25,8 +35,11 @@ const _batch = 1000;
 /// beside the new one) are refused: neither loads, as which is meant is
 /// unclear, and loading both would replace the source twice at every start.
 /// Nothing here throws: the server starts it in the background, where an
-/// error would end the process.
-Future<void> loadDictionaries(Session session, Directory directory) async {
+/// error would end the process. Loads run one at a time.
+Future<void> loadDictionaries(Session session, Directory directory) =>
+    _loads = _loads.then((_) => _loadFolder(session, directory));
+
+Future<void> _loadFolder(Session session, Directory directory) async {
   void failed(String message, Object error, StackTrace st) => session.log(
     message,
     level: LogLevel.error,

halkyon_learn_server/lib/src/dictionary/dictionary_reload_future_call.dart

@@ -0,0 +1,11 @@
+import 'package:halkyon_learn_server/src/dictionary/dictionary_loader.dart';
+import 'package:serverpod/serverpod.dart';
+
+/// Rereads the dictionary folder and loads the files that changed: the
+/// dictionaries service replaces them weekly (`deploy/compose.yaml`); the
+/// server schedules it daily when it starts (`server.dart`).
+class DictionaryReloadFutureCall extends FutureCall {
+  /// Loads the dictionary files that changed since they were loaded.
+  Future<void> reloadDictionaries(Session session) =>
+      loadDictionaries(session, dictionaryFolder());
+}

halkyon_learn_server/lib/src/generated/future_calls.dart

@@ -16,6 +16,7 @@ import 'package:clock/clock.dart' as _io0w16m8;
 import 'package:serverpod/serverpod.dart' as _is;
 import '../asset/file_pruning_future_call.dart' as _ie5h5r65;
 import '../auth/pruning_future_call.dart' as _igevg04y;
+import '../dictionary/dictionary_reload_future_call.dart' as _ikbvcxb6;
 
 /// Invokes a future call.
 typedef _InvokeFutureCall =
@@ -62,6 +63,8 @@ class FutureCalls extends _is.FutureCallDispatch<_FutureCallRef> {
       'FilePruningPruneFilesFutureCall': FilePruningPruneFilesFutureCall(),
       'PruningPruneAnonymousPeopleFutureCall':
           PruningPruneAnonymousPeopleFutureCall(),
+      'DictionaryReloadReloadDictionariesFutureCall':
+          DictionaryReloadReloadDictionariesFutureCall(),
     };
     _futureCallManager = futureCallManager;
     _serverId = serverId;
@@ -185,6 +188,10 @@ class _FutureCallRef {
   late final filePruning = _FilePruningFutureCallDispatcher(_invokeFutureCall);
 
   late final pruning = _PruningFutureCallDispatcher(_invokeFutureCall);
+
+  late final dictionaryReload = _DictionaryReloadFutureCallDispatcher(
+    _invokeFutureCall,
+  );
 }
 
 class _FilePruningFutureCallDispatcher {
@@ -213,6 +220,19 @@ class _PruningFutureCallDispatcher {
   }
 }
 
+class _DictionaryReloadFutureCallDispatcher {
+  _DictionaryReloadFutureCallDispatcher(this._invokeFutureCall);
+
+  final _InvokeFutureCall _invokeFutureCall;
+
+  Future<void> reloadDictionaries() {
+    return _invokeFutureCall(
+      'DictionaryReloadReloadDictionariesFutureCall',
+      null,
+    );
+  }
+}
+
 /// Prunes one batch and logs how many files went.
 class FilePruningPruneFilesFutureCall extends _is.FutureCall
     implements _is.InvokableFutureCall {
@@ -236,3 +256,15 @@ class PruningPruneAnonymousPeopleFutureCall extends _is.FutureCall
     await _igevg04y.PruningFutureCall().pruneAnonymousPeople(session);
   }
 }
+
+/// Loads the dictionary files that changed since they were loaded.
+class DictionaryReloadReloadDictionariesFutureCall extends _is.FutureCall
+    implements _is.InvokableFutureCall {
+  @override
+  _ida.Future<void> invoke(
+    _is.Session session,
+    _is.SerializableModel? object,
+  ) async {
+    await _ikbvcxb6.DictionaryReloadFutureCall().reloadDictionaries(session);
+  }
+}

halkyon_learn_server/test/integration/dictionary/dictionary_test.dart

@@ -395,6 +395,19 @@ void main() {
         expect(await search('|'), isEmpty);
       });
 
+      test('loads asked for at once leave the dictionary loaded once, '
+          'whole', () async {
+        write([taberu, terebi]);
+
+        await Future.wait([
+          loadDictionaries(session, folder),
+          loadDictionaries(session, folder),
+        ]);
+
+        expect(await DictionaryEntry.db.count(session), 2);
+        expect(await DictionarySource.db.count(session), 1);
+      });
+
       test('two files of one source are refused, and neither loads, as '
           'which one is meant is unclear', () async {
         write([taberu], name: 'jmdict-en-old.jsonl');

halkyon_learn_server/test/integration/test_tools/serverpod_test_tools.dart

@@ -228,6 +228,8 @@ class _FutureCalls {
   late final filePruning = _FilePruningFutureCall();
 
   late final pruning = _PruningFutureCall();
+
+  late final dictionaryReload = _DictionaryReloadFutureCall();
 }
 
 class _AssetEndpoint {
@@ -847,3 +849,20 @@ class _PruningFutureCall {
     }
   }
 }
+
+class _DictionaryReloadFutureCall {
+  Future<void> reloadDictionaries(
+    _ist.TestSessionBuilder sessionBuilder,
+  ) async {
+    var _localUniqueSession =
+        (sessionBuilder as _ist.InternalTestSessionBuilder).internalBuild();
+    try {
+      await _iip9210p.DictionaryReloadReloadDictionariesFutureCall().invoke(
+        _localUniqueSession,
+        null,
+      );
+    } finally {
+      await _localUniqueSession.close();
+    }
+  }
+}