feat(server): dictionaries, loaded from files and searched

The server now holds dictionaries for everyone: `dictionary_entry` and
`dictionary_source`, reference data owned by no one, never synced. It
loads the dictionary files in its `dictionaries` folder at every start,
those whose SHA-256 changed: in the background, each source replaced in
one transaction, so searches meanwhile find the old entries; a file it
can't read is logged and left out. All of JMdict loads in under half a
minute. On an instance the folder is deploy/dictionaries, mounted
read-only (compose.yaml); the server's README says how.

DictionaryEndpoint (signed in): the dictionaries with their credits; a
search (exact key, exact loose key, key starting with the query, loose
key starting with it, each in the source's order; katakana as hiragana,
macrons written out), some 150-450 ms over all of JMdict; and the
entries with a label, counted and a page at a time (all JLPT N5 words).

The curator now keeps spellings that aren't quite right apart, as
`looseKeys` (romaji with long vowels typed once): "sushi" finds 寿司
before 数詞 (すうし).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>

halkyon_learn_curator/README.md

@@ -51,6 +51,7 @@ The **first line** says what the file holds:
 | `labels` | What it can be picked by: `common`, `jlpt:n5` … `jlpt:n1` (later e.g. `hsk:3`, `topik:1`). |
 | `rank` | Its order among the source's entries, lower first; JMdict: N5 to N1 in their lists' order, then other common words, then the rest. |
 | `keys` | What a search matches, lower case: written forms, readings in their own script and romanised, meanings. Everything specific to a language happens here, so the server's search knows no language. |
+| `looseKeys` | What a search matches after every `keys` match: spellings people type that aren't quite right (romaji with long vowels typed once, `sushi` for すうし). Left out when empty. |
 | `entry` | The entry itself, as cards take it: `forms` and `readings` (`text`, `common`, the source's `tags`; for a reading the `forms` it belongs to), `senses` (`glosses`; the source's codes in `pos`, `misc`, `field`, `dialect`; `info`; the `forms` and `readings` a sense is limited to). Lists are left out when empty. |
 
 ## Adding a source

halkyon_learn_curator/lib/src/dictionary_file.dart

@@ -55,8 +55,10 @@ final class const DictionaryHeader({
 /// One entry of a dictionary file: its [id] in the source, the [headword]
 /// and its [reading] (null when the headword is its own reading), [labels]
 /// to choose entries by (`common`, `jlpt:n5`), a [rank] to order them by
-/// (lower first), the [keys] a search matches against (lower case), and
-/// the [entry] itself as the app turns it into a card.
+/// (lower first), the [keys] a search matches against (lower case), the
+/// [looseKeys] it matches after them (spellings people type that aren't
+/// quite right, as `sushi` for すうし), and the [entry] itself as the app
+/// turns it into a card.
 final class const DictionaryEntry({
   required final String id,
   required final String headword,
@@ -65,6 +67,7 @@ final class const DictionaryEntry({
   required final int rank,
   required final List<String> keys,
   required final Map<String, Object?> entry,
+  final List<String> looseKeys = const [],
 }) {
   Map<String, Object?> toJson() => {
     'id': id,
@@ -73,6 +76,7 @@ final class const DictionaryEntry({
     'labels': labels,
     'rank': rank,
     'keys': keys,
+    if (looseKeys.isNotEmpty) 'looseKeys': looseKeys,
     'entry': entry,
   };
 }

halkyon_learn_curator/lib/src/sources/jmdict.dart

@@ -124,10 +124,14 @@ DictionaryEntry jmdictEntry(
       form.text,
       toHiragana(form.text),
       toRomaji(form.text),
-      ?withShortVowels(toRomaji(form.text)),
     ],
     for (final gloss in glosses) ...glossKeys(gloss),
   };
+  // Long vowels typed once (`tokyo`), after the right spellings: `sushi`
+  // finds 寿司 (すし) before 数詞 (すうし).
+  final looseKeys = {
+    for (final form in kana) ?withShortVowels(toRomaji(form.text)),
+  }.difference(keys);
 
   return DictionaryEntry(
     id: id,
@@ -140,6 +144,7 @@ DictionaryEntry jmdictEntry(
       null => common ? 600000 : 900000,
     },
     keys: keys.toList(),
+    looseKeys: looseKeys.toList(),
     entry: {
       'forms': [
         for (final form in kanji)

halkyon_learn_curator/test/jmdict_test.dart

@@ -66,8 +66,15 @@ void main() {
     expect(word['keys'], containsAll(['テレビ', 'てれび', 'terebi', 'tv']));
   });
 
-  test('long vowels are found typed once', () {
-    expect(entry('東京')['keys'], containsAll(['toukyou', 'tokyo']));
+  test('long vowels typed once are loose keys, found after the right '
+      'spellings', () {
+    final tokyo = entry('東京');
+    expect(tokyo['keys'], containsAll(['toukyou', 'toukei']));
+    // Its other reading, とうけい, typed with one o.
+    expect(tokyo['looseKeys'], ['tokei']);
+    // "tokyo" is a key already, its meaning in English.
+    expect(tokyo['keys'], contains('tokyo'));
+    expect(entry('食べる').containsKey('looseKeys'), isFalse);
   });
 
   test('the reading is one that belongs to the headword', () {