The curator:
- A word JMdict says is usually written in kana gets a kana headword,
even where the JLPT lists give kanji (その, not 其の), and a rare or
search-only spelling is never the headword while a usual reading
exists (そして, not 而して); 9,611 headwords change, 96 of them N5.
- Keys hold full-width letters as typed (Ｔシャツ as tシャツ), デュ/テュ/
フュ/イェ romanised, long a, i and e typed once as loose keys (kado for
カード), and a blank JLPT line is skipped.
The server:
- A search is one query ordered by tier, then rank: a loose or prefix
match no longer pushes an exact one out of the list (75-150 ms over
all of JMdict, before 150-450).
- Queries find long vowels with macrons spelt おお or ー and typed once
(ōkii, kōhī), ん typed n' or nn, and full-width letters.
- Pages of a label are ordered by rank and id, so entries that share a
rank are neither repeated nor skipped.
- The loader never throws (an unreadable folder ended the server from
the background), refuses two files of one source rather than loading
both at every start, reads no rows back when replacing a source, and
takes its time from clock.
- Dictionary files stay out of the Docker build context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
A package of its own in the workspace, without Serverpod (AGPL, like the
tooling): an adapter per source writes one format for all, documented in
its README, which the server will load. The format is JSON Lines: a
header (source, version, languages, licence, credits), then entries with
a headword and reading, labels (`common`, `jlpt:n5`), a rank, the keys a
search matches, and the entry as cards take it. Everything specific to a
language happens in its adapter (Japanese: forms, readings in kana and
Hepburn romaji, long vowels typed once, meanings without "to" and
articles), so the server's search knows no language.
JMdict comes as jmdict-simplified's JSON, read one entry at a time: all
218,863 entries in some 20 seconds and 300 MB, 16.8 MB gzipped. The JLPT
lists (yomitan-jlpt-vocab, CC BY-SA 4.0, from Waller's CC BY lists) add
levels by JMdict entry, the easiest when a word is on several, and give
the headword its common spelling. Words rank N5 to N1, then the other
common words, then the rest.
The server's image leaves the package out, as it does the client's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
