halkyon-days/halkyon
Watch
1
0
Fork
You've already forked halkyon
 0
Code Issues Pull requests Projects Releases 37 Packages Wiki Activity Actions

feat(curator): turn JMdict and the JLPT lists into dictionary files

Browse source 
A package of its own in the workspace, without Serverpod (AGPL, like the
tooling): an adapter per source writes one format for all, documented in
its README, which the server will load. The format is JSON Lines: a
header (source, version, languages, licence, credits), then entries with
a headword and reading, labels (`common`, `jlpt:n5`), a rank, the keys a
search matches, and the entry as cards take it. Everything specific to a
language happens in its adapter (Japanese: forms, readings in kana and
Hepburn romaji, long vowels typed once, meanings without "to" and
articles), so the server's search knows no language.

JMdict comes as jmdict-simplified's JSON, read one entry at a time: all
218,863 entries in some 20 seconds and 300 MB, 16.8 MB gzipped. The JLPT
lists (yomitan-jlpt-vocab, CC BY-SA 4.0, from Waller's CC BY lists) add
levels by JMdict entry, the easiest when a word is on several, and give
the headword its common spelling. Words rank N5 to N1, then the other
common words, then the rest.

The server's image leaves the package out, as it does the client's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-09 00:25:16 +02:00
commit a875a91f40
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
1 changed files with 1664 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

16
halkyon_learn_server/Dockerfile
View file

@ -14,14 +14,16 @@ WORKDIR /app
COPY pubspec.yaml pubspec.lock ./
COPY halkyon_learn_server halkyon_learn_server
# The workspace root with only the server package: the client package isn't needed to
# build the server, and the Flutter app would need Flutter. Everything else stays as the
# repository has it, above all the dependency overrides, without which the offline sync
# package's exact Serverpod pin can't be resolved. The check fails the build loudly if
# the root's member list changes shape.
RUN sed -i -e '/^ - halkyon_learn_client$/d' -e '/^ - halkyon_learn_flutter$/d' pubspec.yaml \
# The workspace root with only the server package: neither the client package nor the
# curator is needed to build the server, and the Flutter app would need Flutter.
# Everything else stays as the repository has it, above all the dependency overrides,
# without which the offline sync package's exact Serverpod pin can't be resolved. The
# check fails the build loudly if the root's member list changes shape.
RUN sed -i -e '/^ - halkyon_learn_client$/d' -e '/^ - halkyon_learn_flutter$/d' \
-e '/^ - halkyon_learn_curator$/d' pubspec.yaml \
&& grep -q '^ - halkyon_learn_server$' pubspec.yaml \
&& ! grep -q -e 'halkyon_learn_client' -e 'halkyon_learn_flutter' pubspec.yaml
&& ! grep -q -e 'halkyon_learn_client' -e 'halkyon_learn_flutter' \
-e 'halkyon_learn_curator' pubspec.yaml
RUN dart pub get
WORKDIR /app/halkyon_learn_server
RUN dart build cli --target bin/main.dart --output build