Scaffold the curator: jmdict-simplified → Halkyon archive (V3-1) #2

Merged

raisondetredev merged 1 commit from refs/pull/2/head into main

2026-06-04 22:43:38 +02:00

raisondetredev commented

2026-06-04 22:43:22 +02:00

(Migrated from codeberg.org)

First working slice of the curator (roadmap §27 / V3). A Dart CLI that
reads a jmdict-simplified JSON file and emits a Halkyon archive
(.halkyon) of vocabulary flashcards — the inverse end of the import
flow shipped in halkyon-learn.

Pipeline:

jmdict_simplified.dart — minimal parser for the fields we need:
headword forms (+ common flag), kana readings, per-sense
parts-of-speech + English glosses.
part_of_speech_tags.dart — maps JMdict POS entity codes to
learner-facing tag names (v5* → godan verb + verb; v1 → ichidan
verb + verb; adj-i → i-adjective + adjective; adj-na →
na-adjective + adjective; n → noun; …). Partial by design — codes
without a useful tag contribute nothing rather than leaking raw
codes.
vocabulary_curator.dart — JmdictDictionary → HalkyonArchive: one
studyset of vocabulary cards (expression / kana reading / meanings;
kana-only words get an empty reading), tagged with their POS + a
common tag. --limit and --common-only narrow the output.
halkyon_archive.dart — emit-side DTOs byte-compatible with
halkyon-learn's archive reader; flashcard_content.dart mirrors
plainTextToFlashcardContent so imported cards open cleanly.

Licensing: JMdict is © EDRDG, CC BY-SA 4.0. Every emitted studyset is
stamped license: CC-BY-SA-4.0 with an EDRDG attribution line in its
description. (POS + common are intrinsic to JMdict; JLPT tags via
yomitan-jlpt-vocab are the next slice, V3-2.)

Tests cover the parser (common flag, non-English gloss filtering), the
curator (kanji vs kana-only mapping, meaning dedup, POS + common tags,
tag dedup/reuse, limit, common-only) and a JSON round-trip. The CLI
runs end to end on a sample file.

Signed-off-by: Tizian 「ティツィアーン」 raisondetredev@pm.me

First working slice of the curator (roadmap §27 / V3). A Dart CLI that reads a jmdict-simplified JSON file and emits a Halkyon archive (.halkyon) of `vocabulary` flashcards — the inverse end of the import flow shipped in halkyon-learn. Pipeline: * jmdict_simplified.dart — minimal parser for the fields we need: headword forms (+ common flag), kana readings, per-sense parts-of-speech + English glosses. * part_of_speech_tags.dart — maps JMdict POS entity codes to learner-facing tag names (v5* → godan verb + verb; v1 → ichidan verb + verb; adj-i → i-adjective + adjective; adj-na → na-adjective + adjective; n → noun; …). Partial by design — codes without a useful tag contribute nothing rather than leaking raw codes. * vocabulary_curator.dart — JmdictDictionary → HalkyonArchive: one studyset of vocabulary cards (expression / kana reading / meanings; kana-only words get an empty reading), tagged with their POS + a `common` tag. --limit and --common-only narrow the output. * halkyon_archive.dart — emit-side DTOs byte-compatible with halkyon-learn's archive reader; flashcard_content.dart mirrors plainTextToFlashcardContent so imported cards open cleanly. Licensing: JMdict is © EDRDG, CC BY-SA 4.0. Every emitted studyset is stamped `license: CC-BY-SA-4.0` with an EDRDG attribution line in its description. (POS + common are intrinsic to JMdict; JLPT tags via yomitan-jlpt-vocab are the next slice, V3-2.) Tests cover the parser (common flag, non-English gloss filtering), the curator (kanji vs kana-only mapping, meaning dedup, POS + common tags, tag dedup/reuse, limit, common-only) and a JSON round-trip. The CLI runs end to end on a sample file. Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>