Add dictionary_entries seed tool + share JMdict field extraction #8

Merged
raisondetredev merged 1 commit from refs/pull/8/head into main 2026-06-05 20:26:47 +02:00
raisondetredev commented 2026-06-05 20:26:37 +02:00 (Migrated from codeberg.org)

A new bin (halkyon_curator:seed_dictionary) loads a jmdict-simplified
JSON file and upserts it into the server-side dictionary_entries catalog
via PostgREST, batched, on the JMdict id primary key (merge-duplicates),
so re-runs are idempotent. The service-role key is read from
SUPABASE_SERVICE_ROLE_KEY (never argv); --dry-run builds and samples
rows without sending. Optional --jlpt-dir sets each row's level;
--common-only / --limit scope the seed.

DictionarySeedRow.fromWord builds a catalog row, reusing the same
expression/reading/meanings extraction as the card curator — now pulled
out into vocabulary_extraction.dart so the two can't drift (an 'Add to
my deck' card must match the curated deck's card for the same word).
search_blob holds every surface form, lowercased, for trigram lookup;
English glosses are excluded by design. Adds http; 11 new tests green,
existing curator suite unaffected by the extraction refactor.

Signed-off-by: Tizian 「ティツィアーン」 raisondetredev@pm.me

A new bin (halkyon_curator:seed_dictionary) loads a jmdict-simplified JSON file and upserts it into the server-side dictionary_entries catalog via PostgREST, batched, on the JMdict id primary key (merge-duplicates), so re-runs are idempotent. The service-role key is read from SUPABASE_SERVICE_ROLE_KEY (never argv); --dry-run builds and samples rows without sending. Optional --jlpt-dir sets each row's level; --common-only / --limit scope the seed. DictionarySeedRow.fromWord builds a catalog row, reusing the same expression/reading/meanings extraction as the card curator — now pulled out into vocabulary_extraction.dart so the two can't drift (an 'Add to my deck' card must match the curated deck's card for the same word). search_blob holds every surface form, lowercased, for trigram lookup; English glosses are excluded by design. Adds http; 11 new tests green, existing curator suite unaffected by the extraction refactor. Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
project-halkyon/halkyon-curator!8
No description provided.