Commit graph halkyon/halkyon_learn_curator/test/fixtures/cedict-sample.txt
Author SHA1 Message Date
7b94eda592
fix(curator): HSK levels by the syllabus's tones; variants find their word
The syllabus's reading is matched with its tones first, so 和 hé gets
HSK 1 and 和 hè doesn't; only where no entry of a word reads it so (the
syllabus marks the tone changes of 一 and 不, 一下 yíxià, CC-CEDICT
doesn't) are the letters alone compared. CC-CEDICT's 2,885 entries that
are only "variant of …" (咊, an old variant of 和) are no entries of
their own any more, which repeated their word in the simplified file,
but written forms of that word, finding it in text and searches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 09:39:30 +02:00
9d21f7f20e
feat(curator): Chinese from CC-CEDICT, with the HSK levels of 2025
Chinese comes from CC-CEDICT (CC BY-SA 4.0), made for learners of
Chinese: `curate cedict` writes two dictionaries of its entries,
`cc-cedict-hans-en` (zh-Hans) and `cc-cedict-hant-en` (zh-Hant), so a
studyset finds its words as it writes them, each finding a word by
either script, by pinyin with or without tones and by its meanings.
Readings are pinyin with tone marks, a syllable to a character; measure
words are a remark beside the meanings.

The HSK levels are those of the 2025 examination syllabus, which HSK
3.0's examination follows since July 2026 (decided 2026-10-14: level
tags, credited), as hsk30 extracted them (MIT): `hsk:1` … `hsk:6`, and
`hsk:7` for 7 to 9, each given to the entry read as the syllabus reads
the word (的 de, not 的 dì; 书 shū, not the Book of Documents, Shū),
ranking the words, HSK 1 first.

`curate update --languages de,el,en,zh` takes over from `--wiktionary`:
Chinese from MDBG's file of the day as often as EDRDG's files, the rest
from Wiktionary's dump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 09:39:30 +02:00