fix(curator): HSK levels by the syllabus's tones; variants find their word

The syllabus's reading is matched with its tones first, so 和 hé gets
HSK 1 and 和 hè doesn't; only where no entry of a word reads it so (the
syllabus marks the tone changes of 一 and 不, 一下 yíxià, CC-CEDICT
doesn't) are the letters alone compared. CC-CEDICT's 2,885 entries that
are only "variant of …" (咊, an old variant of 和) are no entries of
their own any more, which repeated their word in the simplified file,
but written forms of that word, finding it in text and searches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-14 07:34:46 +02:00
commit 7b94eda592
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64

View file

@ -82,6 +82,9 @@ void main() {
'de, not dì; 书 shū, not the Book of Documents, Shū', () {
expect(entryOf('的 的 [de5]')['labels'], contains('hsk:1'));
expect(entryOf('的 的 [di4]')['labels'], isNot(contains('hsk:1')));
// Read with its tones: 说 shuō, not shuì.
expect(entryOf('說 说 [shuo1]')['labels'], contains('hsk:1'));
expect(entryOf('說 说 [shui4]')['labels'], isEmpty);
expect(entryOf('書 书 [shu1]')['labels'], contains('hsk:1'));
expect(entryOf('書 书 [Shu1]')['labels'], isEmpty);
expect(entryOf('中國 中国 [Zhong1 guo2]')['labels'], contains('hsk:1'));
@ -97,6 +100,18 @@ void main() {
);
});
test('an entry that is only a variant of another is no entry, but finds '
'that one', () {
final ids = [
for (final line in linesOf(traditional: false).skip(1)) line['id'],
];
expect(ids, isNot(contains('咊 和 [he2]')));
expect(
(entryOf('和 和 [he2]')['textKeys']! as List).cast<String>(),
contains('咊'),
);
});
test('measure words are a remark beside the meanings, pinyin marked', () {
final book = entryOf('書 书 [shu1]')['entry']! as Map;
expect(book['classifiers'], ['本 běn', '冊|册 cè', '部 bù']);