For parity with Japanese's KANJIDIC2 and kana:
- Unihan (Unicode License v3) makes characters' dictionaries: the 8,105
standard simplified characters (unihan-hans) and the traditional ones in
use (unihan-hant), told apart and ranked by CC-CEDICT's words (Unihan
calls 这 its own traditional variant; 後 ranks as the 后 of 以后), with
pinyin, meanings, strokes, Kangxi radical, HSK 2025 character levels,
the 3,500 common ones and Hong Kong's grades; and Korean Hanja
(unihan-ko) with their sounds, the 1,800 of education, and their native
readings from Wiktionary (물 수), with the 40 Hangul letters as kana are.
- The Greek alphabet, the curator's own (CC0): letters with names,
transliteration and sound.
- KRDict, the Basic Korean Dictionary for learners (CC BY-SA 2.0 KR,
downloaded without an account), replaces Wiktionary for Korean's words:
learners' levels, pronunciation where it differs from the spelling
(같이 [가치]), Hanja, English senses; Wiktionary's update keeps the
words' forms (가다's 99) and the Hanja's readings beside it for both.
- The update converts KRDict when its page offers another export and the
characters weekly, and removes the files of languages no longer kept
or of sources replaced.
Measured on the real data: Unihan 3 s; KRDict 54,870 entries in 24 s
and under 600 MB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
Wiktionary tags a contraction as an abbreviation and an alternative of
its first word (im: in + dem), so German's im, zum and ins, which every
subtitle has, were neither entries nor found as forms. A contraction is
now an entry of its own, as text writes it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
Korean comes from the same dump as German and Greek, with English
meanings (decided 2026-10-14: Korean last, from Wiktionary for now): a
verb's conjugations are among its forms, so text finds them; a word's
Hanja and romanisation find it in a search and show beside it. The
counter words Wiktionary's tables name and its notes on what it couldn't
read are no forms of a word.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
The languages besides Japanese come from the English Wiktionary as
Wiktextract reads it (decided 2026-10-14: German, Greek and English
first, then Chinese, then Korean): kaikki.org's raw dump, every language
in one file, as its per-language files are being withdrawn.
`curate wiktionary --languages de,el,en` reads it once, streamed, and
writes German and Greek with Wiktionary's English meanings and English
with what its translation tables give in German, the app's other
language (`wiktionary-de-en`, `-el-en`, `-en-de`; sources
`wiktionary-<language>`, as the server holds one source per meaning
language). Inflected forms, and Wiktionary's entries for forms alone
("Häusern: dative plural of Haus"), find their word instead of being
entries; names and affixes are left out, as are articles that tables
give cells of their own, abbreviations and rare spellings, which would
match common words. Entries keep their part of speech, genders, IPA,
principal forms and examples keyed by language as JMdict's.
wordfreq's small lists (CC BY-SA 4.0, crediting SUBTLEX) in data/ rank
the words by how often they are used and label the 15,000 most used
`common`. Typing without accents or umlauts finds a word (`strasse`,
`haeuser`, `spiti`).
The format gains `textKeys`: what finds an entry in running text, its
own forms, never its meanings, which would match words of the meaning
language in a subtitle. `curate update --wiktionary de,el,en` keeps the
languages current: it asks kaikki.org when the dump last changed and
converts it again only when it is newer, or the languages or the
conversion changed. The whole dump converts in about ten minutes in
under 300 MB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
