docs: ADR 0009, feature parity for the other languages
What applies of Japanese's features to Chinese, Korean, Greek, German and English, with the developer's choices and each decision's reason: Unihan's characters, KRDict's words and levels, Make Me a Hanzi's and the curator's own strokes, typed pinyin, radicals and readings by language; what doesn't apply or has no open data. AGENTS.md, the server's README and the instance's example environment follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
e7f3abdb8d
commit
6541af937a
1 changed files with 127 additions and 9 deletions
114
docs/adr/0009-language-parity.md
Normal file
114
docs/adr/0009-language-parity.md
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
# ADR 0009: Feature parity for Chinese, Korean, Greek, German and English
|
||||
|
||||
- **Status:** accepted
|
||||
- **Date:** 2026-10-14
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0008 gave five more languages dictionaries of words and cut their subtitles into
|
||||
words. Japanese had more: a dictionary of its characters with their levels and school
|
||||
grades, a tab of the 214 radicals, the kana to add whole, how each character is written
|
||||
(KanjiVG) and writing it by hand, typed readings, furigana, and the levels of its words
|
||||
to study and filter by. The developer asked for the same "where applicable".
|
||||
|
||||
What applies differs by script. Chinese and Korean write Han characters (Korean as
|
||||
Hanja, beside its own alphabet); their learners learn characters as Japanese learners
|
||||
learn kanji. Hangul and the Greek alphabet are scripts to learn as kana are, but their
|
||||
words are written as they sound. German and English need nothing for their script.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Characters from Unihan** (the Unicode Consortium's database, Unicode License v3,
|
||||
permissive; 8 MB, a new version yearly): a characters' dictionary per script as the
|
||||
words have one. Simplified Chinese: the 8,105 General Standard Chinese Characters
|
||||
(2013, `kTGH`). Traditional Chinese: each of them as Taiwan and Hong Kong write it,
|
||||
but only where it is a form of its own or a word of CC-CEDICT writes it so, as Unihan
|
||||
calls 这 its own traditional variant though no traditional word is written with it;
|
||||
with the characters Hong Kong's primary schools teach. Readings in pinyin (the usual
|
||||
one first: the mainland's in simplified, Taiwan's in traditional characters), the
|
||||
Kangxi radical, strokes. A character ranks by the most used word of CC-CEDICT written
|
||||
with it in its script (wordfreq), so 後 ranks as the 后 of 以后 and 后 (queen) by 皇后;
|
||||
a simplified character standing alone for several traditional ones gives its place
|
||||
to the one the longer words use most.
|
||||
2. **HSK 2025's character levels** label Chinese characters (`hsk:1` … `hsk:7`), from
|
||||
the same extraction of the syllabus as the words' (hsk30, MIT; ADR 0008), and the
|
||||
3,500 standard characters in common use `common`. A traditional character takes its
|
||||
simplified one's level only as its usual form (後 for 后, not 后 itself).
|
||||
3. **Korean's words from KRDict** (the developer's choice), the National Institute of
|
||||
Korean Language's Basic Korean Dictionary for learners (CC BY-SA 2.0 KR, which
|
||||
allows later versions; its JSON export downloads without an account, some 84 MB a
|
||||
month), replacing Wiktionary for Korean's words: its learners' levels (초급, 중급, 고급)
|
||||
label and rank the words, as JLPT and HSK do, with each word's pronunciation, Hanja
|
||||
and English senses. Its examples have no translations, so Tatoeba's stay the hints.
|
||||
KRDict gives few forms of a word (5 of 가다); Wiktionary's tables give 99, so the
|
||||
Wiktionary update keeps every Korean word's forms beside the files
|
||||
(`.wiktionary-ko.json.gz`), and KRDict's file takes them as text keys. An idiom
|
||||
carries the id of the word it is filed under; its own written form is added to it.
|
||||
4. **A Korean card's reading is its pronunciation where it differs from the
|
||||
spelling** (the developer's choice: 같이 [가치]), shown in brackets as dictionaries
|
||||
print it, shown and quizzed as a reading but not typed (it is Hangul already).
|
||||
5. **Hanja from Unihan** with the 1,800 taught in Korea's schools labelled
|
||||
(`kKoreanEducationHanja`), their sounds (음, the one taught first) and their native
|
||||
meanings (훈) from Wiktionary's Korean entries of Han characters (물 수 for 水), as
|
||||
Korean dictionaries name a Hanja: meaning, then sound. The characters' tab holds the
|
||||
40 Hangul letters as KANJIDIC2's holds the kana (the developer's choice), each with
|
||||
its sound and name.
|
||||
6. **The Greek alphabet** (the developer's choice) is a characters' dictionary of its
|
||||
own, written by the curator (CC0): each letter with its capital, transliteration
|
||||
(ELOT 743), names and sound, to add whole. No stroke order: no standard to check
|
||||
handwriting against, and learners write Latin-like letters already.
|
||||
7. **How Chinese characters are written comes from Make Me a Hanzi** (taken as the
|
||||
developer's yes to parity): 9,574 characters of both scripts in the mainland's order,
|
||||
its strokes under the Arphic Public License (APL), its decompositions under the
|
||||
LGPL. The APL is incompatible with the GPL for code, not for data beside it (FSF:
|
||||
"mere aggregation"), so the strokes are a data file of the app's own, never in
|
||||
cards: a card names its character, and the app draws it. Each stroke is the line
|
||||
Make Me a Hanzi draws through its outline's middle (its median), in KanjiVG's 109
|
||||
square, so the drawing and the stroke checker stay as they are; each file says how
|
||||
and when it was changed, as the APL asks. 2.7 MB beside KanjiVG's 2.8 MB.
|
||||
8. **The Hangul letters' strokes are drawn by the curator** (CC0, the developer's
|
||||
choice): no openly licensed data draws them with checkable strokes. A Korean word is
|
||||
written letter by letter (닭: ㄷ ㅏ ㄺ), each full size: the letters' order of strokes
|
||||
is what is learnt; where each sits in its syllable's block is not checked.
|
||||
9. **Stroke sets by language**: Japanese KanjiVG; Chinese Make Me a Hanzi, then
|
||||
KanjiVG for what it lacks; Korean its letters, then KanjiVG, then Make Me a Hanzi
|
||||
(Korea writes Hanja in the traditional order, which KanjiVG's Japanese order follows
|
||||
more often than the mainland's). Writing by hand covers cards in a script whose
|
||||
strokes are drawn (Han characters, kana, Hangul) and no others: a German or Greek
|
||||
word is studied as typed.
|
||||
10. **Pinyin is typed** as kana are: a Chinese word's reading, tone numbers turning into
|
||||
marks as typed (ni3 → nǐ, v as ü), checked with its tones.
|
||||
11. **Radicals** for Chinese and Korean in their own forms: simplified Chinese's 讠 钅 门
|
||||
for 言 金 門, traditional Chinese and Korean without Japanese's own (竜, 歯).
|
||||
12. **What is shown where follows the language**: the characters' tab is Hanzi, Hangul &
|
||||
Hanja or Alphabet; readings are On and Kun, Pinyin, or Eum and Hun; level filters
|
||||
name their list (no KRDict level, no HSK level). A studyset's title is drawn in its
|
||||
language's shapes outside it too (the library's and Explore's cards), as Han
|
||||
characters differ between the fonts (骨, 直).
|
||||
13. **The card browser searches and sorts as typed on any keyboard**: without accents,
|
||||
tone marks or umlauts (strasse finds Straße, xue xi finds xué xí).
|
||||
14. **A German verb is found with its particle**: where a clause ends in a separable
|
||||
particle, each word before it is also looked up with the particle joined on, as a
|
||||
subordinate clause writes the verb (rufe … an: anrufe, which Wiktionary lists among
|
||||
anrufen's forms), and offered as the longer words running over a Japanese word are.
|
||||
15. **Files of languages no longer kept go**: the curator removes its files for
|
||||
languages no longer named or sources replaced (Korean's words from Wiktionary), and
|
||||
the server unloads every dictionary no file holds, once every file could be read.
|
||||
|
||||
## Consequences
|
||||
|
||||
- An instance downloads Unihan (8 MB) and CC-CEDICT again weekly for the characters, and
|
||||
KRDict monthly (84 MB, a gigabyte of JSON read an entry at a time: 25 s, under 600 MB,
|
||||
within the dictionaries container's 1 GB). Measured on the real data: 8,105
|
||||
simplified and 8,376 traditional characters, 7,799 Hanja and 40 letters, 24 Greek
|
||||
letters, 54,870 Korean words (2,545 초급, 9,067 중급, 37,023 고급).
|
||||
- The app grows by 2.8 MB of stroke data; its licence list shows the APL and the LGPL
|
||||
beside KanjiVG's notice.
|
||||
- Not applicable, or not open: levels for German, Greek and English (the Goethe-Institut's
|
||||
and the English Vocabulary Profile's lists are copyrighted, CEFR word lists such as
|
||||
CEFRLex non-commercial), TOPIK lists (unclear licences; KRDict's levels stand in),
|
||||
subnational maps (a geography feature, not a language one), Hanja example words for
|
||||
Korean words (cards keep no Hanja), Anki's pinyin filters, and checking where a letter
|
||||
sits within a Hangul syllable.
|
||||
- Traditional characters are drawn in the mainland's stroke order where Make Me a Hanzi
|
||||
draws them; animCJK has Taiwan's order for only 1,425 characters so far.
|
||||
Loading…
Reference in a new issue