halkyon/apps/learn/halkyon_learn_curator/data

Data the curator brings along

What the dictionaries draw on besides their sources, and never changes, so curate update needs only EDRDG's files of the day, MDBG's CC-CEDICT and Wiktionary's dump.

jlpt/n1.csv … n5.csv are the JLPT vocabulary lists of yomitan-jlpt-vocab (original_data, as of 2026-10-09), CC BY-SA 4.0 (jlpt/LICENSE.txt): Jonathan Waller's lists (tanos.co.uk, CC BY) with each word's JMdict entry and its most common spelling, by stephenmk.

kanji-jlpt.json maps each kanji on Jonathan Waller's JLPT kanji lists (tanos.co.uk, CC BY) to its level, 5 for N5 to 1 for N1: the jlpt_new of kanji-data's kanji.json (MIT, as of 2026-10-09), nothing else of it.

The lists are over ten years old and won't change; replace them only for a list of the same shape.

wordfreq/small_<language>.msgpack.gz are wordfreq's lists of how often words are used (wordfreq v3.0, as of 2021-03-29, the last change to its data), unchanged: CC BY-SA 4.0, by Robyn Speer, drawn from SUBTLEX (Marc Brysbaert et al., whose authors wordfreq's licence asks to be credited, and who made SUBTLEX freely available), OpenSubtitles, Wikipedia, Google Books Ngrams and others, as wordfreq's README lists them. The Wiktionary dictionaries take their ranks and their common label from them and credit both. wordfreq's data won't change: it was sunset in 2024.

hsk/syllabus2025_words.tsv lists the 10,896 words of the HSK examination syllabus published in November 2025 by the Center for Language Education and Cooperation (中外语言交流合作中心), the one HSK 3.0's examination follows since July 2026, each with its level (1 to 6, 7 for 7 to 9), the level of a homograph graded twice, its pinyin and its part of speech: as extracted from the syllabus's PDF by Alvaro Serrano's hsk30 (corpus/syllabus2025/, as of 2026-09-06), MIT (hsk/LICENSE.txt), unchanged. Only each word's level reaches the dictionaries (hsk:N), with the pinyin to tell CC-CEDICT's readings of one word apart; the curator credits both. The Ministry claims copyright on the standard's documents; a word's level is a fact about it, which the dictionaries keep with attribution (decided 2026-10-14). hsk/syllabus2025_chars.tsv lists the syllabus's 3,088 characters to recognise (认读字) with their levels (246, 125, 284, 441, 431 and 413 for levels 1 to 6, 1,148 for 7 to 9), from the same repository and commit (corpus/syllabus2025/), MIT, unchanged; they label the Chinese characters (hsk:N) as the words' label the words.