halkyon/halkyon_learn_curator/test/kana_test.dart
Tizian 「ティツィアーン」 50ca46f6b1
fix: dictionary headwords, search order and a loader that can't crash
The curator:
- A word JMdict says is usually written in kana gets a kana headword,
  even where the JLPT lists give kanji (その, not 其の), and a rare or
  search-only spelling is never the headword while a usual reading
  exists (そして, not 而して); 9,611 headwords change, 96 of them N5.
- Keys hold full-width letters as typed (Tシャツ as tシャツ), デュ/テュ/
  フュ/イェ romanised, long a, i and e typed once as loose keys (kado for
  カード), and a blank JLPT line is skipped.

The server:
- A search is one query ordered by tier, then rank: a loose or prefix
  match no longer pushes an exact one out of the list (75-150 ms over
  all of JMdict, before 150-450).
- Queries find long vowels with macrons spelt おお or ー and typed once
  (ōkii, kōhī), ん typed n' or nn, and full-width letters.
- Pages of a label are ordered by rank and id, so entries that share a
  rank are neither repeated nor skipped.
- The loader never throws (an unreadable folder ended the server from
  the background), refuses two files of one source rather than loading
  both at every start, reads no rows back when replacing a source, and
  takes its time from clock.
- Dictionary files stay out of the Docker build context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 18:09:28 +02:00

37 lines
1.4 KiB
Dart
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import 'package:halkyon_learn_curator/src/japanese/kana.dart';
import 'package:test/test.dart';
void main() {
test('katakana becomes hiragana; the rest stays', () {
expect(toHiragana('テレビ'), 'てれび');
expect(toHiragana('カード'), 'かーど');
expect(toHiragana('食べる'), '食べる');
});
test('kana become Hepburn romaji, as people type them', () {
expect(toRomaji('たべる'), 'taberu');
expect(toRomaji('とうきょう'), 'toukyou');
expect(toRomaji('しゃしん'), 'shashin');
expect(toRomaji('きって'), 'kitte');
expect(toRomaji('まっちゃ'), 'matcha');
expect(toRomaji('カード'), 'kaado');
expect(toRomaji('パーティー'), 'paatii');
expect(toRomaji('ふぁいる'), 'fairu');
expect(toRomaji('ぢ'), 'ji');
expect(toRomaji('デュエット'), 'dyuetto');
expect(toRomaji('フュージョン'), 'fyuujon');
});
test('long vowels written once, where that differs', () {
expect(withShortVowels('toukyou'), 'tokyo');
expect(withShortVowels('kaado'), 'kado');
expect(withShortVowels('oniisan'), 'onisan');
expect(withShortVowels('taberu'), isNull);
});
test('full-width letters and digits become the ones people type', () {
expect(halfWidth('Tシャツ'), 'Tシャツ');
expect(halfWidth('3D CD'), '3D CD');
expect(halfWidth('東京'), '東京');
});
}