feat(dictionary): Japanese cut into words and looked up on the device

Once the person agrees, the device holds MeCab's IPADIC and the
instance's JMdict and cuts lines as Jisho.org does, so their text never
leaves it (ADR 0012). MeCab comes through mecab_for_dart, whose build
hook compiles it; its tokens are grouped by Ve's rules, written anew in
Dart, with に kept a word of its own. Each word is found as it stands or
as a form of a word, else by its dictionary form, else as the longest
words it holds from token to token; longer words over token boundaries
are offered as before.

The curator packs Ubuntu's compiled IPADIC beside the dictionary files
(curate ipadic, and every update run). The server exports its JMdict
from its tables, so the device finds the same keys and keeps the same
versions, and serves both at /device-dictionaries/, named in
dictionary.deviceFiles('ja') with their hashes; Caddy forwards the path.

The app checks each file against its hash, unpacks IPADIC, builds a
SQLite database from the export on an isolate, and asks the instance
for newer files every 30 days, downloading only what changed. The Watch
screen offers the download once per device with its size (34 MB, some
150 MB once set up); Settings downloads and removes them. Without them,
or should cutting on the device fail, lines go to the instance as
before. MeCab's, IPADIC's and Ve's licences join the licence list.

Every build now needs a C++ compiler; mecab_for_dart holds code_assets
at 1.x.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-15 21:55:00 +02:00
commit 1d2011a676
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64

View file

@ -11112,9 +11112,93 @@ abstract class AppLocalizations {
/// No description provided for @jellyfinPrivacy.
///
/// In en, this message translates to:
/// **'Your server\'s address and your sign-in stay on this device, and what you tie to a studyset follows you as ids, never titles. To find the words of a line, the app asks the instance about its text, which it answers and doesn\'t keep. Words you add are tagged with where you met them, kept with your flashcards like any tag.'**
/// **'Your server\'s address and your sign-in stay on this device, and what you tie to a studyset follows you as ids, never titles. To find the words of a line, the app asks the instance about its text, which it answers and doesn\'t keep, unless Japanese\'s dictionaries are on this device (Settings): then it looks them up here. Words you add are tagged with where you met them, kept with your flashcards like any tag.'**
String get jellyfinPrivacy;
/// No description provided for @deviceDictionaryOfferTitle.
///
/// In en, this message translates to:
/// **'Look words up on this device?'**
String get deviceDictionaryOfferTitle;
/// No description provided for @deviceDictionaryOfferBody.
///
/// In en, this message translates to:
/// **'Download Japanese\'s dictionaries once ({size}, more once set up) to cut lines into words here, as Jisho.org does. Their text then never leaves this device.'**
String deviceDictionaryOfferBody(String size);
/// No description provided for @deviceDictionaryOfferDownload.
///
/// In en, this message translates to:
/// **'Download'**
String get deviceDictionaryOfferDownload;
/// No description provided for @deviceDictionaryOfferLater.
///
/// In en, this message translates to:
/// **'Not now'**
String get deviceDictionaryOfferLater;
/// No description provided for @deviceDictionaryDownloading.
///
/// In en, this message translates to:
/// **'Downloading: {received} of {total}'**
String deviceDictionaryDownloading(String received, String total);
/// No description provided for @deviceDictionaryPreparing.
///
/// In en, this message translates to:
/// **'Setting them up…'**
String get deviceDictionaryPreparing;
/// No description provided for @deviceDictionaryReady.
///
/// In en, this message translates to:
/// **'Words are now looked up on this device.'**
String get deviceDictionaryReady;
/// No description provided for @deviceDictionaryFailed.
///
/// In en, this message translates to:
/// **'Couldn\'t download the dictionaries. Try again in Settings.'**
String get deviceDictionaryFailed;
/// No description provided for @settingsDeviceDictionaryTitle.
///
/// In en, this message translates to:
/// **'Japanese on this device'**
String get settingsDeviceDictionaryTitle;
/// No description provided for @settingsDeviceDictionaryAbout.
///
/// In en, this message translates to:
/// **'With Japanese\'s dictionaries on this device, the lines of videos and songs are cut into words here, as Jisho.org does, and their text never leaves it. Without them, the app asks the instance, which answers and keeps nothing.'**
String get settingsDeviceDictionaryAbout;
/// No description provided for @settingsDeviceDictionaryHeld.
///
/// In en, this message translates to:
/// **'On this device: {size}'**
String settingsDeviceDictionaryHeld(String size);
/// No description provided for @settingsDeviceDictionaryDownload.
///
/// In en, this message translates to:
/// **'Download ({size})'**
String settingsDeviceDictionaryDownload(String size);
/// No description provided for @settingsDeviceDictionaryRemove.
///
/// In en, this message translates to:
/// **'Remove'**
String get settingsDeviceDictionaryRemove;
/// No description provided for @settingsDeviceDictionaryFailed.
///
/// In en, this message translates to:
/// **'The download failed.'**
String get settingsDeviceDictionaryFailed;
/// No description provided for @jellyfinSignedInAs.
///
/// In en, this message translates to: