halkyon-days/halkyon
Watch
1
0
Fork
You've already forked halkyon
 0
Code Issues Pull requests Projects Releases 37 Packages Wiki Activity Actions

feat(dictionary): Japanese cut into words and looked up on the device

Browse source 
Once the person agrees, the device holds MeCab's IPADIC and the
instance's JMdict and cuts lines as Jisho.org does, so their text never
leaves it (ADR 0012). MeCab comes through mecab_for_dart, whose build
hook compiles it; its tokens are grouped by Ve's rules, written anew in
Dart, with に kept a word of its own. Each word is found as it stands or
as a form of a word, else by its dictionary form, else as the longest
words it holds from token to token; longer words over token boundaries
are offered as before.

The curator packs Ubuntu's compiled IPADIC beside the dictionary files
(curate ipadic, and every update run). The server exports its JMdict
from its tables, so the device finds the same keys and keeps the same
versions, and serves both at /device-dictionaries/, named in
dictionary.deviceFiles('ja') with their hashes; Caddy forwards the path.

The app checks each file against its hash, unpacks IPADIC, builds a
SQLite database from the export on an isolate, and asks the instance
for newer files every 30 days, downloading only what changed. The Watch
screen offers the download once per device with its size (34 MB, some
150 MB once set up); Settings downloads and removes them. Without them,
or should cutting on the device fail, lines go to the instance as
before. MeCab's, IPADIC's and Ve's licences join the licence list.

Every build now needs a C++ compiler; mecab_for_dart holds code_assets
at 1.x.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-15 21:55:00 +02:00
commit 1d2011a676
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
1 changed files with 5487 additions and 245 deletions
Download patch file Download diff file Expand all files Collapse all files

14
deploy/Caddyfile
View file

@ -1,7 +1,7 @@
# Caddy forwards to the API server, and the assistant connector's paths (ADR 0006) and
# profile pages (ADR 0011) to the web server (8082), which serves nothing else. Insights (8081) doesn't run in
# production; if it is ever re-enabled, reach it through an SSH tunnel, never through
# this file.
# Caddy forwards to the API server, and the assistant connector's paths (ADR 0006),
# profile pages (ADR 0011) and the dictionaries devices download (ADR 0012) to the web
# server (8082), which serves nothing else. Insights (8081) doesn't run in production;
# if it is ever re-enabled, reach it through an SSH tunnel, never through this file.
#
# {$DOMAIN} is `learn.halkyon.eu` on its server, where Caddy obtains and renews a
# Let's Encrypt certificate by itself, and `http://localhost` for the local rehearsal.
@ -30,6 +30,12 @@
reverse_proxy server:8082
}
# The dictionaries devices download to look Japanese up by themselves, named by
# their hashes; already compressed.
handle /device-dictionaries/* {
reverse_proxy server:8082
}
# Serverpod's streaming methods use WebSockets; reverse_proxy upgrades them as-is.
handle {
reverse_proxy server:8080