| Filename | Latest commit message | Latest commit date |
|---|---|---|
Tizian 「ティツィアーン」
8e3a4e56dc
A search matched LIKE '%|eat%' over every entry's barred keys, which no index can answer, so each one read the whole dictionary: 43-57 ms in JMdict's 218,863 entries. dictionary_search_key holds a row per key and loose key with its first one, two and three characters in columns of their own and one index on them, so the keys a query starts are an equality the index answers whatever the database's collation (Serverpod's models can't declare COLLATE "C" or text_pattern_ops). A search now takes 5-26 ms, at 1.4 million rows and 190 MB for JMdict, and its load takes 72 s instead of 33 s. The loader fills the table beside the entries and loads a dictionary again when it has none. Until then, in the minutes after the deploying server starts, the search reads the barred keys as before; those columns stay until a later release, since the server before a deploy's switch still searches them. The migration only creates the table and its index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me> |
||
| .. | ||
| bin | ||
| config | ||
| data | ||
| dictionaries | ||
| lib | ||
| migrations | ||
| test | ||
| .gitignore | ||
| analysis_options.yaml | ||
| CHANGELOG.md | ||
| dart_test.yaml | ||
| Dockerfile | ||
| LICENSE | ||
| pubspec.yaml | ||
| README.md | ||
halkyon_learn_server
This is the starting point for your Serverpod server.
Start your server by running:
cd halkyon_learn
serverpod start
When you are finished, you can shut down the running server with Q.
Dictionaries
The dictionaries people look words up in (JMdict, …) are reference data the
server holds for everyone. They come as dictionary files, which
halkyon_learn_curator makes from the sources' downloads (its README says how).
At every start, and every hour at seven past, the server loads the files in
its dictionaries folder that changed since they were last loaded (by their
SHA-256): in the background, a source at a time, and all at once, so searches
meanwhile still find the old entries. A file it can't read is logged and left
out. A dictionary whose file is gone is unloaded, once every file in the folder
could be read (the curator removes the files of languages an instance no longer
keeps). DICTIONARIES_DIR names another folder.
EDRDG's licence asks a server showing its dictionaries to update them at least once a month:
- An instance keeps them current by itself: the
dictionariesservice indeploy/compose.yaml(the curator, in a container of its own) downloads EDRDG's files of the day once its files are a week old, converts them, and replaces them in a volume the server reads (curate update, the curator's README); the server takes them up within the hour. To supply files yourself instead, remove the service and mountdeploy/dictionaries/into the server (compose.yamlsays how), and update them monthly. - Development:
dart run halkyon_learn_curator:curate update --into ../halkyon_learn_server/dictionariesin the curator's folder (about a minute); the server takes them up within the hour.
JMdict in English, with the JLPT levels and Tatoeba's example sentences, is
about 20 MB gzipped and takes the server half a minute to load (219,000
entries). KANJIDIC2 in English, with the JLPT levels of the kanji and the kana,
is a dictionary of characters beside it (kanjidic-en.jsonl.gz, 0.7 MB,
12,800 entries): the app's Kanji & kana tab. Their licences (CC BY-SA 4.0;
CC BY 2.0 FR for the sentences; CC BY for the JLPT lists) ask for attribution:
the app lists the credits each file carries.
Licence
Copyright (C) 2026 The Halkyon Learn contributors
Licensed under the Server Side Public License, version 1 (SSPL-1.0); the full text
is in LICENSE.