feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update them at least monthly. The instance now does it alone: a `dictionaries` service in compose, built from the curator (halkyon_learn_curator/ Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate update --every 7`, downloading EDRDG's files of the day once its files are a week old and replacing them in a volume the server mounts read-only. The curator never enters the server's image or process (decided 2026-10-09: a container of its own, weekly). The server rereads the folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed, and loads now run one at a time. redeploy.sh builds both images. An operator who'd rather supply the files removes the service and mounts deploy/dictionaries/ into the server, as compose.yaml says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
82b38fadd8
commit
25e01ad128
1 changed files with 211 additions and 32 deletions
|
|
@ -144,7 +144,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
|
|||
- **Tree, tags, modes, settings and files** (ADR 0004): a studyset's `parent` and sibling `position` form the tree, and studysets in a cycle or without their parent show at the top level (`topLevel`); its `archivedAt`, `starredAt` and `updatedAt` are columns, and every change of a studyset writes `updatedAt` with it (the library lists starred studysets first, then the most recently changed); a card's tags are `card_tag` rows, one per assignment, never updated, and a duplicate counts once; `review_log.mode` is `review`, `reverse` (its own schedule), `write` (by hand, its own schedule) or `cram` (never moves a schedule), and a schedule replays the reviews of one mode; `person_settings` is meant as one row per person, and extra rows fold into the one with the smallest id; an `asset` row stands for a file in private cloud storage, found by its hash. **Never derive a row's id from a person or anything else guessable**: ids are unique across the whole table, so someone could claim it first in their own space and break the owner's sync.
|
||||
- **Files** (ADR 0004, rule 5): `AssetEndpoint` (`requireLogin`) takes uploads into the caller's own space by SHA-256 (`startUpload` gives Serverpod's upload description, or null when the file is there already and claims it anew; `finishUpload` keeps it only if its content has that hash) and serves an asset row's file to its space's owner and members only, `NotFoundException` otherwise, also for a deleted row (the sync engine hides tombstones). Files live in the private database storage at `spaces/<space>/<sha256>`, at most 20 MB each, and within limits (`file_limits.dart`, from the environment: an anonymous person 100 MB, an identified one 500 MB or what a `file_allowance` row grants them, the instance 15 GB), checked when an upload starts (unfinished uploads count, or they could fill the disk unnoticed) and again when it finishes (only what is kept counts), refused with `FileLimitExceededException`; `deletePerson` deletes the person's files and allowance, and `FilePruningFutureCall` deletes daily those no live row refers to (or never finished), 7 days after they were last claimed. In the app, `AssetRepository` (`core/asset/`) keeps files by hash in the person's folder (`files/`, uploads waiting marked in `files/uploads/`), uploads after each sync that went through, and downloads on demand, checking the hash; every hash is checked to be one before it becomes part of a path. Content names a file by the URL `asset:<asset row id>` (`assetUrl`, `RichContent.assetIds`): images are the editor's `image` block, sounds an `audio` block (`core/editor/file_blocks.dart`), and any other URL is never fetched. `StoredFileImage` loads through the repository like Flutter's `NetworkImage`, keyed by the person's repository too, so no image crosses people.
|
||||
- **PDFs** (`feature/pdf_source/`): a `pdf_source` row per PDF in a studyset (payload: title, file name, the asset of its file, page count, reading deadline; last writer wins) with its `position` among the studyset's files in a column of its own (decided 2026-10-08; the Files tab's default order, Custom order, which Reorder changes; PDFs from before it stand at `a0`, in the order they were added) and an append-only `pdf_reading` log (`reached` with a page, `reset`): the furthest page is the furthest reached since the last reset, so devices never conflict. A PDF's file is an asset of up to 50 MB (other files 20 MB): the server takes up to 50 MB and deletes a stored file over 20 MB that doesn't start with `%PDF-`. Deleting a PDF deletes its asset row too unless another PDF has the same file (asset rows are otherwise never deleted). Cards and notes tied to a page keep it in their payload (`source`, `core/content/pdf_page_ref.dart`; card schema 4, note schema 2); a screenshot is an `image` block with `pdf` and `page` attributes.
|
||||
- **Dictionaries** (decided 2026-10-08): reference data the server holds for everyone, owned by no one, never synced (`dictionary_entry`, `dictionary_source`, `lib/src/dictionary/`). `halkyon_learn_curator` (AGPL, no Serverpod, so the server never links its code) turns each source's download into one format, a dictionary file (its README is the contract), with an adapter per source; a dictionary is one source in one meaning language (`jmdict` in `en`), of words or of characters (`kind`: JMdict, KANJIDIC2 with the kana), and the endpoints take it by those two, so a search for words never brings kanji; everything specific to a language (kana, romaji, meanings without "to") goes into each entry's `keys` and `looseKeys`, so the server's search knows no language: exact key, exact loose key, key starting with the query, loose key starting with it, each by the source's `rank` (`DictionaryEndpoint.search`, LIKE on barred keys, no extension needed). The server loads the files in its `dictionaries` folder at start, those whose SHA-256 changed, each source replaced in one transaction (`dictionary_loader.dart`). A card made from an entry copies what it needs and keeps `(source, sourceId)` as a soft reference only (`CardContent.entry`, card schema 6), so loading a newer version never touches cards. Every file carries its credits, which the app shows (About, and at the foot of the dictionary). In the app, a studyset's dictionary follows from its front language (`StudysetDictionary`, through `StudysetLanguageScope.of`): no attaching or detaching, and nothing dictionary-shaped shows without one (Add material › From the dictionary; Look up beside a vocabulary card's front). A word's example sentences (Tatoeba, through JMdict's examples edition) become its card's hints, up to three, the word in bold; Look up gives them to a card without hints of its own. `DictionaryAdder` makes the cards, each entry once per studyset, a word tagged `JLPT-NX` and with its word class by the first sense (`Noun`, `Verb`, `na-Adjective`); Look up in the composer gives a card the same tags (`DictionaryItem.tagNames`), and an entry looked up again on that card swaps the tags the one before brought (`isDictionaryTagName`). Where the instance also holds the language's characters, the dictionary screen has a tab for them (Kanji & kana: the kana, the JLPT levels and the school grades to add whole, each under its name), a character becoming a character card tagged `JLPT-NX`, `Grade N` or `Secondary school`, and `Hiragana` or `Katakana`; a character card's Look up searches the characters.
|
||||
- **Dictionaries** (decided 2026-10-08): reference data the server holds for everyone, owned by no one, never synced (`dictionary_entry`, `dictionary_source`, `lib/src/dictionary/`). `halkyon_learn_curator` (AGPL, no Serverpod, so the server never links its code) turns each source's download into one format, a dictionary file (its README is the contract), with an adapter per source; a dictionary is one source in one meaning language (`jmdict` in `en`), of words or of characters (`kind`: JMdict, KANJIDIC2 with the kana), and the endpoints take it by those two, so a search for words never brings kanji; everything specific to a language (kana, romaji, meanings without "to") goes into each entry's `keys` and `looseKeys`, so the server's search knows no language: exact key, exact loose key, key starting with the query, loose key starting with it, each by the source's `rank` (`DictionaryEndpoint.search`, LIKE on barred keys, no extension needed). The server loads the files in its `dictionaries` folder at start and daily (`DictionaryReloadFutureCall`), those whose SHA-256 changed, each source replaced in one transaction, one load at a time (`dictionary_loader.dart`). EDRDG's licence asks a server showing its dictionaries to update them at least monthly (decided 2026-10-09: instances keep current by themselves, weekly): the `dictionaries` service of `deploy/compose.yaml` runs the curator in a container of its own (`halkyon_learn_curator/Dockerfile`, user ID 10001; the AGPL tool never enters the server's image or process), `curate update --every 7` downloading EDRDG's files of the day (`JMdict_e_examp.gz`, `kanjidic2.xml.gz`; the JLPT lists are static, in the curator's `data/`), converting them and renaming each over its old file in the shared volume, which the server mounts read-only. A card made from an entry copies what it needs and keeps `(source, sourceId)` as a soft reference only (`CardContent.entry`, card schema 6), so loading a newer version never touches cards. Every file carries its credits, which the app shows (About, and at the foot of the dictionary). In the app, a studyset's dictionary follows from its front language (`StudysetDictionary`, through `StudysetLanguageScope.of`): no attaching or detaching, and nothing dictionary-shaped shows without one (Add material › From the dictionary; Look up beside a vocabulary card's front). A word's example sentences (Tatoeba, through JMdict's examples edition) become its card's hints, up to three, the word in bold; Look up gives them to a card without hints of its own. `DictionaryAdder` makes the cards, each entry once per studyset, a word tagged `JLPT-NX` and with its word class by the first sense (`Noun`, `Verb`, `na-Adjective`); Look up in the composer gives a card the same tags (`DictionaryItem.tagNames`), and an entry looked up again on that card swaps the tags the one before brought (`isDictionaryTagName`). Where the instance also holds the language's characters, the dictionary screen has a tab for them (Kanji & kana: the kana, the JLPT levels and the school grades to add whole, each under its name), a character becoming a character card tagged `JLPT-NX`, `Grade N` or `Secondary school`, and `Hiragana` or `Katakana`; a character card's Look up searches the characters.
|
||||
- **The Pomodoro timer** (`feature/pomodoro/`): the profiles follow the person (`PersonSettings.pomodoro`, settings schema 4), the running timer is the device's own (`PomodoroRun` in `<person folder>/pomodoro.json`, `PomodoroRunStore`). `PomodoroRun` is a pure state machine on the time: a phase that ends leads to the next, running if its profile starts it by itself, else paused at its full length; a timer kept while the app was closed catches up (`caughtUpTo`). `PomodoroBloc` sits above the router (`PersonApp`), and `PomodoroPresenter` hands every change to `PomodoroPlatform` as a `PomodoroNotice` in the app's language, and its buttons back. **Android:** the app's own Kotlin (`PomodoroService.kt`; MethodChannel `app.halkyon.learn/pomodoro`, EventChannel `…/buttons`): a `specialUse` foreground service with a countdown notification (chronometer), Pause/Resume, Skip, Stop, on Android 16 a Live Update (`NotificationCompat.ProgressStyle`, the cycle's phases as segments, `setRequestPromotedOngoing`); an exact alarm (`USE_EXACT_ALARM`) alerts at the phase's end with the app closed; Do Not Disturb through the interruption filter, the one before kept in shared preferences and restored. With the app swiped away its buttons open it (a service may not start an activity from a notification's button since Android 12, so the notification is posted again with activity intents), and Stop works alone, leaving a mark the app takes at its next start (`takeStoppedOutside`) to forget its timer; a profile's next phase only starts by itself while the app runs. Dart's timers sleep with the device, so `PomodoroBloc` also looks every 15 s while a phase runs, and the alarm's receiver wakes it (`phaseEnds`); a kept timer is caught up before it first shows, or its ended phase would alert again. No notification plugin: flutter_local_notifications had Live Updates only in a prerelease needing compileSdk 37 (checked 2026-10-07). **Linux:** `LinuxPomodoroPlatform`, over package:dbus: where the top bar shows icons (GNOME with the AppIndicator extension, built into Ubuntu; KDE) the timer counts down there with its buttons in the icon's menu (`StatusNotifierItem`, written in Dart and laid out as libayatana-appindicator lays itself out, within the snap's rules, `desktop-legacy` of snapd v2.77: the item at `/org/ayatana/NotificationItem/<id>`, the only path the label's signal `XAyatanaNewLabel` may come from, registered with the watcher by that path, since a snap may bind no `org.freedesktop.StatusNotifierItem-*` name; the menu at `/MenuBar`; registered again when the watcher restarts); elsewhere the timer shows in the app alone, and on GNOME the Pomodoro card suggests the extension until dismissed, asking GNOME Shell to install or switch it on (`InstallRemoteExtension`, `EnableExtension`) or opening its page (decided 2026-10-07: no notification stands in; xdg-desktop-portal's Background status is for Flatpaks only). An alert with sound comes through `org.freedesktop.Notifications` when a phase ends while the app runs, and GNOME's `show-banners` goes off while focusing, through `gsettings`; the value before and the dismissed suggestion are kept in `<app support>/`. Calls to the bus go one at a time (`SerialQueue`).
|
||||
- **Citations** (`core/citation/`): a `Source` is plain text parts as the person typed them (authors as full names, one per line; year; title; `container`, what it is a part of; publisher; ISBN; link; `raw`, a citation a file brought, shown while nothing else is cited), and `formatCitation` cites it in the device's `CitationStyle` (APA 7, MLA 9, Chicago 18 notes and bibliography, decided 2026-10-07): it splits and abbreviates the names itself (inverted names, particles, suffixes; names written with `&` or "et al." stay as written), sets a work of its own in italics and a part of something apart from the whole by `container`, and writes a missing year as "n.d." in APA and Chicago. A step's sources (step schema 2) and a studyset's bibliography (`bibliography`, studyset schema 4, last writer wins like the rest of its payload) are lists of them, shown and changed through `SourceList` and `showSourcesSheet`, in reference-list order (alphabetical, whatever order they were added in); the path import reads them too, text as `raw` and the old format's `citation` as `container`.
|
||||
- **What a card carries beside its type's fields** (card schema 5): up to six `hints` and an `explanation` (rich text, any type), a `report` that something on it is wrong (category, note, when; studied as before, its badge in the list), and a word's `senses` (glosses, label, JMdict's grammar codes, whose English short forms and meanings are data in `feature/card/model/jmdict_tags.dart`; with senses a word's meanings are their glosses, and `uk` on its first sense shows its reading from the start). Each merges as a field of its own. Deleting a card goes through `CardDeletions` (a `DelayedDeletions`, as notes and studysets), so the list and the session offer Undo. A study session's Undo takes back its steps last first (ratings, burials and deletions alike, across its rounds), so it holds the cards it deletes (`hold`) until it ends, when they go after the usual time to undo; a card deleted in a session that never ended (the app killed) stays. Vocabulary can be quizzed as multiple choice (`quizVocabulary`, `quizAspect` in the study options, settings schema 5; `vocabularyQuiz` draws the wrong answers from the other words studied). **A character card** (`CardType.character`, card schema 7, decided 2026-10-09: one type for kanji and kana) holds the character on its front, its `meanings` (a kana's romaji is its meaning, so kana are studied and written as kanji are) and a kanji's `onReadings` and `kunReadings` as dictionaries write them (katakana; hiragana with the okurigana after a dot); its JLPT level and school grade are tags, its strokes the app's own. **How characters are written** (`feature/kanji/`, decided 2026-10-09: bundled, so writing works offline) is KanjiVG's, in `assets/kanjivg/` (CC BY-SA 3.0, its notice in the licence list): the curator's `curate kanjivg` writes one gzipped JSON file per 256 code points (2.7 MB in all), which `StrokeRepository` (app-wide in `Dependencies`, as the maps) reads when a character of it is first asked for; the strokes are SVG path data in a 109 × 109 square, read with `path_parsing` (pure Dart, so `CharacterStrokes` flattens them into points without `dart:ui`). `StrokeOrderView` traces them in order; `showCharacterSheet` shows a character with its stroke order, meanings, readings and components, from a studied character's answer and from the dictionary's characters. **Writing by hand** (decided 2026-10-09, after Kanji Dojo's practice): the study option Write by hand (`StudyOptions.writeByHand`, settings schema 7) has character cards and words written in the session, mode `write` on a schedule of its own (words so written are neither quizzed nor typed). `WritingCanvas` takes raw pointers and claims every touch on it at once (an `EagerGestureRecognizer`), so a stroke never scrolls or swipes to rate; each stroke is checked as it ends (`checkStroke`, `kanji/model/stroke_check.dart`: both resampled to 22 points, shape compared once centred, position, length and direction, against what a loose hand strays by; a stroke closer to one still to come is out of order), a right one drawn as KanjiVG draws it, and the hand's offset from the middle forgiven in the strokes after. Mistakes are strokes that didn't count and hints asked for; two misses in a row show the next stroke faintly. `WritingBody` writes a word character by character, filling in those without a stroke order, and once written (or given up on, Show it) outlines the rating the mistakes suggest (`suggestedRating`: none Good, one or two OK, more Bad; given up, Bad). `stroke_check_test.dart` holds the checker to real strokes, moved, wavering, larger and smaller, reversed and out of order: change its weights only with that test passing.
|
||||
|
|
|
|||
Loading…
Reference in a new issue