Document files and the Anki import
All checks were successful
CI / check (push) Successful in 6m30s
CI / deploy (push) Successful in 1m3s

AGENTS.md: the file endpoints and the app's files, asset URLs in content,
imported cards, platform plugins behind interfaces, server tests on the sync
engine's database, and GStreamer for Linux builds. ADR 0004: content names
files as asset:<id> URLs, and how stored files are deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-02 04:52:02 +02:00
commit 7b933b2951
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
2 changed files with 17 additions and 4 deletions

View file

@ -82,6 +82,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **One graph per context.** The dependency graph belongs to one instance and one signed-in person; when either changes, a `KeyedSubtree` disposes the whole subtree and rebuilds it. Why: nothing from one person's session (blocs, streams, caches) can leak into another's.
- **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`.
- **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again.
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), playing sounds (`SoundPlayer`, `audioplayers`), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one).
- **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`.
- **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point.
- **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`.
@ -107,6 +108,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **Text inside the editor's views** (`RichContentView`, `RichContentEditor`) is rich text: find it with `findRichText: true`.
- **Layout-sensitive widget tests** set `tester.view.physicalSize` explicitly, so a responsive branch isn't missed for the wrong reason.
- **A menu item acts in the frame after its menu closes** (`MenuAnchor`): pump twice after tapping one before expecting what it does.
- **Server tests of what writes synced rows** run on the sync engine's database, as the server does (the generated test tools build the pod without it): pass the interceptor to `withServerpod`, call `initializeOfflineSync` in `setUp`, and write as a person through `transactionForUser`. Its writes take transactions of their own, so these tests turn rollback off and delete what they made (`test/integration/asset/asset_test.dart`).
- **Sync tests run against a running server** (ADR 0001, rule 9: the client's and the server's sync engines can't share one process): `halkyon_learn_flutter/test/sync/`, tagged `requires-server`, skipped by `tool/check.sh`; run them with `flutter test -t requires-server` while `serverpod start` runs. Every device is a client with in-memory session storage and its own local database, with the app's real gateway and repository on top; tests sync explicitly. What only the operator could see (a blanked payload, a row that must not exist) is checked with SQL against the development database, through the embedded PostgreSQL's Unix socket where there is one. An honest device only syncs the spaces the server announces, so a forged write never leaves it; the server's own check against modified clients (`_assertCanMergeInboundSpace` in the sync engine) has no test yet. They live in the app package because the server's Docker build resolves the server's dependencies without the client package. They must not initialize a Flutter test binding (no `testWidgets`), which would replace real HTTP with a mock.
- **Widget tests of screens start the whole app**: `pumpApp(tester, testDependencies(...), location: …)` (`test/support/pump_app.dart`) pumps `SessionGate` with the real `PersonApp`, opened at the screen's path, and settles once `PersonScope` has opened the person's data. From there a test moves as a person would: `openTab` (the tab's label alone may also be a screen's title), taps, `pageBack`. Data goes into the fake gateways before pumping; they work in memory (`FakeStudysetGateway`, with `failReads` for the error path), and a repository of the test's own on the same fake gateway writes what the app then reads. `pumpAndSettle` never ends while a spinner shows (`AppLoadingIndicator` animates without end): pump frames instead where one is expected. A widget pumped on its own that reads `AppLocalizations` needs `AppLocalizations.localizationsDelegates` in its `MaterialApp`.
@ -133,6 +135,8 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **Reviews are an append-only log** (`ReviewLog`: card, rating, `reviewedAt` in UTC, `durationMs`). A card's schedule is never stored: `scheduleOf` (`feature/study/model/card_schedule.dart`) replays its reviews in time order with FSRS-6 (`fsrs`, pinned exactly), and only the final due date is fuzzed, from the card's id and review count, so every device computes the same. Studying a studyset covers everything inside it but archived studysets (`studiedIn`), and the study queue (`study_queue.dart`) takes each studyset's cards as a group: due cards of all of them earliest first, then each studyset's new cards in tree order, 20 per studyset and local day. The library's and the topics' counts and badges add up the same way. Deleting a card deletes its reviews.
- **Order is a fractional index** (`fractional_indexing_dart`, rocicorp's algorithm) in its own column, last writer wins. Two devices appending at once can pick the same key, so ties sort by id.
- **Tree, tags, modes, settings and files** (ADR 0004): a studyset's `parent` and sibling `position` form the tree, and studysets in a cycle or without their parent show at the top level (`topLevel`); its `archivedAt`, `starredAt` and `updatedAt` are columns, and every change of a studyset writes `updatedAt` with it (the library lists starred studysets first, then the most recently changed); a card's tags are `card_tag` rows, one per assignment, never updated, and a duplicate counts once; `review_log.mode` is `review`, `reverse` (its own schedule) or `cram` (never moves a schedule), and a schedule replays the reviews of one mode; `person_settings` is meant as one row per person, and extra rows fold into the one with the smallest id; an `asset` row stands for a file in private cloud storage, found by its hash. **Never derive a row's id from a person or anything else guessable**: ids are unique across the whole table, so someone could claim it first in their own space and break the owner's sync.
- **Files** (ADR 0004, rule 5): `AssetEndpoint` (`requireLogin`) takes uploads into the caller's own space by SHA-256 (`startUpload` gives Serverpod's upload description, or null when the file is there already and claims it anew; `finishUpload` keeps it only if its content has that hash) and serves an asset row's file to its space's owner and members only, `NotFoundException` otherwise, also for a deleted row (the sync engine hides tombstones). Files live in the private database storage at `spaces/<space>/<sha256>`, at most 20 MB; `deletePerson` deletes the person's, and `FilePruningFutureCall` deletes daily those no live row refers to (or never finished), 7 days after they were last claimed. In the app, `AssetRepository` (`core/asset/`) keeps files by hash in the person's folder (`files/`, uploads waiting marked in `files/uploads/`), uploads after each sync that went through, and downloads on demand, checking the hash; every hash is checked to be one before it becomes part of a path. Content names a file by the URL `asset:<asset row id>` (`assetUrl`, `RichContent.assetIds`): images are the editor's `image` block, sounds an `audio` block (`core/editor/file_blocks.dart`), and any other URL is never fetched. `StoredFileImage` loads through the repository like Flutter's `NetworkImage`, keyed by the person's repository too, so no image crosses people.
- **Imported cards** say where they came from in their first revision's payload (`origin`, card schema 3, e.g. `anki`); the dashboard doesn't count them as written. The Anki import (`feature/anki/`) reads `.apkg`/`.colpkg` packages of every format on an isolate of its own (`AnkiPackageReader`), zstd through our fork of `pure_zstd` (Anki's frames don't record their size), and its fixtures come from Anki itself (`tool/anki/make_fixtures.py`).
- **Every client migration keeps a device's rows** (`test/core/sync/device_migration_test.dart` replays each on a database with a row in every synced table of ours; SQLite rebuilds tables to add foreign keys, which Serverpod's runner does with foreign keys off). A new synced table needs a row in that test, which fails until it has one.
- **Every synced table indexes `spaceId`** (`indexes: <table>_space_idx: fields: spaceId`, e.g. `studyset.spy.yaml`): deleting a space cascades into every synced table, and Postgres doesn't index foreign keys by itself, so account deletion and pruning would otherwise read whole tables.
- **The client's local migration** (in `halkyon_learn_client/lib/migrations/`, written by the same `create-migration`) creates the device's SQLite schema from the whole definition, so it also contains the server's own tables. Only the synced tables and the bookkeeping ever hold data on a device.
@ -143,4 +147,4 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **One person's data per device**: `PersonFolders` wipes every other person's folder when a person's data opens: their sessions ended on this device. Opening waits for the previous person's database to close.
- **Session logs are not persisted** (`persistentEnabled: false` in every run mode) for privacy. Don't re-enable them.
- **Secrets** live in `halkyon_learn_server/config/passwords.yaml` (git-ignored). Never change `serverSideSessionKeyHashPepper`: it would sign everyone out, and anonymous accounts can't sign back in.
- **Linux builds need** `libwebkit2gtk-4.1-dev` and `libsecret-1-dev`. If a Linux build tries to install to `/usr/local`, the CMake cache is stale: delete `build/linux`.
- **Linux builds need** `libwebkit2gtk-4.1-dev`, `libsecret-1-dev`, and for sounds (`audioplayers`) `libgstreamer1.0-dev` and `libgstreamer-plugins-base1.0-dev`. If a Linux build tries to install to `/usr/local`, the CMake cache is stale: delete `build/linux`.

View file

@ -71,7 +71,12 @@ revision's payload, and the per-side merge of ADR 0003 becomes per field.
otherwise), at a path derived from the space and the hash, and is served only by an
endpoint that checks the owner. Devices cache files by hash, and the hash checks each
download. The same file added twice is stored once; two rows for it, from two devices
offline, count as one. Content refers to a file by the row's id.
offline, count as one. Content refers to a file by the row's id, as the URL
`asset:<id>` (amended 2026-10-02): any other URL in content is never fetched, so
content from elsewhere can't make devices call a server of its choice. A space's
owner and members may download its files; uploads go into the caller's own space,
and the server keeps an upload only if its content has the hash it was announced
with.
## Consequences
@ -87,8 +92,12 @@ revision's payload, and the per-side merge of ADR 0003 becomes per field.
row in each of our synced tables and checks that none is lost, and fails if a synced
table has no row there.
- Stored files outlive their rows unless deleted with them: deleting a person, or a
file nobody refers to anymore, has to delete the stored file too. That comes with the
file endpoints.
file nobody refers to anymore, has to delete the stored file too. Built with the file
endpoints (2026-10-02): `deletePerson` deletes the person's files in its transaction,
and a daily job deletes files no live row refers to, and uploads never finished, 7
days after they were last claimed (an upload may reach the server before its row;
asking for a file again claims it anew). Rows of files no content refers to any more
aren't deleted yet, so their files stay until the person goes.
- Media makes the database, and so its backups, grow. The storage can move to S3 or
similar later without changing the rows, since the path is derived from the hash.