refactor(app): studysets and settings are written in the newest schema

Decided 2026-10-07: the codecs write the newest schema every time, as
before, rather than the oldest one that holds what is there. An older
app loses sight of a studyset or of the settings once a newer one has
written them, until it is updated too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-07 10:21:58 +02:00
commit ad28a5d8c2
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64

View file

@ -135,15 +135,15 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **The reference is `DevicesEndpoint`** (`halkyon_learn_server/lib/src/auth/`) with its tests: the owner check sits in the same query (`WHERE "id" = $1 AND "authUserId" = $2`), zero rows means `NotFoundException` (shared, `lib/src/common/`), and every rule has a test that asserts the refusal. Each query carries its SQL as a comment.
- **Serverpod's own helpers aren't guarded**: `ServerSideSessions.revokeSession` deletes by id alone, and `listSessions` without `authUserId` returns everyone's sessions. Never pass them anything a client sent.
- **Granting a scope** goes through a function like `grantIdentified`, which updates the person *and* their session rows.
- **Content envelope**: synced content tables carry `keyId` and `nonce` (nullable; `null` = plaintext, the only value, ADR 0002 rule 6) and `payload` (`ByteData`): UTF-8 JSON with a `schemaVersion`, encoded and decoded by one codec per feature (e.g. `studyset_payload.dart`) on top of `core/content/payload.dart`, which also holds the blanked `redactedPayload`. Content fields live inside `payload`: a new field raises the schema version instead of migrating a synced table, which would lock older app versions out of syncing (ADR 0002, rule 3). Where content doesn't use the newest field, a codec may write the older schema, which older apps still read (the studyset's study options: schema 3 only for a studyset that has them). A payload in a newer schema is skipped with a log, never shown half-read.
- **Content envelope**: synced content tables carry `keyId` and `nonce` (nullable; `null` = plaintext, the only value, ADR 0002 rule 6) and `payload` (`ByteData`): UTF-8 JSON with a `schemaVersion`, encoded and decoded by one codec per feature (e.g. `studyset_payload.dart`) on top of `core/content/payload.dart`, which also holds the blanked `redactedPayload`. Content fields live inside `payload`: a new field raises the schema version instead of migrating a synced table, which would lock older app versions out of syncing (ADR 0002, rule 3). A payload in a newer schema is skipped with a log, never shown half-read.
- **Synced data** (`database: sync`, `serverpod_offline_sync`, ADR 0001): every person has their own local SQLite database (`<app support>/people/<authUserId>/sync.sqlite`). Its gateway writes through the `OfflineSyncDatabaseSession`, **names the columns of every update**, and calls `SyncService.requestSync()` after every write; the repository re-reads after its own writes and when a sync merged changes (`remoteChanges`). `SyncService` syncs on triggers (at start, 2 s after the last write, on resume, every 5 min), one sync at a time. **Deleting blanks the payload first** (ADR 0002, rule 2): the engine keeps deleted rows until the space is deleted. Since blanking can't be undone, an Undo comes before it: `StudysetDeletions`, opened with the person's data, keeps every studyset deletion that can still be undone; lists leave out what it reports as pending, and once the time to undo is over it deletes the whole subtree, what is inside first, each studyset's cards blanked before it. Widgets delete and undo through it directly (`deleteStudysetWithUndo`), not through a bloc: deleting a studyset from its own screen closes that screen and its bloc, and the deletion and its Undo must outlive them. It closes first with the person's data and drops what still waits. Tags differ, being values (a tag is its name, ADR 0004 rule 2): one is blanked and deleted at once, and its Undo makes a tag of that name and colour anew, on the same cards. Order by the UUID v7 id (ADR 0001, rule 7). Access to synced rows is the engine's: a person syncs their personal space and the spaces granted to them; the endpoint rules above guard everything that isn't synced.
- **Content people edit is append-only revisions** (ADR 0001 rule 2, ADR 0003; the reference is `CardRevision`): a revision is never updated except to blank it, and `parents` lists the revisions it replaces. A save's parents are all current heads, which is also how a conflict is resolved. `mergeRevisions` (`feature/card/model/card_merge.dart`) merges several heads per side against their newest common ancestor; a side changed differently is a conflict the person settles, never merged silently. Deleting blanks every revision, then deletes.
- **Reviews are an append-only log** (`ReviewLog`: card, rating, `reviewedAt` in UTC, `durationMs`). A card's schedule is never stored: `scheduleOf` (`feature/study/model/card_schedule.dart`) replays its reviews in time order with FSRS-6 (`fsrs`, pinned exactly), and only the final due date is fuzzed, from the card's id and review count, so every device computes the same. Studying a studyset covers everything inside it but archived studysets (`studiedIn`), and the study queue (`study_queue.dart`) takes each studyset's cards as a group: due cards of all of them earliest first, then each studyset's new cards in tree order, within each studyset's daily limits of new cards and reviews (its study options, `StudysetStudyOptions` in its payload; without, the person's new cards a day and no limit on reviews; cards in their learning steps come anyway), in the order the studied studyset's options ask for (reviews first, mixed, new first). The library's and the topics' counts and badges add up the same way. Deleting a card deletes its reviews.
- **Reviews are an append-only log** (`ReviewLog`: card, rating, `reviewedAt` in UTC, `durationMs`). A card's schedule is never stored: `scheduleOf` (`feature/study/model/card_schedule.dart`) replays its reviews in time order with FSRS-6 (`fsrs`, pinned exactly), and only the final due date is fuzzed, from the card's id and review count, so every device computes the same. Studying a studyset covers everything inside it but archived studysets (`studiedIn`), and the study queue (`study_queue.dart`) takes each studyset's cards as a group: due cards of all of them earliest first, then each studyset's new cards in tree order, within each studyset's daily limits of new cards and reviews (its study options, `StudysetStudyOptions` in its payload, schema 3; without, the person's new cards a day and no limit on reviews; cards in their learning steps come anyway), in the order the studied studyset's options ask for (reviews first, mixed, new first). The library's and the topics' counts and badges add up the same way. Deleting a card deletes its reviews.
- **Order is a fractional index** (`fractional_indexing_dart`, rocicorp's algorithm) in its own column, last writer wins. Two devices appending at once can pick the same key, so ties sort by id.
- **Tree, tags, modes, settings and files** (ADR 0004): a studyset's `parent` and sibling `position` form the tree, and studysets in a cycle or without their parent show at the top level (`topLevel`); its `archivedAt`, `starredAt` and `updatedAt` are columns, and every change of a studyset writes `updatedAt` with it (the library lists starred studysets first, then the most recently changed); a card's tags are `card_tag` rows, one per assignment, never updated, and a duplicate counts once; `review_log.mode` is `review`, `reverse` (its own schedule) or `cram` (never moves a schedule), and a schedule replays the reviews of one mode; `person_settings` is meant as one row per person, and extra rows fold into the one with the smallest id; an `asset` row stands for a file in private cloud storage, found by its hash. **Never derive a row's id from a person or anything else guessable**: ids are unique across the whole table, so someone could claim it first in their own space and break the owner's sync.
- **Files** (ADR 0004, rule 5): `AssetEndpoint` (`requireLogin`) takes uploads into the caller's own space by SHA-256 (`startUpload` gives Serverpod's upload description, or null when the file is there already and claims it anew; `finishUpload` keeps it only if its content has that hash) and serves an asset row's file to its space's owner and members only, `NotFoundException` otherwise, also for a deleted row (the sync engine hides tombstones). Files live in the private database storage at `spaces/<space>/<sha256>`, at most 20 MB each, and within limits (`file_limits.dart`, from the environment: an anonymous person 100 MB, an identified one 500 MB or what a `file_allowance` row grants them, the instance 15 GB), checked when an upload starts (unfinished uploads count, or they could fill the disk unnoticed) and again when it finishes (only what is kept counts), refused with `FileLimitExceededException`; `deletePerson` deletes the person's files and allowance, and `FilePruningFutureCall` deletes daily those no live row refers to (or never finished), 7 days after they were last claimed. In the app, `AssetRepository` (`core/asset/`) keeps files by hash in the person's folder (`files/`, uploads waiting marked in `files/uploads/`), uploads after each sync that went through, and downloads on demand, checking the hash; every hash is checked to be one before it becomes part of a path. Content names a file by the URL `asset:<asset row id>` (`assetUrl`, `RichContent.assetIds`): images are the editor's `image` block, sounds an `audio` block (`core/editor/file_blocks.dart`), and any other URL is never fetched. `StoredFileImage` loads through the repository like Flutter's `NetworkImage`, keyed by the person's repository too, so no image crosses people.
- **PDFs** (`feature/pdf_source/`): a `pdf_source` row per PDF in a studyset (payload: title, file name, the asset of its file, page count, reading deadline; last writer wins) and an append-only `pdf_reading` log (`reached` with a page, `reset`): the furthest page is the furthest reached since the last reset, so devices never conflict. A PDF's file is an asset of up to 50 MB (other files 20 MB): the server takes up to 50 MB and deletes a stored file over 20 MB that doesn't start with `%PDF-`. Deleting a PDF deletes its asset row too unless another PDF has the same file (asset rows are otherwise never deleted). Cards and notes tied to a page keep it in their payload (`source`, `core/content/pdf_page_ref.dart`; card schema 4, note schema 2); a screenshot is an `image` block with `pdf` and `page` attributes.
- **The Pomodoro timer** (`feature/pomodoro/`): the profiles follow the person (`PersonSettings.pomodoro`, settings schema 4, written as schema 3 while they are as they come), the running timer is the device's own (`PomodoroRun` in `<person folder>/pomodoro.json`, `PomodoroRunStore`). `PomodoroRun` is a pure state machine on the time: a phase that ends leads to the next, running if its profile starts it by itself, else paused at its full length; a timer kept while the app was closed catches up (`caughtUpTo`). `PomodoroBloc` sits above the router (`PersonApp`), and `PomodoroPresenter` hands every change to `PomodoroPlatform` as a `PomodoroNotice` in the app's language, and its buttons back. **Android:** the app's own Kotlin (`PomodoroService.kt`; MethodChannel `app.halkyon.learn/pomodoro`, EventChannel `…/buttons`): a `specialUse` foreground service with a countdown notification (chronometer), Pause/Resume, Skip, Stop, on Android 16 a Live Update (`NotificationCompat.ProgressStyle`, the cycle's phases as segments, `setRequestPromotedOngoing`); an exact alarm (`USE_EXACT_ALARM`) alerts at the phase's end with the app closed; Do Not Disturb through the interruption filter, the one before kept in shared preferences and restored. With the app swiped away its buttons open it (a service may not start an activity from a notification's button since Android 12, so the notification is posted again with activity intents), and Stop works alone, leaving a mark the app takes at its next start (`takeStoppedOutside`) to forget its timer; a profile's next phase only starts by itself while the app runs. Dart's timers sleep with the device, so `PomodoroBloc` also looks every 15 s while a phase runs, and the alarm's receiver wakes it (`phaseEnds`); a kept timer is caught up before it first shows, or its ended phase would alert again. No notification plugin: flutter_local_notifications had Live Updates only in a prerelease needing compileSdk 37 (checked 2026-10-07). **Linux:** `LinuxPomodoroPlatform` speaks `org.freedesktop.Notifications` over package:dbus (the snap's `desktop` plug allows it): a resident notification with the buttons, an alert with sound while the app runs, and GNOME's `show-banners` off while focusing, through `gsettings` (the value before kept in `<app support>/pomodoro_quiet_before`). xdg-desktop-portal's Background status is for Flatpaks only, so it isn't used.
- **The Pomodoro timer** (`feature/pomodoro/`): the profiles follow the person (`PersonSettings.pomodoro`, settings schema 4), the running timer is the device's own (`PomodoroRun` in `<person folder>/pomodoro.json`, `PomodoroRunStore`). `PomodoroRun` is a pure state machine on the time: a phase that ends leads to the next, running if its profile starts it by itself, else paused at its full length; a timer kept while the app was closed catches up (`caughtUpTo`). `PomodoroBloc` sits above the router (`PersonApp`), and `PomodoroPresenter` hands every change to `PomodoroPlatform` as a `PomodoroNotice` in the app's language, and its buttons back. **Android:** the app's own Kotlin (`PomodoroService.kt`; MethodChannel `app.halkyon.learn/pomodoro`, EventChannel `…/buttons`): a `specialUse` foreground service with a countdown notification (chronometer), Pause/Resume, Skip, Stop, on Android 16 a Live Update (`NotificationCompat.ProgressStyle`, the cycle's phases as segments, `setRequestPromotedOngoing`); an exact alarm (`USE_EXACT_ALARM`) alerts at the phase's end with the app closed; Do Not Disturb through the interruption filter, the one before kept in shared preferences and restored. With the app swiped away its buttons open it (a service may not start an activity from a notification's button since Android 12, so the notification is posted again with activity intents), and Stop works alone, leaving a mark the app takes at its next start (`takeStoppedOutside`) to forget its timer; a profile's next phase only starts by itself while the app runs. Dart's timers sleep with the device, so `PomodoroBloc` also looks every 15 s while a phase runs, and the alarm's receiver wakes it (`phaseEnds`); a kept timer is caught up before it first shows, or its ended phase would alert again. No notification plugin: flutter_local_notifications had Live Updates only in a prerelease needing compileSdk 37 (checked 2026-10-07). **Linux:** `LinuxPomodoroPlatform` speaks `org.freedesktop.Notifications` over package:dbus (the snap's `desktop` plug allows it): a resident notification with the buttons, an alert with sound while the app runs, and GNOME's `show-banners` off while focusing, through `gsettings` (the value before kept in `<app support>/pomodoro_quiet_before`). xdg-desktop-portal's Background status is for Flatpaks only, so it isn't used.
- **Imported cards** say where they came from in their first revision's payload (`origin`, card schema 3, e.g. `anki`); the dashboard doesn't count them as written. The Anki import (`feature/anki/`) reads `.apkg`/`.colpkg` packages of every format on an isolate of its own (`AnkiPackageReader`), zstd through our fork of `pure_zstd` (Anki's frames don't record their size), and its fixtures come from Anki itself (`tool/anki/make_fixtures.py`).
- **Every client migration keeps a device's rows** (`test/core/sync/device_migration_test.dart` replays each on a database with a row in every synced table of ours; SQLite rebuilds tables to add foreign keys, which Serverpod's runner does with foreign keys off). A new synced table needs a row in that test, which fails until it has one.
- **Every synced table indexes `spaceId`** (`indexes: <table>_space_idx: fields: spaceId`, e.g. `studyset.spy.yaml`): deleting a space cascades into every synced table, and Postgres doesn't index foreign keys by itself, so account deletion and pruning would otherwise read whole tables.