perf: what everyone follows is read once, and two server hot spots
Studysets, settings, tags and every studyset's progress are now a LatestValue (core/async/): a screen starting to follow gets the value last read, and only the first has it read. Before, every screen opening had the whole table read again and handed to every screen open. While nobody follows, nothing is kept or read, so the tags are no longer read after every card change with no tag screen open. StudysetRepository serialises its reads with SerialQueue instead of a kept future chain. The card browser, path runner and path list blocs no longer add events from stream callbacks once closing, which the later arrival of the tags made one test hit. Server: publishing gathers each item's tag names in one pass instead of scanning every assignment per item, and deleting a person deletes their stored files without reading their bytes back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
301751607d
commit
bbc357f3ef
1 changed files with 335 additions and 104 deletions
|
|
@ -144,7 +144,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
|
|||
- **Serverpod's own helpers aren't guarded**: `ServerSideSessions.revokeSession` deletes by id alone, and `listSessions` without `authUserId` returns everyone's sessions. Never pass them anything a client sent.
|
||||
- **Granting a scope** goes through a function like `grantIdentified`, which updates the person *and* their session rows.
|
||||
- **Content envelope**: synced content tables carry `keyId` and `nonce` (nullable; `null` = plaintext, the only value, ADR 0002 rule 6) and `payload` (`ByteData`): UTF-8 JSON with a `schemaVersion`, encoded and decoded by one codec per feature (e.g. `studyset_payload.dart`) on top of `core/content/payload.dart`, which also holds the blanked `redactedPayload`. Content fields live inside `payload`: a new field raises the schema version instead of migrating a synced table, which would lock older app versions out of syncing (ADR 0002, rule 3). A payload in a newer schema is skipped with a log, never shown half-read.
|
||||
- **Synced data** (`database: sync`, `serverpod_offline_sync`, ADR 0001): every person has their own local SQLite database (`<app support>/people/<authUserId>/sync.sqlite`). Its gateway writes through the `OfflineSyncDatabaseSession`, **names the columns of every update**, and calls `SyncService.requestSync()` after every write; the repository re-reads after its own writes and when a sync merged changes (`remoteChanges`). **What is read is kept until it changes** (decided 2026-10-17; on a copy of a real library, 8,000 cards and 21,000 reviews, every screen's counts took over a second after each change, now about 65 ms once): `CardRepository` keeps each studyset's cards and every card's stamp until a card changes, `ReviewRepository` the whole log once many cards' reviews are asked for, updated by this device's writes and read anew after a sync brings others', and `StudysetProgressRepository` works out every studyset's progress once for the library, every studyset open and its topics, handing the last to whoever follows it next. A read begun before a change never fills a cache (each counts the changes), and a cache keeps values, never futures. A test reading right after a change that reaches the repository another way (a fake remote change, a bloc's write still under way) lets the event queue run first. `SyncService` syncs on triggers (at start, 2 s after the last write, on resume, every 5 min), one sync at a time. **Deleting blanks the payload first** (ADR 0002, rule 2): the engine keeps deleted rows until the space is deleted. Since blanking can't be undone, an Undo comes before it: a moment's for cards, notes, paths and PDFs (`DelayedDeletions`), 14 days for a studyset (ADR 0007, decided 2026-10-12, as Vaia keeps them): `StudysetDeletions`, opened with the person's data, marks it deleted (`studyset.deletedAt`), and it waits in the library's Recently deleted with everything inside it, restorable, while every other list, count and session reads without it (`StudysetRepository.all` and `watchAll` leave out what is deleted and inside it, `notDeleted`; `withDeleted` reads everything); once the 14 days are over, whichever of the person's devices looks next (as their data opens, after every sync) deletes the whole subtree for good, what is inside first, each studyset's cards blanked before it, and Delete now does at once after asking. Deleting doesn't ask, offering Undo. Widgets delete, undo and delete for good through it directly (`deleteStudysetWithUndo`, `deleteStudysetForGood`), not through a bloc: deleting a studyset from its own screen closes that screen and its bloc, and the deletion and its Undo must outlive them. The server never deletes them (it can't blank a payload; ADR 0007), and its connector, publishing and learner count leave them out too (`notDeleted` in `lib/src/studyset/`). Tags differ, being values (a tag is its name whatever its case, which it keeps; ADR 0004 rule 2): one is blanked and deleted at once, and its Undo makes a tag of that name and colour anew, on the same cards. Order by the UUID v7 id (ADR 0001, rule 7). Access to synced rows is the engine's: a person syncs their personal space and the spaces granted to them; the endpoint rules above guard everything that isn't synced.
|
||||
- **Synced data** (`database: sync`, `serverpod_offline_sync`, ADR 0001): every person has their own local SQLite database (`<app support>/people/<authUserId>/sync.sqlite`). Its gateway writes through the `OfflineSyncDatabaseSession`, **names the columns of every update**, and calls `SyncService.requestSync()` after every write; the repository re-reads after its own writes and when a sync merged changes (`remoteChanges`). **What is read is kept until it changes** (decided 2026-10-17; on a copy of a real library, 8,000 cards and 21,000 reviews, every screen's counts took over a second after each change, now about 65 ms once): `CardRepository` keeps each studyset's cards and every card's stamp until a card changes, `ReviewRepository` the whole log once many cards' reviews are asked for, updated by this device's writes and read anew after a sync brings others', and `StudysetProgressRepository` works out every studyset's progress once for the library, every studyset open and its topics. What everyone follows (the studysets, the settings, the tags, the progress) is a `LatestValue` (`core/async/`): a screen starting to follow gets the value last read, and only the first has it read, rather than every screen opening having it read again for every screen open; while nobody follows, nothing is kept or read. A read begun before a change never fills a cache (each counts the changes), and a cache keeps values, never futures. A test reading right after a change that reaches the repository another way (a fake remote change, a bloc's write still under way) lets the event queue run first. `SyncService` syncs on triggers (at start, 2 s after the last write, on resume, every 5 min), one sync at a time. **Deleting blanks the payload first** (ADR 0002, rule 2): the engine keeps deleted rows until the space is deleted. Since blanking can't be undone, an Undo comes before it: a moment's for cards, notes, paths and PDFs (`DelayedDeletions`), 14 days for a studyset (ADR 0007, decided 2026-10-12, as Vaia keeps them): `StudysetDeletions`, opened with the person's data, marks it deleted (`studyset.deletedAt`), and it waits in the library's Recently deleted with everything inside it, restorable, while every other list, count and session reads without it (`StudysetRepository.all` and `watchAll` leave out what is deleted and inside it, `notDeleted`; `withDeleted` reads everything); once the 14 days are over, whichever of the person's devices looks next (as their data opens, after every sync) deletes the whole subtree for good, what is inside first, each studyset's cards blanked before it, and Delete now does at once after asking. Deleting doesn't ask, offering Undo. Widgets delete, undo and delete for good through it directly (`deleteStudysetWithUndo`, `deleteStudysetForGood`), not through a bloc: deleting a studyset from its own screen closes that screen and its bloc, and the deletion and its Undo must outlive them. The server never deletes them (it can't blank a payload; ADR 0007), and its connector, publishing and learner count leave them out too (`notDeleted` in `lib/src/studyset/`). Tags differ, being values (a tag is its name whatever its case, which it keeps; ADR 0004 rule 2): one is blanked and deleted at once, and its Undo makes a tag of that name and colour anew, on the same cards. Order by the UUID v7 id (ADR 0001, rule 7). Access to synced rows is the engine's: a person syncs their personal space and the spaces granted to them; the endpoint rules above guard everything that isn't synced.
|
||||
- **Content people edit is append-only revisions** (ADR 0001 rule 2, ADR 0003; the reference is `CardRevision`): a revision is never updated except to blank it, and `parents` lists the revisions it replaces. A save's parents are all current heads, which is also how a conflict is resolved. `mergeRevisions` (`feature/card/model/card_merge.dart`) merges several heads per side against their newest common ancestor; a side changed differently is a conflict the person settles, never merged silently. Deleting blanks every revision, then deletes.
|
||||
- **Reviews are an append-only log** (`ReviewLog`: card, rating, `reviewedAt` in UTC, `durationMs`); taking back a rating deletes its row. Besides ratings it holds a fresh start (`reset`), which never counts as studying; older app versions skip a mode they don't know. A studyset's Reset progress asks what of it starts afresh, naming only what has progress (decided 2026-10-12): its flashcards (a fresh start each), reading its PDFs (a `reset` in their reading log) and its learning paths (a `reset` answer per step), one Undo taking all of it back. A burial isn't in the log but on the card: `card.buriedAt` sets it aside until the next local day begins on the device looking, out of every queue and count (Unbury clears it), read with the cards every list reads anyway. A card's schedule is never stored: `scheduleOf` (`feature/study/model/card_schedule.dart`) replays its reviews in time order with FSRS-6 (`fsrs`, pinned exactly), and only the final due date is fuzzed, from the card's id and review count, so every device computes the same. Studying a studyset covers everything inside it but archived studysets (`studiedIn`), and the study queue (`study_queue.dart`) takes each studyset's cards as a group: due cards of all of them earliest first, then each studyset's new cards in tree order, within each studyset's daily limits of new cards and reviews (its study options, `StudysetStudyOptions` in its payload, schema 3; without, the person's new cards a day and no limit on reviews; cards in their learning steps come anyway), and the studied studyset's limits hold for everything inside it together, as Anki's do for a deck with its subdecks (decided 2026-10-17: a language's twenty topics started 20 new cards a day each), in the order the studied studyset's options ask for (reviews first, mixed, new first). **Exam plans** (`feature/studyset/model/exam_plan.dart`, decided 2026-10-14): a studyset plans for exams (`examPlans` in its payload, schema 7: each a name, the topics it takes whole by their ids, the tags whose cards it takes by their names), its author's published with it (the copier and the updater point the topics at the copy's studysets), and plans are offered from level tags too (`automaticExamPlans`: JLPT, HSK, KRDict's levels, each taking the levels below it, ids `auto:…`). The person's choice (`exam`: the plan's id and the exam's day, `YYYY-MM-DD`) is theirs alone, never published, like a PDF's deadline. `ExamCoverage` reads what a plan covers (its topics' studysets with their paths and PDFs, its tags' cards and learning paths anywhere in the studyset; a path's tags offer automatic plans as a card's do, decided 2026-10-17); the Study plan shows how far the person has come with it and the pace (`examPace`: what is new spread so that it is learnt a week before the exam, or halfway there when it is closer; new cards counted as the day began), and Study for the exam (`StudyFocus.exam`) studies what it covers alone, as one group whose new cards a day are that pace, reviews unlimited; once the exam's day is over, neither shows, and a session for it studies as usual. The library's and the topics' counts and badges add up the same way, each card on the schedule a session studies it on (`card_study_mode.dart`: written by hand while Write by hand is on, back first while Back first is; `ReviewRepository.schedulesAsStudied`); Home's due forecast still counts front to back, as it reads cards without their content. Deleting a card deletes its reviews.
|
||||
- **Order is a fractional index** (`fractional_indexing_dart`, rocicorp's algorithm) in its own column, last writer wins. Two devices appending at once can pick the same key, so ties sort by id.
|
||||
|
|
|
|||
Loading…
Reference in a new issue