feat: PDF sources: read, take notes and cards from pages, deadlines
Some checks failed
CI / check (push) Failing after 26s
CI / deploy (push) Has been skipped

PDFs live in a studyset's new Files tab: uploaded from Add material
(a file the studyset holds already asks first), each row with its
pages, cards, notes and size, how far it is read and its reading
deadline; renamed, given a deadline (a page or a chapter by a day),
marked as read, started afresh, deleted with a moment to undo.

The reader (/pdfs/:id) opens where reading left off: on wide windows
a rail of tools (search, fit, zoom, fullscreen), a page counter with
the chapter, and a sidebar; on phones a top bar, a page scrubber and
a sheet. The sidebar holds the PDF's flashcards, by page or chapter,
made from selected text or written there, each tied to its page and
tagged with its chapter; and the page's note, an ordinary note tied
to the page. Screenshots of a region become images that know their
page. Cards and screenshots lead back to their page; reading on past
a chapter with cards suggests studying it. Home shows the deadlines
running and to come, and a timeline of them.

Server: pdf_source rows (content in the payload, last writer wins)
and an append-only pdf_reading log, the furthest page derived from
it so devices never conflict. Files up to 50 MB for PDFs (20 MB for
others; a larger file that isn't a PDF is deleted once stored). A
deleted PDF takes its file's asset row along unless another PDF has
the same file. Card payload 4 and note payload 2 keep the page.

The reader draws with pdfrx (PDFium) behind PdfEngine in
Dependencies; widget tests use a fake, and
integration_test/pdf_engine_test.dart checks the real engine on a
device or the desktop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-03 14:40:08 +02:00
commit b96db731c1
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64

View file

@ -84,7 +84,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **One graph per context.** The dependency graph belongs to one instance and one signed-in person; when either changes, a `KeyedSubtree` disposes the whole subtree and rebuilds it. Why: nothing from one person's session (blocs, streams, caches) can leak into another's.
- **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`.
- **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again.
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one).
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading PDFs (`PdfEngine`, pdfrx on PDFium, which `flutter test` can't load: widget tests use `FakePdfEngine`, whose PDF bytes say their pages and chapters), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one).
- **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`.
- **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point.
- **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`.
@ -139,6 +139,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **Order is a fractional index** (`fractional_indexing_dart`, rocicorp's algorithm) in its own column, last writer wins. Two devices appending at once can pick the same key, so ties sort by id.
- **Tree, tags, modes, settings and files** (ADR 0004): a studyset's `parent` and sibling `position` form the tree, and studysets in a cycle or without their parent show at the top level (`topLevel`); its `archivedAt`, `starredAt` and `updatedAt` are columns, and every change of a studyset writes `updatedAt` with it (the library lists starred studysets first, then the most recently changed); a card's tags are `card_tag` rows, one per assignment, never updated, and a duplicate counts once; `review_log.mode` is `review`, `reverse` (its own schedule) or `cram` (never moves a schedule), and a schedule replays the reviews of one mode; `person_settings` is meant as one row per person, and extra rows fold into the one with the smallest id; an `asset` row stands for a file in private cloud storage, found by its hash. **Never derive a row's id from a person or anything else guessable**: ids are unique across the whole table, so someone could claim it first in their own space and break the owner's sync.
- **Files** (ADR 0004, rule 5): `AssetEndpoint` (`requireLogin`) takes uploads into the caller's own space by SHA-256 (`startUpload` gives Serverpod's upload description, or null when the file is there already and claims it anew; `finishUpload` keeps it only if its content has that hash) and serves an asset row's file to its space's owner and members only, `NotFoundException` otherwise, also for a deleted row (the sync engine hides tombstones). Files live in the private database storage at `spaces/<space>/<sha256>`, at most 20 MB each, and within limits (`file_limits.dart`, from the environment: an anonymous person 100 MB, an identified one 500 MB or what a `file_allowance` row grants them, the instance 15 GB), checked when an upload starts (unfinished uploads count, or they could fill the disk unnoticed) and again when it finishes (only what is kept counts), refused with `FileLimitExceededException`; `deletePerson` deletes the person's files and allowance, and `FilePruningFutureCall` deletes daily those no live row refers to (or never finished), 7 days after they were last claimed. In the app, `AssetRepository` (`core/asset/`) keeps files by hash in the person's folder (`files/`, uploads waiting marked in `files/uploads/`), uploads after each sync that went through, and downloads on demand, checking the hash; every hash is checked to be one before it becomes part of a path. Content names a file by the URL `asset:<asset row id>` (`assetUrl`, `RichContent.assetIds`): images are the editor's `image` block, sounds an `audio` block (`core/editor/file_blocks.dart`), and any other URL is never fetched. `StoredFileImage` loads through the repository like Flutter's `NetworkImage`, keyed by the person's repository too, so no image crosses people.
- **PDFs** (`feature/pdf_source/`): a `pdf_source` row per PDF in a studyset (payload: title, file name, the asset of its file, page count, reading deadline; last writer wins) and an append-only `pdf_reading` log (`reached` with a page, `reset`): the furthest page is the furthest reached since the last reset, so devices never conflict. A PDF's file is an asset of up to 50 MB (other files 20 MB): the server takes up to 50 MB and deletes a stored file over 20 MB that doesn't start with `%PDF-`. Deleting a PDF deletes its asset row too unless another PDF has the same file (asset rows are otherwise never deleted). Cards and notes tied to a page keep it in their payload (`source`, `core/content/pdf_page_ref.dart`; card schema 4, note schema 2); a screenshot is an `image` block with `pdf` and `page` attributes.
- **Imported cards** say where they came from in their first revision's payload (`origin`, card schema 3, e.g. `anki`); the dashboard doesn't count them as written. The Anki import (`feature/anki/`) reads `.apkg`/`.colpkg` packages of every format on an isolate of its own (`AnkiPackageReader`), zstd through our fork of `pure_zstd` (Anki's frames don't record their size), and its fixtures come from Anki itself (`tool/anki/make_fixtures.py`).
- **Every client migration keeps a device's rows** (`test/core/sync/device_migration_test.dart` replays each on a database with a row in every synced table of ours; SQLite rebuilds tables to add foreign keys, which Serverpod's runner does with foreign keys off). A new synced table needs a row in that test, which fails until it has one.
- **Every synced table indexes `spaceId`** (`indexes: <table>_space_idx: fields: spaceId`, e.g. `studyset.spy.yaml`): deleting a space cascades into every synced table, and Postgres doesn't index foreign keys by itself, so account deletion and pruning would otherwise read whole tables.
@ -150,4 +151,6 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **One person's data per device**: `PersonFolders` wipes every other person's folder when a person's data opens: their sessions ended on this device. Opening waits for the previous person's database to close.
- **Session logs are not persisted** (`persistentEnabled: false` in every run mode) for privacy. Don't re-enable them.
- **Secrets** live in `halkyon_learn_server/config/passwords.yaml` (git-ignored). Never change `serverSideSessionKeyHashPepper`: it would sign everyone out, and anonymous accounts can't sign back in.
- **Checks on a device** live in `halkyon_learn_flutter/integration_test/` and run by hand, never in `tool/check.sh` or CI: `flutter test integration_test/<file> -d <device>`. `pdf_engine_test.dart` checks the real PDF engine (on the Linux desktop too), `youtube_player_test.dart` the embedded YouTube player (Android). Never on the emulator from an agent session: it takes the editor down with it.
- **Windows builds need Developer Mode** (pdfrx's build makes symbolic links). PDFium itself comes with the build as a Dart native asset.
- **Linux builds need** `libwebkit2gtk-4.1-dev`, `libsecret-1-dev`, and for sounds (`audioplayers`) `libgstreamer1.0-dev` and `libgstreamer-plugins-base1.0-dev`. If a Linux build tries to install to `/usr/local`, the CMake cache is stale: delete `build/linux`.