halkyon/docs/adr/0019-private-studysets.md

# ADR 0019: Private studysets

- **Status:** decided 21 Oct 2026, by the developer (D1–D8 below).
- **Amends:** ADR 0002, by the way back it left open: a private studyset's content is no
  longer readable by the instance.
- **Builds on:** ADR 0017 (the account's keys; D5, private per studyset and by default),
  ADR 0004 (tags, settings, files), ADR 0005 (publishing, Explore's copies, the learner
  count).
- **Followed by:** ADR 0018 (the agenda), whose store uses the same keys.

## Context

ADR 0017 gave every account key pairs that only its devices can open, and decided (D5)
that a studyset, with everything inside it, is private or not, new ones private. ADR 0002
had kept `keyId` and `nonce` in the envelope for exactly this, with the server's features
skipping what they can't read.

What a studyset holds is spread over Learn's synced tables: the studyset's own payload;
its cards', notes' and steps' revisions; its learning paths and PDFs; the answers given in
its paths (a plain `answer` column); review and reading logs; the links to tags; and the
files its content names, kept per space by their hash. Tags and settings belong to the
person rather than to a studyset, yet hold content too: tag names (Jellyfin's show names
among them), each studyset's study tags, its Jellyfin ties.

The server reads payloads in four places: the Claude connector, the learner count (each
person's `countLearning` and the publication a copy came from), publishing (the tree, its
revisions, its files and its tags' names), and feedback (threads about others'
publications, readable by design). It checks that a file over 20 MB is a PDF. Every
change the sync engine records carries its time (the writing device's clock, to the
millisecond), its table and the columns it changed.

## Decision

1. **Privacy belongs to a top-level studyset** and holds for everything inside it. A
   topic moved into another tree takes on that tree's privacy, its rows encrypted anew on
   the device.
2. **Keys.** A private studyset has a data key of its own (ADR 0017's `DataKey`), and
   every person has one personal key besides, for their tags and settings. Each is sealed
   to the account's X25519 key and kept in a new synced table in the person's space
   (`content_key`), so keys arrive with the data, work offline and go with the space
   (D6). An anonymous person's keys are sealed to the device's own key pairs (ADR 0017,
   9). A person whose account has no keys at the register (signed in with GitHub, no
   password yet) gets nothing private until they set a password, which the app leads to
   (D8); then the library makes the offer of 7.
3. **What is encrypted.** With the studyset's key: the payloads of the studyset, of its
   cards', notes' and steps' revisions, of its learning paths and PDFs, and of the asset
   rows of its files; the answers in its paths (`learning_path_answer` gains `keyId` and
   `nonce`); and the files themselves (D5), stored under the hash of what is encrypted, so
   the same picture in two studysets is stored twice. With the personal key: the payloads
   of tags and of the settings. Each is ADR 0017's sealed content: XChaCha20-Poly1305 with
   the key's ID and the row (table, id, column) as additional data, so the instance can't
   move a payload to another row, and the author's Ed25519 signature, checked before
   anything is decrypted, which the payload column carries before the ciphertext. The
   plaintext inside is the payload as before, its `schemaVersion` with it.
4. **What stays readable** (D7): the studyset tree and positions; when a studyset changed,
   was starred, archived or deleted; which card was rated how, when, for how long and in
   which mode; the PDF pages reached; path answers' outcomes; file sizes; and, from the
   sync engine, when each row changed, how many rows each table holds and how they hang
   together. The privacy page lists it: the instance sees a private studyset's shape, size
   and rhythm, never its content.
5. **The server's features.**
   - Publishing refuses a private studyset and says how to open it up. A non-private
     release's tag names come from the author's device with the call (D2); the server
     checks they name exactly the tags its items carry.
   - The connector leaves private studysets out and tells Claude so.
   - The learner count: while a person's Count me is on, their device reports which
     publications their copies come from (D3), kept server-side and replaced with each
     report; the review log still gives the days studied. Count me itself moves into a
     setting the server keeps (D2), as the settings' payload is encrypted.
   - Feedback is unchanged: it is about others' publications, written to be read.
   - Dictionary lookups are unchanged: searches, a studyset's corrections (by its cards'
     entry ids) and the Watch screen's lines (unless the device holds the dictionaries)
     still go to the instance, as the privacy page says.
   - A file's upload says whether it is encrypted, and only a readable one is checked to
     be a PDF; the quotas count both alike.
6. **Changing it.** Making a studyset private encrypts each of its rows in place (a
   payload update, as blanking is) and its files anew; making it non-private decrypts
   them likewise. The readable copies in the instance's backups stay until the backups
   expire; the app says so where it asks.
7. **What exists already** stays as it is (D4): the library offers once to make every
   studyset private, and each studyset's menu changes it later. New studysets are
   private, copies from Explore too.
8. **Keys lost** (an account reset without the recovery phrase): private studysets
   whose keys no longer open show as locked, with Delete alone.
9. **Exports** decrypt on the device: an archive is readable wherever it is kept, which
   the export sheet says.
10. **Released with build 22** (D1), whose `minimumAppBuild` already locks older apps out,
    so the new synced table and columns cost no second lockout.

## Decisions (21 Oct 2026, the developer's)

- **D1** Ship with build 22: one lockout of older apps for passwords and private
  studysets alike.
- **D2** Tags and settings are encrypted with a personal key; a release's tag names come
  from the author's device; Count me becomes a setting the server keeps.
- **D3** Copies are counted by the device reporting their publications, only while Count
  me is on.
- **D4** Existing studysets stay as they are, with a single offer to make them all private.
- **D5** A private studyset's files are encrypted too.
- **D6** Data keys live in a synced table in the person's space.
- **D7** The shape, size and rhythm stay readable and are listed; encrypting the values
  (ratings, durations, positions, pages) would hide how well, not when or how much, as the
  sync engine records every change's time.
- **D8** An account without keys at the register keeps its studysets readable until it
  sets a password.

## Consequences

- Build 22 brings a migration of synced tables (`content_key`; `keyId` and `nonce` on
  `learning_path_answer`), explained with its SQL when made.
- Every payload codec reads and writes through the keys; a row whose key the device lacks
  is shown as locked, never half-read.
- ADR 0002's rule 4 (server features read payloads) now holds for what isn't private;
  rule 6's way back is taken.
- Making a large library private takes a sync of everything in it.
- Nobody, the operator included, can restore a private studyset whose keys are lost.

## As built (21 Oct 2026)

- **Files** (D5, built 21 Oct): a private studyset's file is sealed with its
  tree's key for its asset row before it is hashed; the row's hash and size
  and the file uploaded (`encrypted`, which skips the PDF check) are the
  sealed file's, and its sealed payload carries the file name and the hash
  of what was sealed (asset schema 2). The device keeps the sealed file to
  upload and check, and the opened one by its own hash for what reads files
  by path. A file is a row per key. Every place adding a file names its
  studyset (the editors through `StudysetFiles`, which every screen of a
  studyset and the PDF reader set; PDFs, screenshots, imports, copies,
  whose top studyset is now made before their files; accepted suggestions
  by their item's tree). Making a tree private or not, or moving a topic
  into another tree, adds its files anew under the tree's key, names the
  new rows in its content (whole `asset:` values and a PDF's file, never
  text) and blanks the old rows nothing else names.
- **The marker** (the developer's, 21 Oct): on the library's cards and the
  studyset's Share alike, a lock for private (published before or not), a
  shield with a warning for private but partly readable or unsealed (the
  reading of 22 Oct), an open lock for readable by the instance and not
  published, a globe for published. The locks open
  one dialog saying what that means and changing it (and, while not
  private, leading to the Publish sheet); the globe opens the Publish
  sheet.
- **Count me** stays in the person's settings, which follow them to every
  device, and the device mirrors it to the instance after each sync
  (`counted_learner`), reporting the copies only while it is on: the
  instance keeps the setting, as D2 asks, but the person's choice is still
  made where it was.
- **Tag names:** the instance refuses a name sent for a tag that isn't the
  person's and a release whose encrypted tag has none; since the reading of
  22 Oct the device sends only the names of the tags the release carries.
- **Locked rows:** a studyset whose key doesn't open shows as locked; its
  other rows read as blank, which the lists leave out; a tag or settings
  row that doesn't open is left out entirely, so nothing folds or deletes
  it on a device without the keys.
- **Identity binding:** each payload is sealed for its table, row id and
  column, and signed by the account; the signature goes before the
  ciphertext in the payload column.


## The reading (22 Oct 2026)

A second, independent reading found that the device decided what was
private, and which key to write with, from columns the instance controls
(the top studyset's `keyId`, `parentId`, `card.studysetId`), and trusted
every readable row; and that content stayed readable in ordinary use (cards
moved, pictures from the full-window editors, a rewrite cut short, a device
that hadn't heard yet). The developer decided:

- **R1** Private studysets hold against an instance that alters data, not
  only one that reads it.
- **R2** Rollback is guarded for the settings alone (a counter inside their
  sealed payload); other old sealed content handed back is integrity, not
  confidentiality, and is accepted.

What changed:

- **Privacy is a signed fact.** A data key's ID says what it is for
  (`s1.<studyset>.<ms>.<nonce>` a studyset's key, `p1.<ms>.<nonce>` the
  person's, `r1.<studyset>.<ms>.<nonce>` a record that the studyset was
  made readable, which ends its earlier keys); the ID is sealed with the
  key and signed by the account, so the row's columns are never trusted. A
  studyset is private while the latest signed record about it says so; the
  device keeps the latest it saw (`privacy.json` in the person's folder,
  never synced), so a record withheld or a row replayed changes nothing. A
  tree's key is the nearest studyset's with such a record, itself or above,
  so moving a private studyset under a readable one changes nothing.
- **Items keep their key.** The device keeps the key each item (a card, a
  note, a path, a step, a PDF, a step's answers, a studyset, a file) was
  last seen sealed with, and never writes it with a weaker one unless it
  moves the item itself or a signed record makes its tree readable.
- **Readable rows are signed or distrusted.** A device holding the keys
  signs every readable row it writes (the account's signature in place of
  the nonce, for its table, row and column). Where privacy holds (a private
  tree, an item once sealed, tags and settings once sealed), a readable row
  nobody signed is locked: never shown, folded or adopted. A signed one,
  from a device that hadn't heard yet, is shown and sealed anew.
- **Rewrites finish.** A rewrite is kept as under way (in the ledger) until
  done, writes the top studyset last, and holds writes into its tree
  meanwhile; as the data opens and after every sync that merged, unfinished
  ones are finished and every private tree is swept for rows not under its
  key. What stays unsealed (locked rows, files not had) shows on the marker
  as partly readable, with Finish in its dialog.
- **Moves.** Cards moved into a private tree are sealed with their files
  before they move; cards of a private tree are never moved into a
  readable one: the app asks to make the target private first. A private
  topic moved out of its tree (into a readable one, or to the top) keeps its
  privacy with a key of its own (this changes Decision 1: privacy can now
  be a topic's own).
- **Tags and settings** are sealed once with the personal key, all readable
  ones then; from then on a readable one nobody signed is left out, so the
  instance can't confirm a guessed name by having a sealed tag folded into
  it. Settings carry a counter (`HLKS`, 8 bytes, then the payload, inside
  the seal); one lower than seen is refused.
- **A device without the account's keys,** on an account that has them,
  writes nothing that would be private (new studysets, tags, settings are
  refused, and it says so), reports nothing for the learner count, and
  shows private studysets as needing the password here, not as lost
  (archiving stays). An anonymous person's keys kept but unreadable are
  never replaced.
- **Files.** The full-window editors carry the studyset they were opened
  from; where none is known, an account with keys adds no file (a new
  studyset's description takes pictures once it is made). A file no live
  row names is never uploaded. A suggestion from a private copy uploads
  readable copies of its files for the author; accepting swaps whole
  `asset:` values only. Old file rows are kept while a row this device
  can't open, or a sent suggestion, might name them.
- **The instance:** an upload said to be encrypted must start as a sealed
  file does (`HLKF`, version 1); a person's room counts the files a live
  row names and those claimed within the week of grace; copy reports are
  limited to 60 an hour; publishing gets only the release's tags' names.

Residual risks, written down:

- **Any device can be held at any earlier genuine record:** the instance
  can withhold the records made since a device last synced, so it acts on
  the latest it saw. A fresh device, which saw none, is the case that
  matters: shown a studyset never made private as far as it knows, and an
  old readable row, it can be made to write readable there. The person's
  other devices still see the signed records once they come.
- **First sealing of tags and settings** on each device trusts what is
  readable then: rows the instance planted before become the person's.
- **Old sealed content handed back** (a title, a card's earlier revision)
  shows as it was: integrity, not confidentiality (R2).
- **Structure and rhythm** stay readable as before (D7); a card's place in
  a tree is the instance's to change, which the item's own key keeps from
  mattering for what is written, not for where it shows.

## The short reading (22 Oct 2026)

A short reading of the reading's fixes, its findings confirmed by probes on
the real gateways; each probe is a test now.

- **Rewrites stay in their own keys.** A rewrite or sweep changes only rows
  under no key, under the target key, or under a key whose signed ID names
  the studyset rewritten; rows under another studyset's key (which the
  instance could have moved in) are left as they are. Only the person's
  own moves re-key rows of other keys.
- **Item keys can't be led astray.** An item's key is never replaced by a
  key a record ended, nor by another studyset's while the one kept holds,
  and an item is written with its own key before its tree's.
- **Pending rewrites** the latest record no longer asks for are dropped,
  so a make-readable cut short never runs over a later private record.
- **Failures stay small.** A file that won't move is counted and owed (the
  tree partly readable meanwhile); each step of the upkeep fails alone.
- **Settings carry a version vector** (each device's count of writes,
  `HLKV`): a row is a rollback only when it brings news from no device, so
  two devices' changes at once both count.
- **Cheaper upkeep:** sweeps look rows up by their key, skip those known
  unmovable, and ask for a sync only when they wrote.
- **One rewrite at a time,** each writing back only rows unchanged since it
  read them.
- **Records outrun the clock:** a new record is made later than the latest
  known about its studyset.
- **Fail-closed:** an unreadable `privacy.json` is kept aside and nothing
  private is written until the keys are opened again; whether the account
  has keys is the register's to say (`signInWays().password`), not the key
  rows'.
- **Publishing** is refused on the device where anything in the tree is
  sealed, before any tag's name goes out, and the Publish sheet lists the
  names of the tags a release sends before Publish.
- **Suggestions** carry the item's own id: a sealed file goes along readable
  only where it is under the key of that item's tree; another private
  tree's file, named by a row the instance planted, is left out.
- **Finish** counts the edits a device without the keys wrote readable while
  the studyset was made private, by kind, and asks: taken in, they are
  sealed as the person's; left, they stay locked.

Each test of a fixed finding was run on the code before (`a1adc18`) and
failed there; one of an older readable record against a newer private one
passed there too, as it guards what was right already.

## The final reading (22 Oct 2026)

A third reading found that an item learned its first key from any sealed
row pointed at it (the instance pointed a revision of private B at a card
of private A; the card's next edit was sealed under B's key, and making B
readable opened it), and that a device without the keys trusted a "no" from
the register forever. The readings kept finding variants, each closing one
path while most columns stayed unsigned, so the developer chose to bind
what decides keys and to test against a hostile instance:

- **What decides a key is bound into what is sealed or signed.** A sealed
  payload's additional data, and a readable row's signature, name the row it
  belongs to wherever that decides its key: a revision's card, note or
  step, an answer's step, a path's or a PDF's studyset (sealed format v2;
  nothing sealed before was released). A row the instance points at another
  item doesn't open there and reads as locked, and an item learns its key
  only from its own rows, before any write decides on one. A studyset's
  `parentId` isn't bound: a move on one device and an edit on another merge
  column by column, and the row would lock for nobody's fault. Instead each
  studyset on the way up decides on its own before any above it: its own
  record (private, or made readable) or the key its own row was seen sealed
  with, learned from that row before a write if not known yet; one whose
  row is sealed under a key not here is private, and a write into it is
  refused.
- **Before a write decides on a key,** the device reads the records and the
  tree anew: a studyset missing from a tree read before, or moved since,
  would look readable.
- **A device without the account's keys,** on an account that has them,
  writes no content at all: it opens no record, so every tree would look
  readable to it. That the account has keys is any sign, never taken back:
  the keys open here, the register's yes, key rows (remembered once seen),
  records or sealed tags and settings seen; the register's answer comes
  through the instance unsigned, so its no outweighs nothing and is asked
  again.
- **Settings** keep the newest versions seen (each row's vector that no
  other seen is newer than): a row older than any of them, or than another
  read with it, isn't read, so withholding a newer row and handing back an
  older one rolls nothing back; rows folded away here are never read again.
  Count me folds off where any row says off, so an instance handing back a
  genuine older on can't switch it on (the person switches it on again at
  worst). A row seen as newest that is gone, not folded here and covered by
  no row read (another device's fold always leaves one), leaves out every
  row it doesn't cover. The short reading's `HLKS` counter is gone.
- **Publishing** sends exactly the tag names the sheet showed; names that
  changed in the sync before Publish are refused on the device.
- **Finish** lists what it would take in, each edit with what it says, and
  takes in exactly those rows, each only while it still says what was
  shown.
- **Rewrites:** a pending one keeps whether it may take in what nobody
  signed (only the person's own make-private of a readable tree may; never a
  sweep), and a sweep never takes over the person's own. A sweep reads only
  rows not under the key already, yielding now and then; a write waits only
  for a rewrite of its own tree, reading for none. A tree stays partly
  readable while rows a sweep can't move remain, which are looked at again
  once more keys open.
- **A damaged `privacy.json`** is kept aside; once the keys open, the device
  goes on as a fresh one would, with every residual risk of a fresh device.

**The hostile-instance test** (`test/core/privacy/hostile_instance_test.dart`)
holds the invariant: seeded runs interleave what the person does on a
laptop, a phone and a device without the keys (studysets, topics, cards,
notes, paths, steps, answers, PDFs, files, moves, making private or readable,
tags, Count me, Finish, publishing, reading back) with what an instance that
alters rows does to the local database, as a sync would bring it (changing
any unsigned column, replaying older versions under their ids, pointing
revisions and answers elsewhere, planting unsigned rows and signed rows
copied from elsewhere, withholding, handing back or forging key records,
linking tags, withholding files, folding away and handing back settings,
answering the register no), each followed by the upkeep after a merge.
After every step it checks, against a model kept from the person's actions
alone, that no marker of content private when written is readable in any
row the device keeps or file it stores unless the person made it readable,
that a release sends only the tag names shown, and that Count me is on
only where the person turned it on last, on the device they did it on (any
other can be held at an earlier record, below). 50 seeds of 40 steps run
with `tool/check.sh` (about a minute); `HOSTILE_SEEDS` and `HOSTILE_STEPS`
run more. Suggesting from a copy isn't among its actions (a test of its own
covers it).

Run on the code before the final reading (`6179c98`), 117 of 200 seeds
broke the invariant: 73 by H2's class, content a device without the keys
wrote readable (seeds 0, 2, 3, …); the rest by what is listed below
(writes on a stale tree, a rollback of Count me, release names not shown).
H1 itself wasn't reached by chance there, in 1,000 seeds without the device
without the keys, nor on the final code with only the binding taken out:
its chain is five particular steps long, and on the old code the more
common faults end each seed first. Its own test fails on the old code.

Writing it found, beyond H1 and H2: a write deciding on a key from records
and a tree read before; a topic the instance moved to the top level, its
own row never read here; a studyset whose key record was withheld from a
device that never saw it private; a studyset made readable by its own
record taken as private once the instance moved it under a private one;
settings rolled back by withholding a newer row; and two queries that
failed at run time (`id IN` with nullable ids: a path's step revisions and
answers couldn't be read); then, over 400 more seeds, a settings row of
the device's own taken away so that another device's older on was read
alone. Each has a test of its own now.

Residual risks, as they stand:

- **Any device can be held at any earlier genuine record.** A fresh device
  shown an old readable version of a studyset, its records withheld, writes
  readable there; so does a device whose `privacy.json` was lost. A device
  the instance keeps from a later Count me off goes on reporting copies;
  the device that turned it off never does.
- **First sealing of tags and settings** on each device trusts what is
  readable then.
- **Old sealed content handed back** shows as it was (R2), except settings.
- **Structure and rhythm** stay readable (D7).
- **Tag links aren't signed:** the instance can link a private tag to an
  item of a studyset being published; the Publish sheet shows every name
  before it goes.