feat(publish): releases in git (public tier, stage 2)
A person with an identity and a public profile publishes a studyset with
everything inside it as its next release (ADR 0005, rule 3): the server
reads their synced rows and writes them through the git command as a
commit on main and the annotated tag vN, in a bare repository of the
studyset's own, named by a random public id made at the first publish.
A release is a directory per studyset with a file per card, note, path,
step and PDF (its id, position, tag names and payload as synced, keys
sorted and indented) and the files the content names, without what is
only the author's (a card's report, a PDF's deadline). Publishing is
refused without a licence allowing passing it on in every studyset ("no
derivatives" allowed), with revisions to settle, with files not uploaded
yet, when nothing changed, or for a name that won't do; the name is
fixed from the first release, as is the person's handle.
In the app, Share in a studyset's action row opens the Publish sheet;
publishing syncs first, and settles by itself the revisions that merge
cleanly, listing only real conflicts. The server's image gets git, the
repositories a volume of their own, and the hourly backup a git bundle of
each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
7f6147d689
commit
f635b0816a
1 changed files with 20287 additions and 77 deletions
|
|
@ -121,7 +121,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
|
|||
|
||||
- **CI** (`.forgejo/workflows/ci.yaml`) runs `tool/check.sh` on every push, on the Forgejo runner on sahaquiel (`deploy/runner/`), except for a commit that passed before: the job marks passed commits in the runner's home (`checked/`, kept 90 days), so a branch's commit fast-forwarded into `main` goes straight to its deploy. Jobs run directly on the host as the unprivileged user `runner`, never in containers: Docker access would be root on the production server. The runner's systemd unit keeps jobs away from production (no `/home`, no Docker socket, Docker's container networks unreachable, memory and CPU capped); keep it that way when changing it. `runner` has user ID 61000 and must never share one with a container's processes (PostgreSQL and the server run as 999, not remapped): the same user ID can read their environment and files. The job's first step (*Sandbox*) fails if any of this leaks. Flutter is pinned in the workflow (`FLUTTER_VERSION`): move it together with the local one. The server tests there get throwaway passwords from the environment (`SERVERPOD_…`, new for every job), since `passwords.yaml` is never committed; without them they exit with code 1 and no message (Serverpod's own template does the same). CI also runs `serverpod generate` and fails if the committed generated code differs from its output. So never run `dart format` over the server's `lib/` or `test/` as a whole: it reformats the generated code (`lib/src/generated/`, `serverpod_test_tools.dart`); format the files you wrote. CI runs on pushes only: a pull request from a fork would run its code on the production server.
|
||||
- **A green `main` deploys learn.halkyon.eu** through `deploy/redeploy.sh`, the only command the runner's key may run as `deploy` (pinned in `authorized_keys`, from sahaquiel itself only). It deploys the commit CI checked while it is still the tip, as a fast-forward, only if every new commit is signed by a key in `deploy/allowed-signers` (the deployed copy decides), after dumping the database (the last 3 dumps stay in `~deploy/backups`: the way back from a deploy, not backups), applies the migrations in a run of their own before switching (`--role=maintenance --apply-migrations`: a failing migration stops the deploy while the old server keeps serving, so migrations must keep the old server working until the switch), and fails unless the new server answers. A failed deploy leaves the checkout at the deployed commit; run the workflow again to retry. By hand: `ssh sahaquiel-deploy halkyon-learn/deploy/redeploy.sh <commit>`. Never `docker compose down -v` there.
|
||||
- **Backups** (`deploy/backups.md`): every hour, `deploy/backup.sh` (a systemd timer, as `deploy`) dumps the database, which holds people's files too, and restic stores it in a Hetzner Storage Box (24 hourly, 7 daily, 4 weekly, 3 monthly); the box's own snapshots protect it from a taken-over server. A restore takes the instance back up to an hour, and people who started since lose what their device held: their app no longer finds its session, starts a new anonymous account and deletes the old folder. So a change that makes devices lose their session, or keeps deleted people's data longer, changes what a restore costs; say so in `deploy/backups.md`.
|
||||
- **Backups** (`deploy/backups.md`): every hour, `deploy/backup.sh` (a systemd timer, as `deploy`) dumps the database, which holds people's files too, then bundles each published studyset's repository (`git bundle`, from the server's `repositories` volume), and restic stores both in a Hetzner Storage Box (24 hourly, 7 daily, 4 weekly, 3 monthly); the box's own snapshots protect it from a taken-over server. A restore takes the instance back up to an hour, and people who started since lose what their device held: their app no longer finds its session, starts a new anonymous account and deletes the old folder. So a change that makes devices lose their session, or keeps deleted people's data longer, changes what a restore costs; say so in `deploy/backups.md`.
|
||||
- **Releases** are the project's, not an instance's: the release, the Snap Store listing and the website point to learn.halkyon.app; the builds connect to learn.halkyon.eu, the project's instance, until the app lets people choose one. They come from a developer's laptop, never from CI: `tool/release.sh` on `main` as pushed and green in CI. Versions follow semantic versioning, worked out from the commits since the last release (`feat` raises the minor number, `fix`/`perf` the patch number, breaking ones the major from 1.0 on; other types release nothing), so **commit messages keep their type prefix**. It writes the version and a raised build number into the app's pubspec in a signed release commit (`chore(release): vX.Y.Z`) with a signed tag (a key in `deploy/allowed-signers`), builds one universal APK from a checkout of the tag, signed with the app's key, and the snap (`snap/snapcraft.yaml`, the committed HEAD, built in LXD), both with `SERVER_URL=https://learn.halkyon.eu/`; signs `SHA256SUMS` with GPG; pushes only once both builds passed their checks; and publishes on code.halkyon.app through Forgejo's API and in the F-Droid repository. The Snap Store upload is off until `snap_store_channel` in the script names a channel. A run that stopped after its release commit and tag carries on with them. The key's password and the Forgejo token are asked for at each run and never stored. The app's key is a PKCS #12 keystore outside the repository (also in the maintainer's password manager); there is no Google Play, so it is the app's identity: lost, installed apps can't be updated. Its certificate's SHA-256 is in `tool/release/android-certificate.sha256`, and the script releases no APK signed otherwise; other release builds fall back to the debug key (`android/app/build.gradle.kts`), so `flutter run --release` works anywhere. The APK also goes into the project's own F-Droid repository, https://halkyon.app/fdroid/repo, through `tool/fdroid_repo.sh` (fdroidserver v2.4.5): its settings and the app's listing are in `tool/release/fdroid/` (the anti-features TetheredNet and NonFreeNet declared with their reasons and F-Droid's own icons); its index is signed with a key of its own (a PKCS #12 keystore beside the app's, password asked for, certificate pinned in `tool/release/fdroid-index-certificate.sha256`, since a new key makes the F-Droid app refuse the repository); and it is uploaded to ramiel as the user `fdroid`, whose key can only write into the folder Caddy serves (rrsync), the APKs first and the indexes last.
|
||||
|
||||
## Rules the code relies on
|
||||
|
|
@ -157,6 +157,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
|
|||
- **Serverpod's version moves together with `serverpod_offline_sync`**, which allows only a narrow range of Serverpod (v0.0.10: `>=4.0.4 <4.1.0`; v0.0.8 pinned v4.0.0 exactly, which the root `pubspec.yaml` overrode until 2026-10-09). Every package pins Serverpod's packages to one version: upgrade them all together with the sync packages and the CLI version CI generates code with (`SERVERPOD_VERSION` in `.forgejo/workflows/ci.yaml`), after reading the sync package's changelog. Never override a version the sync package doesn't allow without checking what changed in `serverpod_database`, which it builds on.
|
||||
- **The source-code link**: both licences require offering the source of what an instance runs to the people who use it. `instance.info().sourceCodeUrl` names it (the operator's `SOURCE_URL`, else this project's repository) and the app's About dialog shows it next to the licences and every package's notice. Keep it working when changing the dialog or the endpoint.
|
||||
- **Public profiles** (ADR 0005, stage 1 of the public tier): `public_profile` (`serverOnly`, never sent; the app gets a `Profile`, without the person's id) holds a handle (Mastodon's rules: 1 to 30 letters, digits or underscores, unique whatever their case through `handleKey`, a few reserved for the instance; changeable until the first publish, `firstPublishedAt`), a display name, a bio, up to 4 links and an avatar. `ProfileEndpoint` (`identified` only: a profile is for others to see) reads, saves and removes the caller's own; `ProfilesEndpoint` finds anyone's by handle. Handles are on the account domain (`ACCOUNT_DOMAIN`, else the server's public host; `instance.info().accountDomain`), which can never change once anyone follows a handle on it. An avatar is a PNG of at most 1 MB that the app draws anew from the picture chosen (`avatarPng`, so a photo's metadata never leaves the device), kept in the public storage at `avatars/<SHA-256>.png`, named by its content so its URL tells nothing about whose it is, and deleted once no profile shows it. In the app, the Profile tab holds it as a form, for people with an identity.
|
||||
- **Publishing** (ADR 0005, stage 2 of the public tier): `PublishEndpoint` (`identified` only, and a public profile: a release names its author) publishes a studyset with everything inside it as its next release. The server reads the person's synced rows (`readReleaseFiles`, `lib/src/publish/`) and writes them through the `git` command (`ReleaseRepository`, never a library linked in) as a commit on `main` and the annotated tag `v<N>` in a bare repository of the studyset's own, `<REPOSITORIES_DIR>/<public id>.git` (`published_studyset`, `serverOnly`; the public id is random, made at the first publish, never a synced row's id). A release holds a directory per studyset (the published one at the top, each inside it named by its id) with `studyset.json`, `cards/<id>.json`, `notes/<id>.json`, `paths/<id>/path.json` with `steps/<id>.json`, `pdfs/<id>.json`, and the files the content names in `files/<sha256>`, described in `files.json`; each item file holds its id, position, tag names and payload as synced, keys sorted and indented, without what is only its author's (a card's `report`, a PDF's `deadline`), and items being deleted (blank payloads) are left out. Publishing is refused (`PublishRefusedException`) without a profile, unless every studyset of it is under a licence allowing it to be passed on (`release_license.dart`; "no derivatives" allowed, and marked in the app), with a card, note or step of more than one current revision, with files not uploaded yet, when nothing changed, or for a name (`slug`: lower-case letters, digits and hyphens, one per studyset of a person, fixed from the first publish) that won't do. Versions count past both the database's highest and the tags', so a version git has and the database lost is never given again, and a release the database can't record is taken back from git. The first publish fixes the person's handle. Git runs apart from the machine's configuration (no system or user configuration, no prompts) with `gc.auto` off, so objects are never removed. Deleting a person deletes their published studysets with their repositories. In the app, Share in a studyset's action row (a lock, or a globe once published; an icon below medium windows) opens the Publish sheet (`feature/publish/`): its name, whether its licence allows publishing, what changed, its releases. Publishing syncs first, since the server builds the release from what it holds; on a refusal for revisions to settle, the app saves merged those whose changes merge (ADR 0003) and tries once more, and lists those changed differently on one side, each opening where it is settled.
|
||||
- **Deleting a person** goes through `deletePerson` (`lib/src/auth/account.dart`), in one transaction and in ADR 0001 rule 6's order: their sessions end first (a device with a valid session would re-upload everything), then their space goes with all its content through the foreign keys, then the person with their identities. `AccountEndpoint.delete` deletes the caller; the app also calls it when an anonymous account is given up (signed out of, or switched away from to GitHub), since nobody could ever sign into it again.
|
||||
- **Pruning**: `PruningFutureCall` deletes anonymous people unused for 90 days, daily, scheduled when the server starts. Scheduling a future call always adds an entry, so the server cancels the previous one by its identifier first.
|
||||
- **One person's data per device**: `PersonFolders` wipes every other person's folder when a person's data opens: their sessions ended on this device. Opening waits for the previous person's database to close.
|
||||
|
|
|
|||
Loading…
Reference in a new issue