build(deploy): the server's checkout is ~deploy/halkyon, the repository's name
All checks were successful
CI / check (push) Successful in 40m10s
CI / deploy (push) Successful in 58s

The folder kept its name from before the monorepo (ADR 0015), though
Train and Todo deploy from it too. redeploy.sh, the backup's units and
the operator's steps name ~deploy/halkyon; the Compose project keeps the
name halkyon-learn, as Docker names its volumes after it.

The server moves after this is deployed: the running redeploy.sh is read
whole before it runs, so the deploy of this commit still works in the
old folder; then the folder is moved, the pinned commands in deploy's
authorized_keys and the backup's units changed, and the containers
recreated on the new paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-11-02 18:53:11 +01:00
commit 5469ce791f
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
6 changed files with 36 additions and 33 deletions

View file

@ -64,7 +64,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
## CI and deploys
- **CI** checks Learn with every other package (root AGENTS.md, *CI and deploys*): its server's, curator's and app's tests through Melos, its server's generated code through `generate`.
- **A green `main` deploys learn.halkyon.eu** through `deploy/redeploy.sh`, the only command `deploy`'s keys may run (pinned in its `authorized_keys`: the runner's from that server itself only, the developer's for deploys by hand). It deploys the commit CI checked while it is still the tip, as a fast-forward, only if every new commit is signed by a key in `deploy/allowed-signers` (the deployed copy decides), after dumping the database (the last 3 dumps stay in `~deploy/backups`: the way back from a deploy, not backups), applies the migrations in a run of their own before switching (`--role=maintenance --apply-migrations`: a failing migration stops the deploy while the old server keeps serving, so migrations must keep the old server working until the switch), and fails unless the new server answers. A failed deploy leaves the checkout at the deployed commit; run the workflow again to retry. By hand: `ssh sahaquiel-deploy <commit>`, the commit alone, as CI passes it (the server's checkout keeps its folder `halkyon-learn`, so the command the keys are pinned to stays). Never `docker compose down -v` there.- **From the single repository** (ADR 0015): learn.halkyon.eu deploys this repository's `main`, whichever app or shared package a push changed. Its images are built from the repository's root with only what they need (Learn's server and the module it builds on, with the module's client, which the module's tests depend on; the curator), each Dockerfile narrowing the workspace to those packages (`tool/workspace_only.sh`). The first deploy after the move starts the server once more than it needs to, as the deployed copy of `redeploy.sh` looks for migrations at their old path.
- **A green `main` deploys learn.halkyon.eu** through `deploy/redeploy.sh`, the only command `deploy`'s keys may run (pinned in its `authorized_keys`: the runner's from that server itself only, the developer's for deploys by hand). It deploys the commit CI checked while it is still the tip, as a fast-forward, only if every new commit is signed by a key in `deploy/allowed-signers` (the deployed copy decides), after dumping the database (the last 3 dumps stay in `~deploy/backups`: the way back from a deploy, not backups), applies the migrations in a run of their own before switching (`--role=maintenance --apply-migrations`: a failing migration stops the deploy while the old server keeps serving, so migrations must keep the old server working until the switch), and fails unless the new server answers. A failed deploy leaves the checkout at the deployed commit; run the workflow again to retry. By hand: `ssh sahaquiel-deploy <commit>`, the commit alone, as CI passes it (the server's checkout is `~deploy/halkyon`, renamed from `halkyon-learn` on 2 Nov 2026 with the command the keys are pinned to; the Compose project keeps the name `halkyon-learn`, as Docker names its volumes after it). Never `docker compose down -v` there.- **From the single repository** (ADR 0015): learn.halkyon.eu deploys this repository's `main`, whichever app or shared package a push changed. Its images are built from the repository's root with only what they need (Learn's server and the module it builds on, with the module's client, which the module's tests depend on; the curator), each Dockerfile narrowing the workspace to those packages (`tool/workspace_only.sh`). The first deploy after the move starts the server once more than it needs to, as the deployed copy of `redeploy.sh` looks for migrations at their old path.
- **Sidecars** (`deploy/sidecars/README.md`, decided 2026-10-14): services an instance may run beside itself, each a compose file of its own beside `deploy/compose.yaml` (`compose.jellyfin.yaml`, `compose.languagetool.yaml`; later Fedify), switched on in `deploy/.env` (`COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml`), so `redeploy.sh` deploys them with the instance. The instance's compose has two networks: `edge` (`halkyon-learn-edge`, 172.30.80.0/24 and the unique local fd4a:6c6b:7e80::/64, so Docker hands Caddy IPv6 visitors' own addresses), Caddy's at fixed addresses (172.30.80.2, fd4a:6c6b:7e80::2; the server's `TRUSTED_PROXIES`), shared with the server, and the internal default one, where the server reaches Postgres; a sidecar joins `edge` alone and publishes no port (the CI runner may reach `127.0.0.1`, never Docker's 172.16.0.0/12 or fd00::/8). A sidecar reached from outside mounts its site from `deploy/sidecars/` into Caddy's `/etc/caddy/sidecars/`, which the Caddyfile imports (none without), and `redeploy.sh` recreates Caddy when a site changes. Each runs as a user ID of its own (Jellyfin 61096, LanguageTool 61081), never one a person, another container or the runner uses; its files on the server are git-ignored (`deploy/jellyfin/`, `deploy/languagetool/`). The Jellyfin sidecar sets the server's `JELLYFIN_URL`, so the instance suggests it; the LanguageTool sidecar (ADR 0024) sets `LANGUAGETOOL_URL` and has no site, as LanguageTool has no sign-in: only the server reaches it, and logs nothing (`logging: none`).
- **Backups** (`deploy/backups.md`): every hour, `deploy/backup.sh` (a systemd timer, as `deploy`) dumps the database, which holds people's files too, then bundles each published studyset's repository (`git bundle`, from the server's `repositories` volume), and restic stores both in a Hetzner Storage Box (24 hourly, 7 daily, 4 weekly, 3 monthly); the box's own snapshots protect it from a taken-over server. A restore takes the instance back up to an hour, and people who started since lose what their device held: their app no longer finds its session and asks how to start, and an account made since is gone with it. So a change that makes devices lose their session, or keeps deleted people's data longer, changes what a restore costs; say so in `deploy/backups.md`.
- **Releases** are the project's, not an instance's: the release, the Snap Store listing and the F-Droid listing point to the project's page on Halkyon's website, halkyon.app/learn (its own repository, `project-halkyon/website`; learn.halkyon.app redirects there); the builds name learn.halkyon.eu, the project's instance (`SERVER_URL`), which a new install offers first (see *Instances*). They come from a developer's laptop, never from CI: `dart run tool/release.dart learn` on `main` as pushed and green in CI (the apps' shared release tool, ADR 0015, configured in `apps/learn/release.dart`; `--dry-run` shows the version, the commits that count, every step and the notes without changing anything). Build 21 (`v0.18.0`) was the last from `tool/release.sh`, the Bash script it replaced, removed since. Versions follow semantic versioning, worked out from the commits touching Learn since its last release (`apps/learn/`, the shared packages it uses, the root pubspec and lock, its release files, and its packages' folders at the root from before the move; `feat` raises the minor number, `fix`/`perf` the patch number, breaking ones the major from 1.0 on; other types release nothing), so **commit messages keep their type prefix**. It writes the version and a raised build number into the app's pubspec in a signed release commit (`chore(release): learn-vX.Y.Z`) with a signed tag (`learn-vX.Y.Z`, by a key in `deploy/allowed-signers`; Learn's earlier `vX.Y.Z` tags count as its history), builds one universal APK from a checkout of the tag, signed with the app's key, and the snap (`snap/snapcraft.yaml`, the committed HEAD, built in LXD), both with `SERVER_URL=https://learn.halkyon.eu/`; signs `SHA256SUMS` with GPG; drafts what's new in the release for the developer to edit, committed with the version in `assets/whats_new/` (Train's ADR 0030, D61: the sheet after the update, Settings' entry above About, the notes and F-Droid's changelog); pushes only once both builds passed their checks; and publishes on code.halkyon.app through Forgejo's API and in the F-Droid repository. The snap goes to the Snap Store's edge channel last (`storeChannel` in `apps/learn/release.dart`; the release refuses to start unless `snapcraft whoami` shows a login). A run that stopped after its release commit and tag carries on with them. The key's password and the Forgejo token are asked for at each run and never stored. The app's key is a PKCS #12 keystore outside the repository (also in the maintainer's password manager); there is no Google Play, so it is the app's identity: lost, installed apps can't be updated. Its certificate's SHA-256 is in `tool/release/android-certificate.sha256`, and the release tool releases no APK signed otherwise; other release builds fall back to the debug key (`android/app/build.gradle.kts`), so `flutter run --release` works anywhere. The APK also goes into the project's own F-Droid repository, https://halkyon.app/fdroid/repo, through `tool/fdroid_repo.sh` (fdroidserver v2.4.5): its settings and the app's listing are in `tool/release/fdroid/` (the anti-feature NonFreeNet declared with its reasons and F-Droid's own icon; not TetheredNet, as the app chooses its instance); its index is signed with a key of its own (a PKCS #12 keystore beside the app's, password asked for, certificate pinned in `tool/release/fdroid-index-certificate.sha256`, since a new key makes the F-Droid app refuse the repository); and it is uploaded to the halkyon.app server (`ssh ramiel-fdroid`) as the user `fdroid`, whose key can only write into the folder Caddy serves (rrsync), the APKs first and the indexes last.

View file

@ -6,7 +6,7 @@
# would protect little: it keeps to the basics.
[Unit]
Description=Off-site backup of learn.halkyon.eu's database
Documentation=file:///home/deploy/halkyon-learn/deploy/backups.md
Documentation=file:///home/deploy/halkyon/deploy/backups.md
After=network-online.target docker.service
Wants=network-online.target
@ -14,7 +14,7 @@ Wants=network-online.target
Type=oneshot
User=deploy
Group=deploy
ExecStart=/home/deploy/halkyon-learn/deploy/backup.sh
ExecStart=/home/deploy/halkyon/deploy/backup.sh
PrivateTmp=yes
NoNewPrivileges=yes
ProtectSystem=full

View file

@ -112,20 +112,20 @@ echo 'sftp:storagebox:restic' >~/.config/restic/repository
cat ~/.config/restic/password # into the password manager
export RESTIC_REPOSITORY_FILE=~/.config/restic/repository RESTIC_PASSWORD_FILE=~/.config/restic/password
restic init
~/halkyon-learn/deploy/backup.sh # the first backup, by hand
~/halkyon/deploy/backup.sh # the first backup, by hand
```
### 4. The timer (an admin with sudo, on the server)
```sh
sudo cp /home/deploy/halkyon-learn/deploy/backup/halkyon-backup.service \
/home/deploy/halkyon-learn/deploy/backup/halkyon-backup.timer /etc/systemd/system/
sudo cp /home/deploy/halkyon/deploy/backup/halkyon-backup.service \
/home/deploy/halkyon/deploy/backup/halkyon-backup.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now halkyon-backup.timer
systemctl list-timers halkyon-backup.timer
```
The units name `/home/deploy/halkyon-learn`; another checkout needs them changed.
The units name `/home/deploy/halkyon`; another checkout needs them changed.
## Looking at the backups
@ -138,7 +138,7 @@ restic check --read-data-subset=5% # reads a twentieth of the data back
## Restoring
As `deploy`, in `~/halkyon-learn/deploy`, with the two variables from step 3 exported:
As `deploy`, in `~/halkyon/deploy`, with the two variables from step 3 exported:
```sh
export RESTIC_REPOSITORY_FILE=~/.config/restic/repository RESTIC_PASSWORD_FILE=~/.config/restic/password

View file

@ -18,6 +18,9 @@
# switched on in .env (COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml): each adds its
# service and mounts its site into Caddy's sidecars/ folder (sidecars/README.md).
# Never renamed, though the checkout's folder is ~deploy/halkyon since 2 Nov 2026: Docker
# names the volumes (the database, people's files, the releases' repositories) and the
# edge network after the project, and under a new name would start them empty.
name: halkyon-learn
# Logs go to the host's journal, whose size and age limits apply (journald.conf).

View file

@ -2,8 +2,8 @@
# Deploys learn.halkyon.eu from main. The only command deploy's keys may run: deploy's
# ~/.ssh/authorized_keys on the server pins both to this script, CI's from the server itself:
#
# restrict,from="127.0.0.1,::1",command="/home/deploy/halkyon-learn/deploy/redeploy.sh" ssh-ed25519 … (CI deploy trigger)
# restrict,command="/home/deploy/halkyon-learn/deploy/redeploy.sh" ssh-ed25519 … (manual deploys)
# restrict,from="127.0.0.1,::1",command="/home/deploy/halkyon/deploy/redeploy.sh" ssh-ed25519 … (CI deploy trigger)
# restrict,command="/home/deploy/halkyon/deploy/redeploy.sh" ssh-ed25519 … (manual deploys)
#
# CI passes the commit it has checked. This deploys it only
# - while it is still the tip of main (otherwise the CI run of the newer tip deploys),
@ -26,7 +26,7 @@ main() {
esac
[ ${#commit} -eq 40 ] || fail 'expects the full commit to deploy (40 hex digits)'
cd "$HOME/halkyon-learn"
cd "$HOME/halkyon"
# One deploy at a time.
exec 9>"$HOME/.redeploy.lock"
flock 9
@ -176,7 +176,7 @@ wait_until_answering() {
# fit the database the migrations have changed. Before the switch, the running
# containers were never touched.
back_out() {
cd "$HOME/halkyon-learn"
cd "$HOME/halkyon"
git reset -q --hard "$deployed"
echo "The checkout is back at $deployed." >&2
if [ "${switched:-}" = yes ]; then

View file

@ -48,11 +48,11 @@ Steps, in order; ✱ marks a step as root.
```sh
sudo groupadd --system --gid 61096 jellyfin
sudo useradd --system --uid 61096 --gid 61096 --no-create-home --shell /usr/sbin/nologin jellyfin
sudo mkdir -p ~deploy/halkyon-learn/deploy/jellyfin/config \
~deploy/halkyon-learn/deploy/jellyfin/cache ~deploy/halkyon-learn/deploy/jellyfin/media
sudo chown -R jellyfin:jellyfin ~deploy/halkyon-learn/deploy/jellyfin/config \
~deploy/halkyon-learn/deploy/jellyfin/cache
sudo chown -R deploy:deploy ~deploy/halkyon-learn/deploy/jellyfin/media
sudo mkdir -p ~deploy/halkyon/deploy/jellyfin/config \
~deploy/halkyon/deploy/jellyfin/cache ~deploy/halkyon/deploy/jellyfin/media
sudo chown -R jellyfin:jellyfin ~deploy/halkyon/deploy/jellyfin/config \
~deploy/halkyon/deploy/jellyfin/cache
sudo chown -R deploy:deploy ~deploy/halkyon/deploy/jellyfin/media
```
2. **DNS** (the operator, at the domain's DNS): the Jellyfin domain, such as
@ -74,11 +74,11 @@ Steps, in order; ✱ marks a step as root.
```sh
tar -C "<the folder's parent>" -cf - "<folder>" |
ssh <deploy's ssh alias> 'tar -C halkyon-learn/deploy/jellyfin/media/movies -xf -'
ssh <deploy's ssh alias> 'tar -C halkyon/deploy/jellyfin/media/movies -xf -'
```
5. **Start it** (the operator, on the instance's server, as `deploy`):
`cd ~/halkyon-learn/deploy && docker compose up -d`. Caddy and the server are
`cd ~/halkyon/deploy && docker compose up -d`. Caddy and the server are
recreated with it (the site, and the instance's suggestion); the instance is away
for seconds.
@ -149,7 +149,7 @@ home machine.
```sh
sudo apt-get install wireguard-tools
(umask 077 && wg genkey | sudo tee /etc/wireguard/wg-jellyfin.key | wg pubkey)
from=~deploy/halkyon-learn/deploy/home-jellyfin/server
from=~deploy/halkyon/deploy/home-jellyfin/server
sudo install -m 0600 "$from/wg-jellyfin.conf" /etc/wireguard/wg-jellyfin.conf
sudo install -m 0700 "$from/wg-jellyfin-rules.sh" /etc/wireguard/wg-jellyfin-rules.sh
sudo install -D -m 0644 "$from/wg-quick-after-docker.conf" \
@ -165,7 +165,7 @@ home machine.
can't reach the tunnel; then the tunnel.
```sh
sudo install -m 0644 ~deploy/halkyon-learn/deploy/runner/forgejo-runner.service \
sudo install -m 0644 ~deploy/halkyon/deploy/runner/forgejo-runner.service \
/etc/systemd/system/forgejo-runner.service
sudo systemctl daemon-reload
sudo systemctl restart forgejo-runner
@ -225,7 +225,7 @@ home machine.
JELLYFIN_DOMAIN=jellyfin.halkyon.eu
```
then `cd ~/halkyon-learn/deploy && docker compose up -d`. Caddy and the server are
then `cd ~/halkyon/deploy && docker compose up -d`. Caddy and the server are
recreated with it; the instance is away for seconds.
7. **Check from outside** (anyone, anywhere): the sidecar's step 7; and on the instance's
@ -278,15 +278,15 @@ Steps, in order; ✱ marks a step as root.
```sh
sudo groupadd --system --gid 61081 languagetool
sudo useradd --system --uid 61081 --gid 61081 --no-create-home --shell /usr/sbin/nologin languagetool
sudo mkdir -p ~deploy/halkyon-learn/deploy/languagetool/fasttext
sudo chown -R languagetool:languagetool ~deploy/halkyon-learn/deploy/languagetool
sudo mkdir -p ~deploy/halkyon/deploy/languagetool/fasttext
sudo chown -R languagetool:languagetool ~deploy/halkyon/deploy/languagetool
```
2. **Switch it on** (the operator, on the instance's server, as `deploy`): in
`deploy/.env`, `COMPOSE_FILE=compose.yaml:compose.languagetool.yaml`.
3. **Start it** (the operator, on the instance's server, as `deploy`):
`cd ~/halkyon-learn/deploy && docker compose up -d`. The server is recreated with it
`cd ~/halkyon/deploy && docker compose up -d`. The server is recreated with it
(its `LANGUAGETOOL_URL`); the instance is away for seconds. At its first start
LanguageTool fetches fastText's model of languages (`lid.176.bin`, 126 MB, from
Facebook's `dl.fbaipublicfiles.com`) into `deploy/languagetool/fasttext/`, which it
@ -323,7 +323,7 @@ Steps, in order; ✱ marks a step as root.
BRouter's segments take about 3.3 GB for Europe, and a refresh writes each one beside
the old before replacing it; Train's dumps go to `~/backups` with Learn's.
2. **Its database** (the operator, on the instance's server, as `deploy`, in
`~/halkyon-learn/deploy`): a role of Train's own, with the password that goes into
`~/halkyon/deploy`): a role of Train's own, with the password that goes into
`.env` as `TRAIN_POSTGRES_PASSWORD` (`openssl rand -hex 32`), and its database:
```sh
@ -351,8 +351,8 @@ Steps, in order; ✱ marks a step as root.
```sh
sudo groupadd --system --gid 61177 brouter
sudo useradd --system --uid 61177 --gid 61177 --no-create-home --shell /usr/sbin/nologin brouter
sudo mkdir -p ~deploy/halkyon-learn/deploy/brouter/segments
sudo chown -R brouter:brouter ~deploy/halkyon-learn/deploy/brouter/segments
sudo mkdir -p ~deploy/halkyon/deploy/brouter/segments
sudo chown -R brouter:brouter ~deploy/halkyon/deploy/brouter/segments
```
4. **DNS** (the operator, at the domain's DNS): Train's domain, `train.halkyon.eu`,
@ -406,8 +406,8 @@ Steps, in order; ✱ marks a step as root.
```sh
sudo groupadd --system --gid 61178 maptiles
sudo useradd --system --uid 61178 --gid 61178 --no-create-home --shell /usr/sbin/nologin maptiles
sudo mkdir -p ~deploy/halkyon-learn/deploy/maps/tiles
sudo chown -R maptiles:maptiles ~deploy/halkyon-learn/deploy/maps/tiles
sudo mkdir -p ~deploy/halkyon/deploy/maps/tiles
sudo chown -R maptiles:maptiles ~deploy/halkyon/deploy/maps/tiles
```
3. **Switch it on** (the operator, as `deploy`): in `deploy/.env`, `compose.maps.yaml`
@ -452,8 +452,8 @@ Steps, in order; ✱ marks a step as root.
```sh
sudo groupadd --system --gid 61179 photon
sudo useradd --system --uid 61179 --gid 61179 --no-create-home --shell /usr/sbin/nologin photon
sudo mkdir -p ~deploy/halkyon-learn/deploy/photon/data
sudo chown -R photon:photon ~deploy/halkyon-learn/deploy/photon/data
sudo mkdir -p ~deploy/halkyon/deploy/photon/data
sudo chown -R photon:photon ~deploy/halkyon/deploy/photon/data
```
3. **Switch it on** (the operator, as `deploy`): in `deploy/.env`, `compose.search.yaml`
@ -484,7 +484,7 @@ compose file, and `backup.sh` backs its database up beside Learn's.
Steps, in order:
1. **Its database** (the operator, on the instance's server, as `deploy`, in
`~/halkyon-learn/deploy`): a role of Todo's own, with the password that goes into
`~/halkyon/deploy`): a role of Todo's own, with the password that goes into
`.env` as `TODO_POSTGRES_PASSWORD` (`openssl rand -hex 32`), and its database, as
Train's (*Train*, step 2, says what each line does):