docs(adr): halkyon.li at home through ramiel, and the Instance console (ADR 0039, 0040)

ADR 0039: the family's instance halkyon.li on balthasar, with Learn, Todo,
Train, Meditate and Jellyfin, sign-up by approval through SimpleLogin alone,
published through ramiel by WireGuard with TLS passed through by name
(HAProxy), the CI runner moved there in a nightly window, deployed by hand.
Supersedes ADR 0032, whose server end goes with sahaquiel.

ADR 0040: one Instance row in every app's Settings opening a route with
accounts, sign-up, assistants, regions and state; the instance's settings in
a file outside the backups, the keys staying in assistants.env; regions from
CoMaps' polygons, which the sidecars build themselves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-11-08 18:56:58 +01:00
commit 7bf5f6194f
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
3 changed files with 365 additions and 0 deletions

View file

@ -3,6 +3,9 @@
- **Status:** decided 30 Oct 2026, by the developer (D1, D9 and D10 their own choices,
D2–D8 agreed as recommended). Built the same day (*As built*, below); not brought up
yet, which takes the developer's steps as root on the instance's server.
**Superseded** on 8 Nov 2026 by ADR 0039: its server end, learn.halkyon.eu's server,
is retired, and Jellyfin becomes halkyon.li's own sidecar on the machine that was its
home end. Its open question goes with it. The files are removed once ADR 0039 is built.
- **Builds on:** the Jellyfin sidecar (`deploy/compose.jellyfin.yaml`,
`deploy/sidecars/README.md`, decided 14 Oct 2026) and Halkyon Learn's suggested
server (`JellyfinSuggestion`, decided 13 Oct). Train's Jellyfin (ADR 0030, D47, D52,

View file

@ -0,0 +1,185 @@
# ADR 0039: halkyon.li, the family's instance at home, published through ramiel
- **Status:** decided 8 Nov 2026 by the developer: D1 to D6 and D17 to D19 are their
own choices, D7 to D16 the agent's recommendations, which they confirmed. Nothing is
built yet. halkyon.li's DNS zone at netcup is in place (D7, checked 8 Nov).
- **Builds on:** Halkyon Learn's ADR 0021 (a private instance whose operator approves
every account: "The developer may stop the public instance and run halkyon.li for
themselves and their close family"), ADR 0016 (the instance's register), ADR 0020
(SimpleLogin), ADR 0030, 0033 and 0038 (Train's, Todo's and Meditate's servers beside
Learn's).
- **Supersedes:** ADR 0032 (Jellyfin at home through a tunnel to learn.halkyon.eu's
server), whose server end is retired with sahaquiel; its open question goes with it.
- **Followed by:** the Instance console's ADR (to come), through which halkyon.li's
regions are added and removed (D5).
## Context
- **sahaquiel**, learn.halkyon.eu's server, is retired later in November 2026. It holds
learn.halkyon.eu (its server, PostgreSQL, the register), the Forgejo runner that
checks every push and deploys a green `main`, and the backups' timer. The developer
moves its data out themselves.
- **balthasar** is the developer's iMac at home (iMac17,1: 4 cores, 7.2 GB RAM). Its
root filesystem is 19 GB, 91 % full; `/home` is a 916 GB HDD with 150 GB free. It sits
behind a FRITZ!Box on Vodafone cable, with a dynamic IPv4 address (178.24.33.155 on
8 Nov) and IPv6. It already runs ADR 0032's home Jellyfin (media in
`/home/raisondetredev/Media`).
- **ramiel**, the halkyon.app server (45.129.181.128, `2a03:4000:47:9e6:…`): 1 vCPU,
957 MiB RAM, 8.8 GB disk free, Ubuntu 24.04. The system's Caddy (v2.11.4, Ubuntu's
package) holds 80 and 443 (HTTP/3 on UDP 443 too) for `code.halkyon.app` (Forgejo on
`127.0.0.1:3000`), `halkyon.app` (the F-Droid repository) and `learn.halkyon.app`.
ufw and fail2ban are active. It sent about 67 GB in its last 20 days.
- **halkyon.li**: its nameservers are Cloudflare's on 8 Nov, every record proxied, with
a wildcard `*.halkyon.li` and an MX `mail.halkyon.li`. Nothing answers. The MX can't
have been receiving mail: its name points at Cloudflare's proxy, which carries no
SMTP, and there is no SPF record. There is no DNSSEC either: no DS record at the
registry. The developer moves the zone to netcup's DNS (D7), where ramiel's
addresses are too.
- **melchior**, the developer's laptop, is where development and the pre-push check
run from now on.
- **The forge**: `project-halkyon/halkyon` is private since 8 Nov (the developer's
choice: the code is hidden from the public for now; nobody but the developer has
contributed). `appflowy-editor`, `pure-zstd` and `serverpod-offline-sync` stay
public, as every build fetches them anonymously as git dependencies.
## Decisions
### The developer's (8 Nov 2026)
- **D1 halkyon.li is the family's instance, run on balthasar**, with Learn, Todo, Train
and Meditate from the start, and Jellyfin. It is the instance ADR 0021 foresaw: sign-up
by approval (`SIGN_UP=approval`). learn.halkyon.eu ends with sahaquiel.
- **D2 Sign-up and sign-in through SimpleLogin alone** (`SIGN_IN_WAYS=simplelogin`).
No password, no TOTP and no email codes, so no SMTP service. halkyon.li gets a
SimpleLogin app of its own (D9).
- **D3 Published through ramiel by WireGuard, with TLS passed through by name** (over TLS
ended on ramiel, and over a port forward on the FRITZ!Box, which would show the home
address). On ramiel's 443,
HAProxy (Ubuntu's package, kept current by `unattended-upgrades`) reads the name a
connection asks for (SNI) and sends halkyon.li's, unopened, through the tunnel to
balthasar; every other name goes to ramiel's own Caddy, moved to the loopback (over
caddy-l4, which needs a Caddy that apt no longer updates). ramiel sees no request of
halkyon.li's, holds none of its certificates, and spends its one CPU copying bytes
rather than encrypting streams. halkyon.li thus has no HTTP/3 (QUIC can't be routed by
name this simply); clients use HTTP/2.
- **D4 Jellyfin is `jellyfin.halkyon.li` alone**: `jellyfin.halkyon.eu` ends with
sahaquiel.
- **D5 Regions are added and removed in the Instance console**, for Train's map,
routes and search and Todo's search: CoMaps' border polygons (`data/borders/*.poly`,
below countries: *Germany_Free State of Bavaria_Upper Bavaria_South*). The console's
ADR decides how, under two rules already agreed: the settings it keeps are not in the
backups (what belongs to accounts stays in the database and so in them), and no server
gets Docker's socket. Until it ships, halkyon.li runs without those sidecars.
- **D6 The CI runner moves to balthasar and runs at night**, 02:00 to 05:00, started and
stopped by a systemd timer; Forgejo queues the jobs pushed during the day. In the
window the heavy sidecars stop (Photon, BRouter, the tiles, Jellyfin, LanguageTool,
the dictionaries), while Caddy, PostgreSQL and the four servers keep serving. CI only
checks: the deploy job goes with sahaquiel, as halkyon.li deploys by hand (D13).
### Recommended by the agent, confirmed by the developer
- **D7 The names**: `learn.halkyon.li` (Learn's server, which holds the register),
`todo.halkyon.li`, `train.halkyon.li`, `meditate.halkyon.li`, `jellyfin.halkyon.li`,
and `halkyon.li` itself for people's handles and profile pages (`ACCOUNT_DOMAIN`), as
halkyon.eu is for learn.halkyon.eu. Each is an `A` and `AAAA` record at ramiel, in
netcup's DNS zone for halkyon.li (the developer's choice, 8 Nov). The zone is made at
netcup first, then the domain's nameservers are switched to netcup's. With no DS
record at the registry, the switch can't break DNSSEC. Cloudflare's records end with
it, its proxy too. The new zone has no wildcard, so no name answers that isn't meant
to, and no MX, as the old one can't have received mail. Mail for halkyon.li is
outside this record.
- **D8 The tunnel**: `wg-quick@wg-halkyon` on both hosts, `10.61.97.1` ramiel and
`10.61.97.2` balthasar, a /30, IPv4 alone (apart from ADR 0032's `10.61.96.0/30`,
Docker's pools and Multipass' bridges). ramiel listens on UDP 51820, opened in ufw.
balthasar dials out with `PersistentKeepalive = 25`, so the FRITZ!Box opens nothing
and a new home address needs no change. Each side's `AllowedIPs` is the other's
address alone. The keys stay in `/etc/wireguard/` (root, 0600) and in Proton Pass.
- **D9 What the tunnel may carry**, by rules `PostUp` sets and `PreDown` removes, in
chains of their own as ADR 0032's:
- on ramiel: HAProxy's connections to `10.61.97.2:443`, nothing else; nothing new
from the tunnel into ramiel (so a taken-over balthasar can't reach ramiel's sshd or
Forgejo);
- on balthasar: new connections from `10.61.97.1` to Caddy's 443, and to sshd's 22
for the jump from away (D17), and nothing else; nothing new from the tunnel into the
rest of the host, the home network or the other containers.
- **D10 Visitors' addresses through the PROXY protocol**: HAProxy sends each
connection's real source (PROXY v2) to balthasar's Caddy, which accepts that header
from `10.61.97.1` alone (`proxy_protocol` listener wrapper), so the servers' per-network
limits keep counting visitors, not ramiel. ramiel's own Caddy gets the header too,
from the loopback, so Forgejo's and the F-Droid repository's logs keep their visitors.
- **D11 Certificates at home**: balthasar's Caddy gets halkyon.li's names' certificates
from Let's Encrypt by the TLS-ALPN challenge, which passes through HAProxy unopened.
Port 80 stays with ramiel's Caddy, which sends `http://*.halkyon.li` to `https://`;
the home Caddy disables the HTTP challenge. It also switches HTTP/3 off
(`protocols h1 h2`), so it advertises nothing that ramiel's UDP 443 would answer for
the wrong names.
- **D12 balthasar becomes a server**:
- the desktop no longer starts (`systemctl set-default multi-user.target`, undone by
`graphical.target`), which frees about 1 GB; Multipass is disabled if unused;
- Docker's data moves to the HDD (`data-root` in `/etc/docker/daemon.json`), as the
root filesystem has room for no image;
- a `deploy` user holds the checkout (`~deploy/halkyon`) and runs the stack, as on
sahaquiel; the sidecars' user IDs are those of the sidecar README;
- Caddy publishes 443 on the tunnel's address alone (`10.61.97.2:443`), the tunnel
started before Docker (`Before=docker.service`), so nothing of the instance answers
on the home network.
- **D13 Deploys by hand with `redeploy.sh`**, run as `deploy` on balthasar with the
commit, as on sahaquiel: still only the tip of `main`, a fast-forward, every commit
signed by a key in `deploy/allowed-signers`, the databases dumped first, migrations
in a run of their own. A deploy waits for that night's green run of the commit. The
script loops over the apps .env switches on (`train todo`, now `meditate` too) and
its comments stop naming learn.halkyon.eu.
- **D14 Meditate's deploy step (ADR 0038, step 7) is built for it**:
`deploy/compose.meditate.yaml` and `sidecars/meditate.caddy` as Todo's are, the
`halkyon_meditate` database under a role of its own, a member of the register,
`redeploy.sh` and `backup.sh` taking it in.
- **D15 Jellyfin as the instance's sidecar** (`compose.jellyfin.yaml`), on `edge` beside
Caddy, the library mounted read-only from `/home/raisondetredev/Media`; its existing
configuration (`deploy/home-jellyfin/jellyfin/config`) moves with it, so its accounts
and library stay. `deploy/home-jellyfin/` and `compose.jellyfin-home.yaml` go, with
ADR 0032's open question.
- **D16 The runner on balthasar**: `deploy/runner/`'s unit as it is (its own user, no
`/home`, no Docker socket, Docker's networks denied), with these changes:
- its folder on the HDD outside `/home`;
- the home network denied by interface rather than by address, so Vodafone's
changing IPv6 prefix is covered too;
- the tunnel's `10.61.97.0/30` denied;
- the timer's window (D6).
CI's `runs-on` and the Sandbox step's checks follow it.
### Decided after the proposal (8 Nov 2026)
- **D17 melchior reaches balthasar at home and from away** (the developer's choice). At
home it connects over the home network. From away it jumps through ramiel
(`ssh -J ramiel` to `10.61.97.2`), so D9's rules also let `10.61.97.1` reach
balthasar's sshd on TCP 22, and nothing else beyond Caddy's 443. balthasar's sshd
takes keys alone (`PasswordAuthentication no`, `KbdInteractiveAuthentication no`) and
lets in only the developer's user and `deploy`. A taken-over ramiel thus meets one
more port, which answers to no password.
- **D18 Backups go to the same Hetzner Storage Box** (the developer's choice), in a
restic repository of their own with a password of its own. The repository is reached
through a sub-account of the box, confined to its own folder. A taken-over balthasar
then reaches halkyon.li's backups alone, and the box's automatic snapshots, which
only Hetzner's Console removes, still keep those safe. `backup.sh` runs hourly as it
does on sahaquiel, inside the nightly window too, as the servers and PostgreSQL keep
running (D6).
- **D19 Photon builds an index of our own from the chosen polygons**, as polygons below
countries need (Photon's published dumps stop at countries): a Geofabrik extract cut
by the polygons, imported by a short-lived Nominatim. The developer raised no
objection. The console's ADR decides how it runs.
## Consequences
- People see ramiel's address alone; nothing names the home address, which only ramiel
knows (`wg show`). With balthasar off or offline, ramiel's HAProxy finds nothing behind
the tunnel and halkyon.li is away; the apps sync offline-first and catch up.
- Every byte of halkyon.li crosses the home line's upload and ramiel twice. Jellyfin's
streams are bounded by the upload; ramiel's traffic plan is to be checked.
- ramiel gains HAProxy, a listening UDP port, a kernel interface and firewall rules; its
Caddy moves off 443 to the loopback, its sites unchanged.
- The source links (About in each app, the snap's `source-code`, the F-Droid listings,
`halkyon.app`'s and `learn.halkyon.app`'s redirects) lead to Forgejo's sign-in page
while `halkyon` is private. halkyon.li leaves `SOURCE_URL` as it is.
- Learn's `AGENTS.md`, *CI and deploys*, the root's, and `deploy/backups.md` are
rewritten for balthasar once built.

View file

@ -0,0 +1,177 @@
# ADR 0040: The Instance console: accounts, sign-up, assistants, regions and state in the apps
- **Status:** decided 8 Nov 2026 by the developer: D1 to D4 and D17 to D18 are their
own choices, D5 to D16 the agent's recommendations, which they confirmed. Nothing is
built yet.
- **Builds on:** Halkyon Learn's ADR 0021 (operators, approval, the assistants and their
grants: `InstanceOperatorEndpoint`, `AssistantOperatorEndpoint`, the operator's card,
badge and screen in `halkyon_app`), ADR 0016 (the register and `INSTANCE_SECRET`), ADR
0030, D44 and D50 (BRouter, the tiles and Photon for Train), ADR 0033, D32 (Todo's
search on the same Photon), ADR 0039 (halkyon.li, whose regions come from here, D5,
D19).
## Context
An instance's operator already approves, refuses and suspends accounts, names operators
and grants assistants with budgets, in every app (ADR 0021). Everything else about the
instance is in files on its host, changed over SSH, each change followed by a restart:
- who may sign up (`SIGN_UP` in `deploy/.env`);
- which assistants and models are offered, with their keys (`deploy/assistants.env`);
- the regions of Train's map, routes and search: `MAP_REGION` and `BROUTER_REGION` (boxes
in degrees) and `PHOTON_REGION` (a country of GraphHopper's dumps), Europe by default.
Nothing in the apps shows whether the sidecars run or when the last backup was made.
halkyon.li (ADR 0039) runs on a home machine with 7.2 GB of RAM and 150 GB of free disk,
where Europe's tiles (45 GB, twice while cut) and Europe's Photon (8 GB of memory) don't
fit. Its family needs a few areas, smaller than a country: *Upper Bavaria, South*, for
instance. CoMaps (Organic Maps' fork) cuts the world into 1,159 such areas for its own
downloads, each a polygon in `data/borders/*.poly`, named by its place in the hierarchy
(`Germany_Free State of Bavaria_Upper Bavaria_South`, 500 KB). The repository is under
Apache 2.0; the borders come from OpenStreetMap's boundaries, so they carry
"© OpenStreetMap contributors", as the tiles do.
How each sidecar can be cut:
- **Tiles**: `pmtiles extract` takes a GeoJSON polygon (`--region`) instead of a box.
- **BRouter**: its segments are 5° by 5° and can't be cut. A region takes every segment
its polygon touches.
- **Photon**: GraphHopper's dumps stop at countries. Anything smaller is an index of our
own: an extract of OpenStreetMap cut to the polygon (`osmium extract -p`), imported
into a Nominatim database, which Photon then imports from (ADR 0039, D19).
## Decisions
### The developer's (8 Nov 2026)
- **D1 One row in Settings, a route of its own**: operators alone see a single row,
*Instance*, in every app's Settings, with the count of accounts waiting as its subtitle
and badge (replacing today's *This instance* card). It opens the *Instance* route, with
*Accounts*, *Sign-up*, *Assistants*, *Regions* and *State*. On a phone each opens a
screen of its own; on a wide window the route shows the list and the chosen part side by
side. The badge on the Settings cog stays. Not in Settings itself (too much there), not a
web console, not Django: the module's endpoints and `halkyon_app`'s screens, as for
everything an operator does today.
- **D2 The instance's settings are not backed up; what belongs to accounts is**: sign-up,
the assistants offered and the regions are kept in a file of their own (D5), outside
the database and every backup. Approvals, operators, grants, budgets and figures stay in
the database and so in the backups, since a restore without them would leave everyone
waiting.
- **D3 The assistants' keys stay in `assistants.env`**: the console shows whether each
assistant's key is set, never the key.
- **D4 Regions are added and removed in the console**, from CoMaps' polygons. halkyon.li
starts with none, and Train's and Todo's map, routes and search with them.
### Recommended by the agent, confirmed by the developer
- **D5 The settings file**: `settings.json` in a volume of its own, `instance_settings`,
mounted into the register's server alone (Learn's, ADR 0016). It is written whole into
a new file and moved over the old, so it is never read half written. `backup.sh` leaves
the volume out. It holds no secret. Each setting comes from the file once the console
has set it, otherwise from `.env`, as today. The console shows which applies (*Set here*
or *From the server's configuration*) and can go back to `.env`'s. A lost file thus
returns the instance to its `.env`, and the regions to none.
- **D6 The members ask the register**: Todo's, Train's and Meditate's servers read the
settings that concern them (sign-up, which every server enforces at sign-in, and the
assistants, which Learn's server alone forwards today) from the register over the
instance's own network, with `INSTANCE_SECRET`, as they ask it about accounts now. They
cache an answer for a minute. A change thus takes at most a minute, with no restart.
- **D7 Sign-up**: *Open*, *By approval* or *Closed*, a choice on the screen, applied
within a minute. Closing it leaves waiting accounts waiting; they are approved or
refused under *Accounts* as before.
- **D8 Assistants**: each assistant's name, provider (protocol and host), model, efforts
allowed, order and the ceilings of a request are kept in the file; its key stays
`ASSISTANT_<ID>_KEY` in `assistants.env`. Adding an assistant in the console makes it
*Key missing* until its key is in the file and the server restarted: the one restart
left, as keys never pass through the app. Renaming an ID stays impossible (its grants
hang on it); the console makes a new one instead. The grants and budgets move under
*Assistants* as a second part, *Granted*.
- **D9 Regions are CoMaps' polygons, named and fetched at a pinned commit**: the
register's image carries the list of the 1,159 names (some 60 KB) of one CoMaps commit
(`55846720`, 8 Nov 2026; moved by hand). The console shows them as a tree to search
(*Germany › Free State of Bavaria › Upper Bavaria › South*), from the names' parts. An
added region's polygon is downloaded from that commit on Codeberg once, converted to
GeoJSON and kept beside `settings.json`. Nothing of CoMaps' is in the repository but
the list.
- **D10 One list of regions for all three**: a region added is covered by the tiles,
the routes and the search together, as a family needs each where it goes; the console
shows each service's state per region (confirmed by D17).
- **D11 The sidecars bring themselves to what the register wants**: no server ever gets
Docker's socket. Each sidecar's refresher (`map-tiles-refresh`, `brouter-segments`,
and Photon's builder, D12) asks the register every 10 minutes for the wanted regions,
their union as one GeoJSON MultiPolygon and its hash, with `INSTANCE_SECRET`, over a
network it shares with the register's server alone. When the hash differs from what it
last built, or a month has passed, it builds anew beside what it serves and moves it
over, as today. It reports its state back: *building* (with what), *ready* (with size
and date), *failed* (with the reason in a line), and the free space of its volume. The
`.env` variables stay for an instance without the console's regions: with none set
there, nothing is built.
- tiles: `pmtiles extract --region=<the union>`;
- routes: every segment whose 5° square the union's polygons touch (`E10_N45.rd5` for
Upper Bavaria);
- search: D12.
- **D12 Photon's builder**: one container, `photon-build`, replaces `photon-data`. It
carries osmium, a PostgreSQL with Nominatim (`mediagis/nominatim`, pinned by digest)
and Photon's jar. When the regions change or a month has passed:
1. it picks the smallest of Geofabrik's extracts that covers each polygon, from
Geofabrik's index (`index-v1.json`, each extract's own polygon);
2. it downloads them and cuts them to the union;
3. it imports the cut into its Nominatim, then that into a new Photon database;
4. it stops its PostgreSQL, deletes Nominatim's database and marks the new Photon
database ready for `serve.sh`, as `data.sh` does now.
It is held to 3 GB of memory. While nothing is to be built, it holds nothing but its
script. Photon's server then needs about 1 GB of heap for a few regions
(`PHOTON_HEAP`), rather than 6.
- **D13 Builds wait for the day**: refreshers and the builder are among the sidecars
ADR 0039, D6, stops during the CI window from 02:00 to 05:00. A build already running
is interrupted there and starts again afterwards.
- **D14 Room before a region**: the console shows each volume's free space as reported.
It warns, without refusing, when it is less than twice what the services take now,
since a new cut is built beside the old.
- **D15 State**: *State* shows what the instance can know without Docker:
- each sidecar's last report (D11) and whether it is within the nightly window;
- the last backup, from a line `backup.sh` writes (`state/backup.json` under
`deploy/`, mounted read-only into the register's server: when, how large, whether it
succeeded);
- the version each app's server runs.
A sidecar that hasn't reported for two of its rounds shows as *Silent*.
- **D16 Access**: every new endpoint of the console is the operator's alone, with tests
first asserting that a person who isn't an operator, someone signed out and a member
service are refused. The sidecars' two endpoints (the wanted regions, the reports) take
`INSTANCE_SECRET` alone, with tests asserting that a person's session, operator or not,
and a wrong or missing secret are refused. Reports are kept in memory only (the
register's server forgets them when it restarts, and the next round reports again), so
nothing of them reaches the database.
### Order of building
1. **Regions** (D9 to D14), which halkyon.li waits for.
2. **Sign-up** (D5 to D7), with the settings file and the members asking for it.
3. **Assistants** (D8).
4. **State** (D15).
5. The row in Settings and the route (D1), from step 1 on: *Accounts* is today's
operator's screen.
### Decided after the proposal (8 Nov 2026)
- **D17 One list of regions** (the developer's choice), as D10 has it, over a choice per
service, which would have let a region have routes without its search.
- **D18 A region is built whenever it is added or removed** (the developer's choice),
at any hour outside the CI window (D13), with no hours of its own.
## Consequences
- The settings' file joins `.env` as where an instance's configuration lives; after a
restore the operator sets again what was set in the console. Approvals, operators,
grants and budgets come back with the database.
- A region costs disk and memory on its sidecars, which the console shows but does not
refuse.
- The register's server gains an instance network shared with the refreshers, which have
a way out to the internet (`downloads`). Only its two secret-bound endpoints are of use
to them, as with the members today.
- Photon's builder brings Nominatim in, as an import step that leaves nothing running.
- `sidecars/README.md` and `.env.example` describe the console as the way to set
regions, sign-up and assistants, `.env` as their defaults.