halkyon/docs/adr/0039-halkyon-li-at-home.md

ADR 0039: halkyon.li, the family's instance at home, published through ramiel

• Status: decided 8 Nov 2026 by the developer: D1 to D6 and D17 to D20 are their own choices, D7 to D16 the agent's recommendations, which they confirmed. Nothing is built yet. halkyon.li's DNS zone at netcup is in place (D7, checked 8 Nov).
• Builds on: Halkyon Learn's ADR 0021 (a private instance whose operator approves every account: "The developer may stop the public instance and run halkyon.li for themselves and their close family"), ADR 0016 (the instance's register), ADR 0020 (SimpleLogin), ADR 0030, 0033 and 0038 (Train's, Todo's and Meditate's servers beside Learn's).
• Supersedes: ADR 0032 (Jellyfin at home through a tunnel to learn.halkyon.eu's server), whose server end is retired with sahaquiel; its open question goes with it.
• Followed by: the Instance console's ADR (to come), through which halkyon.li's regions are added and removed (D5).

Context

• sahaquiel, learn.halkyon.eu's server, is retired later in November 2026. It holds learn.halkyon.eu (its server, PostgreSQL, the register), the Forgejo runner that checks every push and deploys a green main, and the backups' timer. The developer moves its data out themselves.
• balthasar is the developer's iMac at home (iMac17,1: 4 cores, 7.2 GB RAM). Its root filesystem is 19 GB, 91 % full; /home is a 916 GB HDD with 150 GB free. It sits behind a FRITZ!Box on Vodafone cable, with a dynamic IPv4 address (178.24.33.155 on 8 Nov) and IPv6. It already runs ADR 0032's home Jellyfin (media in /home/raisondetredev/Media).
• ramiel, the halkyon.app server (45.129.181.128, 2a03:4000:47:9e6:…): 1 vCPU, 957 MiB RAM, 8.8 GB disk free, Ubuntu 24.04. The system's Caddy (v2.11.4, Ubuntu's package) holds 80 and 443 (HTTP/3 on UDP 443 too) for code.halkyon.app (Forgejo on 127.0.0.1:3000), halkyon.app (the F-Droid repository) and learn.halkyon.app. ufw and fail2ban are active. It sent about 67 GB in its last 20 days.
• halkyon.li: its nameservers are Cloudflare's on 8 Nov, every record proxied, with a wildcard *.halkyon.li and an MX mail.halkyon.li. Nothing answers. The MX can't have been receiving mail: its name points at Cloudflare's proxy, which carries no SMTP, and there is no SPF record. There is no DNSSEC either: no DS record at the registry. The developer moves the zone to netcup's DNS (D7), where ramiel's addresses are too.
• melchior, the developer's laptop, is where development and the pre-push check run from now on.
• The forge: project-halkyon/halkyon is private since 8 Nov (the developer's choice: the code is hidden from the public for now; nobody but the developer has contributed). appflowy-editor, pure-zstd and serverpod-offline-sync stay public, as every build fetches them anonymously as git dependencies.

Decisions

The developer's (8 Nov 2026)

• D1 halkyon.li is the family's instance, run on balthasar, with Learn, Todo, Train and Meditate from the start, and Jellyfin. It is the instance ADR 0021 foresaw: sign-up by approval (SIGN_UP=approval). learn.halkyon.eu ends with sahaquiel.
• D2 Sign-up and sign-in through SimpleLogin alone (SIGN_IN_WAYS=simplelogin). No password, no TOTP and no email codes, so no SMTP service. halkyon.li gets a SimpleLogin app of its own (D9).
• D3 Published through ramiel by WireGuard, with TLS passed through by name (over TLS ended on ramiel, and over a port forward on the FRITZ!Box, which would show the home address). On ramiel's 443, HAProxy (Ubuntu's package, kept current by unattended-upgrades) reads the name a connection asks for (SNI) and sends halkyon.li's, unopened, through the tunnel to balthasar; every other name goes to ramiel's own Caddy, moved to the loopback (over caddy-l4, which needs a Caddy that apt no longer updates). ramiel sees no request of halkyon.li's, holds none of its certificates, and spends its one CPU copying bytes rather than encrypting streams. halkyon.li thus has no HTTP/3 (QUIC can't be routed by name this simply); clients use HTTP/2.
• D4 Jellyfin is jellyfin.halkyon.li alone: jellyfin.halkyon.eu ends with sahaquiel.
• D5 Regions are added and removed in the Instance console, for Train's map, routes and search and Todo's search: CoMaps' border polygons (data/borders/*.poly, below countries: Germany_Free State of Bavaria_Upper Bavaria_South). The console's ADR decides how, under two rules already agreed: the settings it keeps are not in the backups (what belongs to accounts stays in the database and so in them), and no server gets Docker's socket. Until it ships, halkyon.li runs without those sidecars.
• D6 The CI runner moves to balthasar and runs at night, 02:00 to 05:00, started and stopped by a systemd timer; Forgejo queues the jobs pushed during the day. In the window the heavy sidecars stop (Photon, BRouter, the tiles, Jellyfin, LanguageTool, the dictionaries), while Caddy, PostgreSQL and the four servers keep serving. CI only checks: the deploy job goes with sahaquiel, as halkyon.li deploys by hand (D13).

Recommended by the agent, confirmed by the developer

• D7 The names: learn.halkyon.li (Learn's server, which holds the register), todo.halkyon.li, train.halkyon.li, meditate.halkyon.li, jellyfin.halkyon.li, and halkyon.li itself for people's handles and profile pages (ACCOUNT_DOMAIN), as halkyon.eu is for learn.halkyon.eu. Each is an A and AAAA record at ramiel, in netcup's DNS zone for halkyon.li (the developer's choice, 8 Nov). The zone is made at netcup first, then the domain's nameservers are switched to netcup's. With no DS record at the registry, the switch can't break DNSSEC. Cloudflare's records end with it, its proxy too. The new zone has no wildcard, so no name answers that isn't meant to, and no MX, as the old one can't have received mail. Mail for halkyon.li is outside this record.
• D8 The tunnel: wg-quick@wg-halkyon on both hosts, 10.61.97.1 ramiel and 10.61.97.2 balthasar, a /30, IPv4 alone (apart from ADR 0032's 10.61.96.0/30, Docker's pools and Multipass' bridges). ramiel listens on UDP 51820, opened in ufw. balthasar dials out with PersistentKeepalive = 25, so the FRITZ!Box opens nothing and a new home address needs no change. Each side's AllowedIPs is the other's address alone. The keys stay in /etc/wireguard/ (root, 0600) and in Proton Pass.
• D9 What the tunnel may carry, in chains of their own as ADR 0032's, living as long as the tunnel: on ramiel set by wg-quick's PostUp and removed by its PreDown; at home loaded by systemd before the interface exists and deleted after it is gone (ExecStartPre, ExecStopPost), as Ubuntu 26.04's AppArmor profile for wg-quick lets its nft read nothing in /etc/wireguard (found 8 Nov 2026):
  • on ramiel: HAProxy's connections to 10.61.97.2:443, nothing else; nothing new from the tunnel into ramiel (so a taken-over balthasar can't reach ramiel's sshd or Forgejo);
  • on balthasar: new connections from 10.61.97.1 to Caddy's 443, and to sshd's 22 for the jump from away (D17), and nothing else; nothing new from the tunnel into the rest of the host, the home network or the other containers.
• D10 Visitors' addresses through the PROXY protocol: HAProxy sends each connection's real source (PROXY v2) to balthasar's Caddy, which accepts that header from 10.61.97.1 alone (proxy_protocol listener wrapper), so the servers' per-network limits keep counting visitors, not ramiel. ramiel's own Caddy gets the header too, from the loopback, so Forgejo's and the F-Droid repository's logs keep their visitors.
• D11 Certificates at home: balthasar's Caddy gets halkyon.li's names' certificates from Let's Encrypt by the TLS-ALPN challenge, which passes through HAProxy unopened. Port 80 stays with ramiel's Caddy, which sends http://*.halkyon.li to https://; the home Caddy disables the HTTP challenge. It also switches HTTP/3 off (protocols h1 h2), so it advertises nothing that ramiel's UDP 443 would answer for the wrong names.
• D12 balthasar becomes a server:
  • the desktop no longer starts (systemctl set-default multi-user.target, undone by graphical.target), which frees about 1 GB; Multipass is disabled if unused;
  • Docker's data moves to the HDD, as the root filesystem has room for no image: /var/lib/docker and /var/lib/containerd (where Docker 29 keeps images) as bind mounts from /data, as /home and snapd's folder already are on this machine, rather than Docker's data-root, which would leave containerd's store behind;
  • a deploy user holds the checkout (~deploy/halkyon) and runs the stack, as on sahaquiel; the sidecars' user IDs are those of the sidecar README;
  • Caddy publishes 443 on the tunnel's address alone (10.61.97.2:443), the tunnel started before Docker (Before=docker.service), so nothing of the instance answers on the home network.
• D13 Deploys by hand with redeploy.sh, run as deploy on balthasar with the commit, as on sahaquiel: still only the tip of main, a fast-forward, every commit signed by a key in deploy/allowed-signers, the databases dumped first, migrations in a run of their own. A deploy waits for that night's green run of the commit. The script loops over the apps .env switches on (train todo, now meditate too) and its comments stop naming learn.halkyon.eu.
• D14 Meditate's deploy step (ADR 0038, step 7) is built for it: deploy/compose.meditate.yaml and sidecars/meditate.caddy as Todo's are, the halkyon_meditate database under a role of its own, a member of the register, redeploy.sh and backup.sh taking it in.
• D15 Jellyfin as the instance's sidecar (compose.jellyfin.yaml), on edge beside Caddy, with its existing configuration (deploy/home-jellyfin/jellyfin/config), so its accounts and libraries stay. Its library is mounted read-only from the host (JELLYFIN_MEDIA). Found while building: it can't stay in /home/raisondetredev/Media, as the developer's home is closed to other users and Jellyfin runs as its own (61096). So it moves to /data/media (a rename on one filesystem, /home being a bind mount of /data/home), readable by all, over opening the home to every user. ADR 0032's network settings (LAN 172.29.96.1/32, proxy 10.61.96.1) are set to the sidecar's before it starts, as they would count the admin's requests as remote and lock them out. deploy/home-jellyfin/ and compose.jellyfin-home.yaml go, with ADR 0032's open question.
• D16 The runner on balthasar: deploy/runner/'s unit as it is (its own user, no /home, no Docker socket, Docker's networks denied), with these changes:
  • its folder on the HDD, /data/forgejo-runner, seen as /var/lib/forgejo-runner; the rest of /data, where /home really lies, hidden from it;
  • no IPv6 at all, and no private IPv4 network (10.0.0.0/8, which holds the tunnel, among them). Corrected while building: the proposal had the home network denied by interface, but the home network and the internet come through the same one, and the home network's IPv6 addresses are public ones under a prefix Vodafone changes. Every host CI fetches from answers over IPv4;
  • the timer's window (D6), and memory to fit beside the instance: 5 GB, swap beyond;
  • CI's runs-on (balthasar) and the Sandbox step's checks following it.

Decided after the proposal (8 Nov 2026)

• D17 melchior reaches balthasar at home and from away (the developer's choice). At home it connects over the home network. From away it jumps through ramiel (ssh -J ramiel to 10.61.97.2), so D9's rules also let 10.61.97.1 reach balthasar's sshd on TCP 22, and nothing else beyond Caddy's 443. balthasar's sshd takes keys alone (PasswordAuthentication no, KbdInteractiveAuthentication no) and lets in only the developer's user and deploy. A taken-over ramiel thus meets one more port, which answers to no password.

• D18 Backups go to the same Hetzner Storage Box (the developer's choice), in a restic repository of their own with a password of its own. The repository is reached through a sub-account of the box, confined to its own folder. A taken-over balthasar then reaches halkyon.li's backups alone, and the box's automatic snapshots, which only Hetzner's Console removes, still keep those safe. backup.sh runs hourly as it does on sahaquiel, inside the nightly window too, as the servers and PostgreSQL keep running (D6).

• D19 Photon builds an index of our own from the chosen polygons, as polygons below countries need (Photon's published dumps stop at countries): a Geofabrik extract cut by the polygons, imported by a short-lived Nominatim. The developer raised no objection. The console's ADR decides how it runs.

• D20 balthasar's root disk is unlocked over SSH at boot (the developer's choice, over Tang on ramiel, which would unlock a stolen machine that is still online, and over typing at the iMac's keyboard alone). Found while writing the host's steps: the root disk is LUKS-encrypted with a passphrase (/etc/crypttab: none), and the HDD unlocks from a key file on it, so every boot waits for the passphrase. dropbear-initramfs waits on the home network at boot, on port 2222 (so its host key never clashes with sshd's in a known_hosts). It takes melchior's key alone, which may run nothing but cryptroot-unlock. The USB network adapter's driver (cdc_ncm) is added to the initramfs. From away, a reboot waits for someone at home, as the tunnel isn't up that early. The keyboard still works as before. Ubuntu 26.04 builds its boot images with dracut, for which Ubuntu packages no SSH at boot; so initramfs-tools replaces it, with cryptsetup-initramfs and Ubuntu's dropbear-initramfs (the developer's choice of 8 Nov 2026, over the unpackaged dracut-sshd and over no unlocking by SSH). dracut's last image stays in GRUB's menu, shown for 5 seconds, as the way back until initramfs-tools' images have booted, so a failed image costs a choice at the iMac rather than a USB stick.

• D21 The first start (the developer's choices of 8 Nov 2026; the steps: deploy/host/first-start.md). .env is written on balthasar by deploy/host/halkyon-li-env.sh, every secret made there, never over an existing file, and copied whole into Proton Pass (over rendering it from Proton Pass on melchior at every change). Everything runs from the start: Jellyfin, LanguageTool, BRouter, Train's map and place search beside the four servers. Until the console cuts regions (ADR 0040), the map and routes cover southern Bavaria (the map 10,47.2,13.9,49, BRouter 10,45,15,50) and the search Germany, the smallest Photon's databases come in, with a 2 GB heap in 3 GB, as Europe's (45 GB of tiles and twice that while refreshing, 32.6 GB for Photon with 6 to 8 GB of memory) fits neither the HDD beside the 704 GB library nor the 7.2 GB of memory. The app says Germany for the data and the backups (DE). Meditate starts without a library.

Consequences

• People see ramiel's address alone; nothing names the home address, which only ramiel knows (wg show). With balthasar off or offline, ramiel's HAProxy finds nothing behind the tunnel and halkyon.li is away; the apps sync offline-first and catch up.
• Every byte of halkyon.li crosses the home line's upload and ramiel twice. Jellyfin's streams are bounded by the upload; ramiel's traffic plan is to be checked.
• ramiel gains HAProxy, a listening UDP port, a kernel interface and firewall rules; its Caddy moves off 443 to the loopback, its sites unchanged.
• The source links (About in each app, the snap's source-code, the F-Droid listings, halkyon.app's and learn.halkyon.app's redirects) lead to Forgejo's sign-in page while halkyon is private. halkyon.li leaves SOURCE_URL as it is.
• Learn's AGENTS.md, CI and deploys, the root's, and deploy/backups.md are rewritten for balthasar once built.