feat(deploy): halkyon.li's forward through ramiel and its host at home (ADR 0039)

ramiel's HAProxy passes halkyon.li's TLS connections, unopened, through a
WireGuard tunnel to the instance at home, with the visitor's address in a
PROXY header, and everything else to ramiel's own Caddy, moved to the
loopback. Each end's nftables table lets the tunnel carry HAProxy's 443 and
the SSH jump's 22 alone, and nothing from home to ramiel.

The host at home: sshd with keys alone, the root disk unlocked over SSH at
boot (dropbear-initramfs, D20), Docker's and containerd's data bind-mounted
from the HDD, and home's end of the tunnel before Docker. An instance behind
the forward (compose.forwarded.yaml) publishes 443 on the tunnel's address
alone, trusts the PROXY header from ramiel alone, and gets its certificates
through TLS-ALPN, without HTTP/3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-11-08 19:05:47 +01:00
commit 29844c7fa6
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
20 changed files with 871 additions and 4 deletions

View file

@ -8,6 +8,12 @@
# Let's Encrypt certificate by itself, and `http://localhost` for the local rehearsal.
# {$ACCOUNT_SITE} is the account domain handles are on where it isn't {$LEARN_DOMAIN}
# (`halkyon.eu`), whose profile pages this instance serves; empty for none.
# Global options an instance's setting adds (compose.forwarded.yaml: behind another
# server's HAProxy, ADR 0039); none without.
{
import global/*.caddy
}
{$LEARN_DOMAIN} {$ACCOUNT_SITE} {
encode zstd gzip

View file

@ -0,0 +1,18 @@
# An instance published through another server's HAProxy (ADR 0039, D3, D10 to D12):
# halkyon.li at home, which ramiel reaches through WireGuard (host/, forward/). Caddy
# then publishes 443 on the tunnel's address alone, so nothing of the instance answers
# on the home network, and takes the visitor's address from HAProxy's PROXY header
# (forwarded.caddy). No 80 (ramiel's Caddy sends plain HTTP on to HTTPS) and no HTTP/3
# (HAProxy routes TCP alone). Switched on in .env, after the instance's own file:
#
# COMPOSE_FILE=compose.learn.yaml:compose.forwarded.yaml:…
#
# The tunnel is the host's (wg-quick@wg-halkyon), started before Docker: host/README.md.
services:
caddy:
# In place of compose.learn.yaml's 80, 443 and 443/udp, never beside them.
ports: !override
- "10.61.97.2:443:443"
volumes:
- ./forwarded.caddy:/etc/caddy/global/forwarded.caddy:ro

View file

@ -0,0 +1,33 @@
# What ramiel's Caddy (/etc/caddy/Caddyfile) gains for HAProxy in front of it (ADR 0039,
# D3, D10, D11). Not a Caddyfile of its own: the global options below go at the very
# top of ramiel's, before `code.halkyon.app {`, and the site at its end. Setting it up:
# forward/README.md.
# --- global options: the top of /etc/caddy/Caddyfile ---
{
# HAProxy holds 443 and hands Caddy what isn't halkyon.li's on 8443, which ufw
# keeps closed from outside. Port 80 stays Caddy's.
https_port 8443
servers :8443 {
listener_wrappers {
# The visitor's address from HAProxy's PROXY header, trusted from the
# loopback alone, so the forge's log keeps visitors rather than
# 127.0.0.1.
proxy_protocol {
timeout 5s
allow 127.0.0.1/32 ::1/128
}
tls
}
# HTTP/3 runs over UDP, which HAProxy doesn't route: none, so no client is
# sent to look for it.
protocols h1 h2
}
}
# --- a site: the end of /etc/caddy/Caddyfile ---
# halkyon.li over plain HTTP, sent on to HTTPS, which HAProxy passes home; nothing of
# halkyon.li is served or certified here.
http://halkyon.li, http://*.halkyon.li {
redir https://{host}{uri} 308
}

120
deploy/forward/README.md Normal file
View file

@ -0,0 +1,120 @@
# ramiel's forward for halkyon.li
ramiel, the halkyon.app server, publishes halkyon.li, which runs at home (ADR 0039).
HAProxy takes ramiel's 443, reads the name each TLS connection asks for, and passes
halkyon.li's, unopened, through a WireGuard tunnel to the instance at home. Every other
name goes to ramiel's own Caddy, which serves `code.halkyon.app`, `halkyon.app` and
`learn.halkyon.app` as before, now on `127.0.0.1:8443`. Port 80 stays Caddy's.
| File | On ramiel | What for |
|---|---|---|
| `haproxy.cfg` | `/etc/haproxy/haproxy.cfg` | the front on 443 (D3, D10) |
| `Caddyfile.halkyon-li` | merged into `/etc/caddy/Caddyfile` | Caddy on 8443 with the PROXY header, halkyon.li's HTTP to HTTPS (D11) |
| `wg-halkyon.conf` | `/etc/wireguard/wg-halkyon.conf` | ramiel's end of the tunnel (D8) |
| `wg-halkyon.nft` | `/etc/wireguard/wg-halkyon.nft` | what the tunnel may carry (D9, D17) |
`test/deploy/halkyon_li_forward_test.dart` holds these files to ADR 0039's rules.
## Steps
All as root on ramiel (`sudo`), but step 1 on melchior. The home end is set up
alongside (`../host/README.md`): each side needs the other's public key.
1. **The files**, from a checkout on melchior, into the admin's home on ramiel:
```sh
scp deploy/forward/haproxy.cfg deploy/forward/Caddyfile.halkyon-li \
deploy/forward/wg-halkyon.conf deploy/forward/wg-halkyon.nft ramiel:
```
2. **The packages**:
```sh
sudo apt install wireguard-tools haproxy nftables
```
Ubuntu's HAProxy starts with a configuration that listens nowhere, so nothing changes
yet.
3. **ramiel's key** for the tunnel, made on ramiel and never leaving it but into Proton
Pass (vault *Infra*, an item *ramiel wg-halkyon*: a new server needs it again):
```sh
sudo install -d -m 700 /etc/wireguard
sudo sh -c 'umask 077; wg genkey > /etc/wireguard/wg-halkyon.key'
sudo cat /etc/wireguard/wg-halkyon.key # into Proton Pass
sudo wg pubkey < /etc/wireguard/wg-halkyon.key # ramiel's public key: home's config
```
4. **The tunnel's files**, with home's public key (from `../host/README.md`, step 4) in
place of `<home's public key>`:
```sh
sudo install -m 600 -o root -g root ~/wg-halkyon.conf ~/wg-halkyon.nft /etc/wireguard/
sudoedit /etc/wireguard/wg-halkyon.conf
sudo nft -c -f /etc/wireguard/wg-halkyon.nft # checks the rules, loads nothing
```
5. **WireGuard's port**, opened in ufw:
```sh
sudo ufw allow 51820/udp comment 'WireGuard: halkyon.li at home'
```
6. **The tunnel**, up now and at every boot:
```sh
sudo systemctl enable --now wg-quick@wg-halkyon
sudo wg show wg-halkyon # a handshake once home is up too
sudo nft list table inet halkyon_tunnel
```
7. **Caddy moved behind HAProxy.** Keep the Caddyfile as it is, then put
`Caddyfile.halkyon-li`'s global options at the very top of `/etc/caddy/Caddyfile`
(before `code.halkyon.app {`) and its site at the end, and check it:
```sh
sudo cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.before-haproxy
sudoedit /etc/caddy/Caddyfile
sudo caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
sudo install -m 644 ~/haproxy.cfg /etc/haproxy/haproxy.cfg
sudo haproxy -c -f /etc/haproxy/haproxy.cfg
```
8. **The switch**, the forge away for the seconds between the two commands: Caddy lets go
of 443, HAProxy takes it.
```sh
sudo systemctl reload caddy && sudo systemctl restart haproxy
sudo ss -tlnp | grep -E ':(443|8443) ' # haproxy on 443, caddy on 8443
curl -sI https://code.halkyon.app | head -1 # from melchior: HTTP/2 200
```
If anything fails, step 8 is undone by `sudo systemctl stop haproxy`, the old
Caddyfile copied back and `sudo systemctl reload caddy`.
9. **Checks** once home's Caddy runs (`../host/README.md`):
```sh
curl -sI https://learn.halkyon.li | head -1 # from anywhere: an answer from home
curl -sI http://learn.halkyon.li | head -3 # 308 to https://, from ramiel
```
## Taking it away
```sh
sudo systemctl disable --now wg-quick@wg-halkyon haproxy
sudo ufw delete allow 51820/udp
sudo cp /etc/caddy/Caddyfile.before-haproxy /etc/caddy/Caddyfile
sudo systemctl reload caddy
```
Caddy then holds 443 and serves HTTP/3 again, as before.
## What changes for the forge
- Caddy serves no HTTP/3 any more (`protocols h1 h2`). Browsers and git use HTTP/2,
which they fall back to on their own.
- Its log keeps visitors' addresses, from HAProxy's PROXY header, which Caddy trusts from
the loopback alone.
- HAProxy logs nothing.

View file

@ -0,0 +1,45 @@
# ramiel's front on 443 (ADR 0039, D3, D10): HAProxy reads the name a TLS connection
# asks for (SNI) and passes halkyon.li's, unopened, through the tunnel to its instance
# at home; every other name goes to ramiel's own Caddy on the loopback, which serves
# code.halkyon.app and halkyon.app as before. Both get the visitor's address in a PROXY
# header (v2). As /etc/haproxy/haproxy.cfg on ramiel; check it with
# `haproxy -c -f /etc/haproxy/haproxy.cfg`. Setting it up: forward/README.md.
#
# Nothing is logged: no `log` line, so neither the visitors of halkyon.li nor those of
# the forge are kept here (Caddy behind each side logs what it logs, as before).
global
user haproxy
group haproxy
daemon
maxconn 2000
defaults
mode tcp
timeout connect 5s
# Jellyfin's streams, Serverpod's WebSockets and a forge's long clones stay open:
# an hour without a byte either way, as a tunnel's idle time.
timeout client 1h
timeout server 1h
timeout tunnel 1h
frontend tls
bind :::443 v4v6
# Wait for the ClientHello, which names the host; a connection that sends none
# within 5 s goes to the local Caddy, which answers it as before.
tcp-request inspect-delay 5s
tcp-request content accept if { req.ssl_hello_type 1 }
# halkyon.li's names exactly, as its DNS zone has them (D7): no wildcard, so no
# other name reaches home.
acl halkyon_li req.ssl_sni -i halkyon.li learn.halkyon.li todo.halkyon.li train.halkyon.li meditate.halkyon.li jellyfin.halkyon.li
use_backend home if halkyon_li
default_backend local
# halkyon.li's Caddy at home, at its address in the tunnel (D8), which it publishes
# 443 on alone.
backend home
server balthasar 10.61.97.2:443 send-proxy-v2
# ramiel's own Caddy, moved off 443 (Caddyfile.halkyon-li).
backend local
server caddy 127.0.0.1:8443 send-proxy-v2

View file

@ -0,0 +1,17 @@
# ramiel's end of the tunnel to halkyon.li at home (ADR 0039, D8), as
# /etc/wireguard/wg-halkyon.conf on ramiel (root, 0600), brought up by
# wg-quick@wg-halkyon. Its private key is a file of its own beside it, never here.
# Setting it up: forward/README.md.
[Interface]
Address = 10.61.97.1/30
ListenPort = 51820
PostUp = wg set %i private-key /etc/wireguard/wg-halkyon.key
# HAProxy's 443 and the jump's 22 into the tunnel, nothing from it (D9, D17).
PostUp = nft -f /etc/wireguard/wg-halkyon.nft
PreDown = nft delete table inet halkyon_tunnel
# Home, which dials in: no endpoint, so ramiel never names its address.
[Peer]
PublicKey = <home's public key>
AllowedIPs = 10.61.97.2/32

View file

@ -0,0 +1,39 @@
# The rules around ramiel's end of the tunnel to halkyon.li (ADR 0039, D9, D17), loaded
# by wg-quick as the tunnel comes up and deleted as it goes (wg-halkyon.conf), so they
# live exactly as long as it. As /etc/wireguard/wg-halkyon.nft on ramiel (root, 0600).
#
# A table of their own, apart from ufw's: a packet the table drops is dropped whatever
# ufw says, and what it accepts still meets ufw's rules after it.
#
# - Into the tunnel: new connections to home's Caddy (443, from HAProxy) and sshd (22,
# the jump from away), then their replies; nothing else.
# - From the tunnel: replies alone, so a taken-over home reaches neither ramiel's sshd
# nor Forgejo nor anything else listening here.
# - Through the tunnel: nothing; ramiel routes nothing between it and anywhere else.
#
# IPv4 alone: the tunnel carries no IPv6 (D8).
# Made and deleted first, so loading this again replaces it whole.
table inet halkyon_tunnel {}
delete table inet halkyon_tunnel
table inet halkyon_tunnel {
chain input {
type filter hook input priority filter - 1; policy accept;
iifname "wg-halkyon" ct state established,related accept
iifname "wg-halkyon" drop
}
chain output {
type filter hook output priority filter - 1; policy accept;
oifname "wg-halkyon" ct state established,related accept
oifname "wg-halkyon" ip daddr 10.61.97.2 tcp dport { 22, 443 } accept
oifname "wg-halkyon" drop
}
chain forward {
type filter hook forward priority filter - 1; policy accept;
iifname "wg-halkyon" drop
oifname "wg-halkyon" drop
}
}

25
deploy/forwarded.caddy Normal file
View file

@ -0,0 +1,25 @@
# Caddy's global options for an instance behind HAProxy (ADR 0039, D10, D11), which
# compose.forwarded.yaml mounts into /etc/caddy/global/ and the Caddyfile's global
# options import.
servers {
listener_wrappers {
# The visitor's address from HAProxy's PROXY header, trusted from ramiel's
# tunnel address alone (where Docker hands its connections on unchanged),
# so the servers' per-network limits count visitors, not ramiel.
proxy_protocol {
timeout 5s
allow 10.61.97.1/32
}
tls
}
# HTTP/3 runs over UDP, which HAProxy doesn't route: none, so no client is sent
# looking for it at ramiel, whose UDP 443 is its own Caddy's.
protocols h1 h2
}
# Certificates through the TLS-ALPN challenge alone, which reaches here through
# HAProxy unopened; port 80 isn't published, as ramiel answers it.
cert_issuer acme {
disable_http_challenge
}

132
deploy/host/README.md Normal file
View file

@ -0,0 +1,132 @@
# The instance's host at home
halkyon.li runs at home on balthasar (ADR 0039) and is published through ramiel
(`../forward/README.md`). This folder holds what the host needs beyond Docker: SSH for
melchior, the root disk unlocked over SSH at boot, Docker's data on the HDD, and home's
end of the tunnel.
| File | On the host | What for |
|---|---|---|
| `sshd-halkyon.conf` | `/etc/ssh/sshd_config.d/10-halkyon.conf` | keys alone, two users (D17) |
| `dropbear.conf` | `/etc/dropbear/initramfs/dropbear.conf` | the passphrase over SSH at boot (D20) |
| `initramfs-modules` | appended to `/etc/initramfs-tools/modules` | the USB network adapter at boot (D20) |
| `fstab-docker` | appended to `/etc/fstab` | Docker's and containerd's data on the HDD (D12) |
| `docker-after-data.conf` | `docker.service.d/` and `containerd.service.d/` | both wait for it (D12) |
| `wg-halkyon.conf` | `/etc/wireguard/wg-halkyon.conf` | home's end of the tunnel (D8) |
| `wg-halkyon.nft` | `/etc/wireguard/wg-halkyon.nft` | what the tunnel may carry (D9, D17) |
| `wg-quick-before-docker.conf` | `wg-quick@wg-halkyon.service.d/` | the tunnel before Docker (D12) |
`test/deploy/halkyon_li_forward_test.dart` holds these files to ADR 0039's rules.
## Steps
As the developer at the iMac (`sudo`), from this checkout (`~/project-halkyon/halkyon`),
in this order: each step leaves the machine working, and the early ones keep a way in
for the later.
1. **sshd for melchior** (D17). On balthasar, then from melchior, then on balthasar:
```sh
sudo apt install openssh-server
```
```sh
ssh-copy-id raisondetredev@192.168.178.53 # on melchior, with balthasar's password
```
```sh
sudo install -m 644 deploy/host/sshd-halkyon.conf /etc/ssh/sshd_config.d/10-halkyon.conf
sudo sshd -t && sudo systemctl reload ssh
```
Then, before closing anything, from melchior: `ssh raisondetredev@192.168.178.53`
must still get in, and `ssh -o PubkeyAuthentication=no raisondetredev@192.168.178.53`
must be refused. In the FRITZ!Box, give balthasar always the same address
(*Heimnetz › Netzwerk › balthasar › Diesem Netzwerkgerät immer die gleiche
IPv4-Adresse zuweisen*), so that 192.168.178.53 stays its address.
2. **The passphrase over SSH at boot** (D20):
```sh
sudo apt install dropbear-initramfs
sudo install -m 644 deploy/host/dropbear.conf /etc/dropbear/initramfs/dropbear.conf
# melchior's public key alone (from ~/.ssh/authorized_keys, step 1), restricted:
printf 'no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="/usr/bin/cryptroot-unlock" %s\n' \
"$(cat ~/.ssh/authorized_keys)" | sudo tee /etc/dropbear/initramfs/authorized_keys
grep -v '^#' deploy/host/initramfs-modules | sudo tee -a /etc/initramfs-tools/modules
echo 'IP=::::balthasar:enx0050b6bffcff:dhcp' | sudo tee -a /etc/initramfs-tools/initramfs.conf
sudo update-initramfs -u
```
At the next boot, at home: `ssh -p 2222 root@192.168.178.53` from melchior asks for the
root disk's passphrase, and the boot goes on. The keyboard at the iMac still works as
before, so a mistake here costs no more than the walk to it.
3. **No desktop** (D12): it stops starting at boot, and Docker Desktop, which runs a VM of
its own beside Docker, goes. Multipass only if it isn't used:
```sh
sudo systemctl set-default multi-user.target # back: graphical.target
sudo apt remove docker-desktop
sudo snap disable multipass
```
4. **Docker's data on the HDD** (D12). Everything Docker holds now (the home Jellyfin's
images and containers) moves with it. Its configuration and library are in
`deploy/home-jellyfin/`, untouched:
```sh
(cd deploy/home-jellyfin && sudo docker compose down)
sudo systemctl stop docker docker.socket containerd
sudo mkdir -p /data/docker /data/containerd
sudo rsync -aHAX --numeric-ids /var/lib/docker/ /data/docker/
sudo rsync -aHAX --numeric-ids /var/lib/containerd/ /data/containerd/
sudo mv /var/lib/docker /var/lib/docker.old
sudo mv /var/lib/containerd /var/lib/containerd.old
sudo mkdir /var/lib/docker /var/lib/containerd
grep -v '^#' deploy/host/fstab-docker | sudo tee -a /etc/fstab
for unit in docker containerd; do
sudo install -D -m 644 deploy/host/docker-after-data.conf \
/etc/systemd/system/$unit.service.d/after-data.conf
done
sudo systemctl daemon-reload
sudo mount /var/lib/docker && sudo mount /var/lib/containerd
sudo systemctl start containerd docker
sudo docker images # as before
```
Once Docker runs as before: `sudo rm -rf /var/lib/docker.old /var/lib/containerd.old`.
5. **Home's end of the tunnel** (D8, D9, D12). Home's key, made here and never leaving
but into Proton Pass (vault *Infra*, an item *balthasar wg-halkyon*):
```sh
sudo apt install wireguard-tools
sudo install -d -m 700 /etc/wireguard
sudo sh -c 'umask 077; wg genkey > /etc/wireguard/wg-halkyon.key'
sudo cat /etc/wireguard/wg-halkyon.key # into Proton Pass
sudo wg pubkey < /etc/wireguard/wg-halkyon.key # home's public key: ramiel's config
```
Then the files, with ramiel's public key (`../forward/README.md`, step 3) in place of
`<ramiel's public key>`:
```sh
sudo install -m 600 deploy/host/wg-halkyon.conf deploy/host/wg-halkyon.nft /etc/wireguard/
sudoedit /etc/wireguard/wg-halkyon.conf
sudo nft -c -f /etc/wireguard/wg-halkyon.nft # checks the rules, loads nothing
sudo install -D -m 644 deploy/host/wg-quick-before-docker.conf \
/etc/systemd/system/wg-quick@wg-halkyon.service.d/before-docker.conf
sudo systemctl daemon-reload
sudo systemctl enable --now wg-quick@wg-halkyon
sudo wg show wg-halkyon # a handshake once ramiel is up too
ping -c 1 10.61.97.1 # refused by home's own rules: nothing new goes to ramiel
```
From melchior, away from home: `ssh -J ramiel raisondetredev@10.61.97.2`.
6. **Reboot**, at home, to see each step hold: the passphrase over SSH (step 2), no
desktop (step 3), Docker on the HDD (step 4), the tunnel up before Docker (step 5).
The instance itself follows (its users, checkout, `.env` and first deploy), once its
compose files for Meditate and Jellyfin are built.

View file

@ -0,0 +1,7 @@
# Docker's and containerd's data are bind mounts from the HDD (ADR 0039, D12:
# fstab-docker): both services start once they are in place. As
# /etc/systemd/system/docker.service.d/after-data.conf and
# /etc/systemd/system/containerd.service.d/after-data.conf on the instance's host at
# home.
[Unit]
RequiresMountsFor=/var/lib/docker /var/lib/containerd

11
deploy/host/dropbear.conf Normal file
View file

@ -0,0 +1,11 @@
# The SSH server of the initramfs, through which the root disk's passphrase is typed at
# boot (ADR 0039, D20), as /etc/dropbear/initramfs/dropbear.conf on the instance's host
# at home. It waits on the home network alone (the tunnel to ramiel isn't up that
# early), on 2222, so its host key never clashes with sshd's in melchior's known_hosts.
#
# -p 2222 its port
# -s no passwords: melchior's key alone
# -j -k no forwarding of any kind
# -I 300 a connection idle for 5 minutes is closed
# -c cryptroot-unlock runs nothing but the unlock, whatever the client asks
DROPBEAR_OPTIONS="-p 2222 -s -j -k -I 300 -c cryptroot-unlock"

6
deploy/host/fstab-docker Normal file
View file

@ -0,0 +1,6 @@
# Docker's and containerd's data on the HDD (ADR 0039, D12), as /home and snapd's folder
# are on this machine: bind mounts from /data, which its LUKS key file unlocks once the
# root disk is open. Added to /etc/fstab on the instance's host at home. containerd's
# too, as Docker 29 keeps images in containerd's store.
/data/docker /var/lib/docker none bind,x-systemd.requires-mounts-for=/data 0 0
/data/containerd /var/lib/containerd none bind,x-systemd.requires-mounts-for=/data 0 0

View file

@ -0,0 +1,4 @@
# Added to /etc/initramfs-tools/modules on the instance's host at home (ADR 0039, D20):
# its network adapter is a USB one, whose driver the initramfs must load for dropbear
# to be reached at boot.
cdc_ncm

View file

@ -0,0 +1,8 @@
# sshd on the instance's host at home (ADR 0039, D17), reached over the home network
# and, from away, through ramiel's tunnel: keys alone, for the operator and deploy
# alone. As /etc/ssh/sshd_config.d/10-halkyon.conf (Ubuntu's sshd_config includes that
# folder first, so these win over its defaults). Check with `sudo sshd -t`.
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
AllowUsers raisondetredev deploy

View file

@ -0,0 +1,20 @@
# Home's end of the tunnel through which ramiel publishes halkyon.li (ADR 0039, D8), as
# /etc/wireguard/wg-halkyon.conf on the instance's host at home (root, 0600), brought
# up by wg-quick@wg-halkyon before Docker (wg-quick-before-docker.conf). Its private key
# is a file of its own beside it, never here. Setting it up: host/README.md.
[Interface]
Address = 10.61.97.2/30
PostUp = wg set %i private-key /etc/wireguard/wg-halkyon.key
# From ramiel only Caddy's 443 and sshd's 22, nothing towards it (D9, D17).
PostUp = nft -f /etc/wireguard/wg-halkyon.nft
PreDown = nft delete table inet halkyon_tunnel
# ramiel, dialled by its address: home opens nothing on the FRITZ!Box, and a new home
# address needs no change anywhere. The keepalive keeps the way back open through the
# router's NAT.
[Peer]
PublicKey = <ramiel's public key>
Endpoint = 45.129.181.128:51820
AllowedIPs = 10.61.97.1/32
PersistentKeepalive = 25

View file

@ -0,0 +1,45 @@
# The rules around home's end of the tunnel from ramiel (ADR 0039, D9, D17), loaded by
# wg-quick as the tunnel comes up and deleted as it goes (wg-halkyon.conf), so they live
# exactly as long as it. As /etc/wireguard/wg-halkyon.nft on the instance's host at home
# (root, 0600).
#
# A table of its own, apart from Docker's and ufw's, at a priority before theirs: what it
# drops is dropped whatever they say, and what it accepts still meets their rules after
# it.
#
# - To the host: new connections from ramiel to sshd alone (22, the jump from away),
# then replies.
# - Through the host: new connections from ramiel to Caddy's 443 alone (172.30.80.2 on
# edge, where Docker has sent the tunnel address's 443 by then), then replies;
# nothing to the home network, the other containers or anywhere else.
# - Towards ramiel: replies alone.
#
# IPv4 alone: the tunnel carries no IPv6 (D8).
# Made and deleted first, so loading this again replaces it whole.
table inet halkyon_tunnel {}
delete table inet halkyon_tunnel
table inet halkyon_tunnel {
chain input {
type filter hook input priority filter - 1; policy accept;
iifname "wg-halkyon" ct state established,related accept
iifname "wg-halkyon" ip saddr 10.61.97.1 tcp dport 22 accept
iifname "wg-halkyon" drop
}
chain forward {
type filter hook forward priority filter - 1; policy accept;
iifname "wg-halkyon" ct state established,related accept
iifname "wg-halkyon" ip saddr 10.61.97.1 ip daddr 172.30.80.2 tcp dport 443 accept
iifname "wg-halkyon" drop
oifname "wg-halkyon" ct state established,related accept
oifname "wg-halkyon" drop
}
chain output {
type filter hook output priority filter - 1; policy accept;
oifname "wg-halkyon" ct state established,related accept
oifname "wg-halkyon" drop
}
}

View file

@ -0,0 +1,6 @@
# The tunnel comes up before Docker (ADR 0039, D12): Caddy publishes 443 on the
# tunnel's address alone (compose.forwarded.yaml), which must exist when Docker starts
# the container. As /etc/systemd/system/wg-quick@wg-halkyon.service.d/before-docker.conf
# on the instance's host at home.
[Unit]
Before=docker.service

View file

@ -105,7 +105,7 @@ main() {
# the container keeps the old one (found 2026-10-10). A changed Caddyfile, or a
# sidecar's site, therefore recreates the container, which mounts the new file; the
# certificates are in a volume, and the instance is away for seconds.
if ! git diff --quiet "$deployed" "$commit" -- Caddyfile 'sidecars/*.caddy'; then
if ! git diff --quiet "$deployed" "$commit" -- Caddyfile 'sidecars/*.caddy' forwarded.caddy; then
docker compose up -d --force-recreate --no-deps caddy
fi
wait_until_answering server

View file

@ -1,6 +1,6 @@
# ADR 0039: halkyon.li, the family's instance at home, published through ramiel
- **Status:** decided 8 Nov 2026 by the developer: D1 to D6 and D17 to D19 are their
- **Status:** decided 8 Nov 2026 by the developer: D1 to D6 and D17 to D20 are their
own choices, D7 to D16 the agent's recommendations, which they confirmed. Nothing is
built yet. halkyon.li's DNS zone at netcup is in place (D7, checked 8 Nov).
- **Builds on:** Halkyon Learn's ADR 0021 (a private instance whose operator approves
@ -116,8 +116,10 @@
- **D12 balthasar becomes a server**:
- the desktop no longer starts (`systemctl set-default multi-user.target`, undone by
`graphical.target`), which frees about 1 GB; Multipass is disabled if unused;
- Docker's data moves to the HDD (`data-root` in `/etc/docker/daemon.json`), as the
root filesystem has room for no image;
- Docker's data moves to the HDD, as the root filesystem has room for no image:
`/var/lib/docker` and `/var/lib/containerd` (where Docker 29 keeps images) as bind
mounts from `/data`, as `/home` and snapd's folder already are on this machine,
rather than Docker's `data-root`, which would leave containerd's store behind;
- a `deploy` user holds the checkout (`~deploy/halkyon`) and runs the stack, as on
sahaquiel; the sidecars' user IDs are those of the sidecar README;
- Caddy publishes 443 on the tunnel's address alone (`10.61.97.2:443`), the tunnel
@ -169,6 +171,17 @@
by the polygons, imported by a short-lived Nominatim. The developer raised no
objection. The console's ADR decides how it runs.
- **D20 balthasar's root disk is unlocked over SSH at boot** (the developer's choice,
over Tang on ramiel, which would unlock a stolen machine that is still online, and over
typing at the iMac's keyboard alone). Found while writing the host's steps: the root
disk is LUKS-encrypted with a passphrase (`/etc/crypttab`: `none`), and the HDD
unlocks from a key file on it, so every boot waits for the passphrase.
`dropbear-initramfs` waits on the home network at boot, on port 2222 (so its host key
never clashes with sshd's in a `known_hosts`). It takes melchior's key alone, which may
run nothing but `cryptroot-unlock`. The USB network adapter's driver (`cdc_ncm`) is
added to the initramfs. From away, a reboot waits for someone at home, as the tunnel
isn't up that early. The keyboard still works as before.
## Consequences
- People see ramiel's address alone; nothing names the home address, which only ramiel

View file

@ -0,0 +1,312 @@
import 'dart:io';
import 'package:test/test.dart';
/// halkyon.li at home, published through ramiel (ADR 0039): TLS passes
/// through ramiel by name, unopened (D3, D10, D11), and the tunnel between
/// them carries what each side needs and nothing else (D8, D9, D17). None of
/// it runs in CI, so its rules are held here, refusals first.
void main() {
String read(String path) => File(path).readAsStringSync();
/// The lines of [text] without comments and blank lines, trimmed.
List<String> lines(String text) => [
for (final line in text.split('\n'))
if (line.trim().isNotEmpty && !line.trim().startsWith('#')) line.trim(),
];
const names = [
'halkyon.li',
'learn.halkyon.li',
'todo.halkyon.li',
'train.halkyon.li',
'meditate.halkyon.li',
'jellyfin.halkyon.li',
];
group("ramiel's HAProxy (D3, D10)", () {
final config = lines(read('deploy/forward/haproxy.cfg'));
test('logs nothing, so no visitor of halkyon.li is kept on ramiel', () {
expect(config.where((line) => line.startsWith('log ')), isEmpty);
expect(config, isNot(contains('option tcplog')));
});
test('passes TLS on unopened: no certificate, no HTTP', () {
expect(config.where((line) => line.contains('crt ')), isEmpty);
expect(config, isNot(contains('mode http')));
expect(config, contains('mode tcp'));
});
test("sends halkyon.li's names alone through the tunnel", () {
final acl = config.singleWhere(
(line) => line.startsWith('acl halkyon_li '),
);
expect(acl, 'acl halkyon_li req.ssl_sni -i ${names.join(' ')}');
expect(config, contains('use_backend home if halkyon_li'));
});
test('sends everything else to the local Caddy', () {
expect(config, contains('default_backend local'));
expect(config, contains('server caddy 127.0.0.1:8443 send-proxy-v2'));
});
test("reaches home's Caddy at its tunnel address alone, with the "
"visitor's address", () {
expect(
config.where((line) => line.startsWith('server ')),
unorderedEquals([
'server caddy 127.0.0.1:8443 send-proxy-v2',
'server balthasar 10.61.97.2:443 send-proxy-v2',
]),
);
});
test('listens on 443 over IPv4 and IPv6', () {
expect(config, contains('bind :::443 v4v6'));
});
});
group("ramiel's Caddy (D3, D11)", () {
final caddy = read('deploy/forward/Caddyfile.halkyon-li');
test('leaves 443 to HAProxy, trusting its PROXY header from the loopback '
'alone', () {
expect(caddy, contains('https_port 8443'));
expect(caddy, contains('allow 127.0.0.1/32 ::1/128'));
});
test('speaks no HTTP/3, which HAProxy would not route', () {
expect(caddy, contains('protocols h1 h2'));
expect(caddy, isNot(contains('h3')));
});
test('sends plain HTTP for halkyon.li on to HTTPS, and serves none of '
'it itself', () {
expect(caddy, contains('http://halkyon.li, http://*.halkyon.li {'));
expect(caddy, contains('redir https://{host}{uri} 308'));
expect(caddy, isNot(contains('reverse_proxy')));
});
});
group("ramiel's end of the tunnel (D8, D9)", () {
final config = read('deploy/forward/wg-halkyon.conf');
final rules = lines(read('deploy/forward/wg-halkyon.nft'));
test('lets home send from its own address alone', () {
expect(
RegExp(
r'^AllowedIPs = (.+)$',
multiLine: true,
).allMatches(config).map((match) => match[1]),
['10.61.97.2/32'],
);
});
test('holds no endpoint, so it never names the home address', () {
expect(config, isNot(contains('Endpoint')));
});
test('keeps its key out of the file', () {
expect(config, isNot(contains('PrivateKey')));
expect(
config,
contains(
'PostUp = wg set %i private-key /etc/wireguard/wg-halkyon.key',
),
);
});
test('sets its rules as it comes up and takes them away as it goes', () {
expect(config, contains('PostUp = nft -f /etc/wireguard/wg-halkyon.nft'));
expect(
config,
contains('PreDown = nft delete table inet halkyon_tunnel'),
);
});
test('accepts no new connection from home', () {
expect(_chain(rules, 'input'), [
'iifname "wg-halkyon" ct state established,related accept',
'iifname "wg-halkyon" drop',
]);
});
test("opens to home only HAProxy's 443 and the jump's 22 (D17)", () {
expect(_chain(rules, 'output'), [
'oifname "wg-halkyon" ct state established,related accept',
'oifname "wg-halkyon" ip daddr 10.61.97.2 tcp dport { 22, 443 } accept',
'oifname "wg-halkyon" drop',
]);
});
test('forwards nothing through the tunnel', () {
final forward = _chain(rules, 'forward');
expect(forward, contains('iifname "wg-halkyon" drop'));
expect(forward, contains('oifname "wg-halkyon" drop'));
expect(forward.where((line) => line.endsWith('accept')), isEmpty);
});
});
group("home's end of the tunnel (D8, D9, D12, D17)", () {
final config = read('deploy/host/wg-halkyon.conf');
final rules = lines(read('deploy/host/wg-halkyon.nft'));
test('lets ramiel send from its own address alone', () {
expect(
RegExp(
r'^AllowedIPs = (.+)$',
multiLine: true,
).allMatches(config).map((match) => match[1]),
['10.61.97.1/32'],
);
});
test('dials ramiel by its address and keeps the way open', () {
expect(config, contains('Endpoint = 45.129.181.128:51820'));
expect(config, contains('PersistentKeepalive = 25'));
});
test('keeps its key out of the file', () {
expect(config, isNot(contains('PrivateKey')));
expect(
config,
contains(
'PostUp = wg set %i private-key /etc/wireguard/wg-halkyon.key',
),
);
});
test('accepts from ramiel only the jump to sshd, nothing else on the '
'host', () {
expect(_chain(rules, 'input'), [
'iifname "wg-halkyon" ct state established,related accept',
'iifname "wg-halkyon" ip saddr 10.61.97.1 tcp dport 22 accept',
'iifname "wg-halkyon" drop',
]);
});
test("passes from ramiel only to Caddy's 443, nothing else of the home "
'network or the containers', () {
expect(_chain(rules, 'forward'), [
'iifname "wg-halkyon" ct state established,related accept',
_caddyFromRamiel,
'iifname "wg-halkyon" drop',
'oifname "wg-halkyon" ct state established,related accept',
'oifname "wg-halkyon" drop',
]);
});
test('opens nothing towards ramiel', () {
expect(_chain(rules, 'output'), [
'oifname "wg-halkyon" ct state established,related accept',
'oifname "wg-halkyon" drop',
]);
});
test('comes up before Docker, which publishes on its address', () {
final dropIn = read('deploy/host/wg-quick-before-docker.conf');
expect(dropIn, contains('Before=docker.service'));
});
test('lets sshd take keys alone, for two users alone', () {
final sshd = lines(read('deploy/host/sshd-halkyon.conf'));
expect(sshd, contains('PasswordAuthentication no'));
expect(sshd, contains('KbdInteractiveAuthentication no'));
expect(sshd, contains('PermitRootLogin no'));
expect(sshd, contains('AllowUsers raisondetredev deploy'));
});
});
group('the host at home (D12, D20)', () {
test('unlocks its disk over SSH at boot, on a port of its own, and runs '
'nothing else there', () {
final dropbear = read('deploy/host/dropbear.conf');
final options = RegExp(
r'^DROPBEAR_OPTIONS="(.+)"$',
multiLine: true,
).firstMatch(dropbear)![1]!.split(' ');
// No password, no forwarding of any kind, nothing but the unlock.
expect(options, containsAll(['-p', '2222', '-s', '-j', '-k']));
expect(options, containsAll(['-c', 'cryptroot-unlock']));
});
test("loads the USB network adapter's driver at boot", () {
expect(lines(read('deploy/host/initramfs-modules')), contains('cdc_ncm'));
});
test("keeps Docker's data on the HDD, which Docker waits for", () {
final fstab = lines(read('deploy/host/fstab-docker'))
.map((line) => line.split(RegExp(r'\s+')).take(2).join(' '));
expect(fstab, [
'/data/docker /var/lib/docker',
'/data/containerd /var/lib/containerd',
]);
expect(
read('deploy/host/docker-after-data.conf'),
contains('RequiresMountsFor=/var/lib/docker /var/lib/containerd'),
);
});
});
group("home's Caddy behind the forward (D10, D11, D12)", () {
final compose = read('deploy/compose.forwarded.yaml');
final global = read('deploy/forwarded.caddy');
test('publishes 443 on the tunnel address alone, and neither 80 nor '
'HTTP/3', () {
final ports = RegExp(r'ports: !override\n((?:\s+- .+\n)+)')
.firstMatch(compose)![1]!
.split('\n')
.map((line) => line.trim())
.where((line) => line.isNotEmpty);
expect(ports, ['- "10.61.97.2:443:443"']);
});
test("trusts the PROXY header from ramiel's tunnel address alone", () {
expect(global, contains('allow 10.61.97.1/32'));
expect(RegExp('allow (.+)').allMatches(global).map((match) => match[1]), [
'10.61.97.1/32',
]);
});
test('gets its certificates through TLS alone, and speaks no HTTP/3', () {
expect(global, contains('disable_http_challenge'));
expect(global, contains('protocols h1 h2'));
});
test("is mounted into Caddy's global options", () {
expect(
compose,
contains('./forwarded.caddy:/etc/caddy/global/forwarded.caddy:ro'),
);
expect(read('deploy/Caddyfile'), contains('import global/*.caddy'));
});
test('recreates Caddy when it changes, as a sidecar site does', () {
expect(read('deploy/redeploy.sh'), contains('forwarded.caddy'));
});
});
}
/// home's one rule passing anything from ramiel on: to Caddy's 443.
const _caddyFromRamiel =
'iifname "wg-halkyon" ip saddr 10.61.97.1 ip daddr 172.30.80.2 '
'tcp dport 443 accept';
/// The rules of [rules]' chain [name], from its first rule to its closing
/// brace, without its `type … hook …` line.
List<String> _chain(List<String> rules, String name) {
final start = rules.indexOf('chain $name {');
expect(start, isNot(-1), reason: 'no chain $name');
final chain = <String>[];
for (final line in rules.skip(start + 1)) {
if (line == '}') {
break;
}
if (!line.startsWith('type ')) {
chain.add(line);
}
}
return chain;
}