A tunnel that fails to come up takes Jellyfin's internet with it, as linuxserver/wireguard's client mode deletes the default route. Named by learn.halkyon.eu, the endpoint needed DNS to come up, which the home end starting or waking hasn't always got yet; by its address the tunnel comes up whatever the network does. Closes ADR 0032's open point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
208 lines
14 KiB
Markdown
208 lines
14 KiB
Markdown
# ADR 0032: Jellyfin at home, published by the instance through WireGuard
|
||
|
||
- **Status:** decided 30 Oct 2026, by the developer (D1, D9 and D10 their own choices,
|
||
D2–D8 agreed as recommended). Built the same day (*As built*, below); not brought up
|
||
yet, which takes the developer's steps as root on the instance's server.
|
||
- **Builds on:** the Jellyfin sidecar (`deploy/compose.jellyfin.yaml`,
|
||
`deploy/sidecars/README.md`, decided 14 Oct 2026) and Halkyon Learn's suggested
|
||
server (`JellyfinSuggestion`, decided 13 Oct). Train's Jellyfin (ADR 0030, D47, D52,
|
||
D54) reaches it the same way.
|
||
|
||
## Context
|
||
|
||
The Jellyfin sidecar runs Jellyfin on the instance's server, with its media copied
|
||
there (`deploy/jellyfin/media/`). The developer's media is at home, on a machine in the
|
||
home network (192.168.178.0/24, behind a FRITZ!Box), and should be reachable as
|
||
`jellyfin.halkyon.eu`, where learn.halkyon.eu's people would find it, under three
|
||
conditions the developer set on 30 Oct:
|
||
|
||
- the home network's address is never exposed: neither DNS nor anything Jellyfin
|
||
answers names it;
|
||
- nobody signs up: accounts are the admin's to make (Jellyfin has no self-registration,
|
||
but a fresh server's setup wizard is open to whoever reaches it first, and a user's
|
||
password can be set from outside, which `deploy/sidecars/jellyfin.caddy` refuses);
|
||
- Halkyon's apps reach Jellyfin over HTTPS alone (Learn's `JellyfinRepository.find`,
|
||
Train's D54), so the name needs a certificate every device trusts.
|
||
|
||
Found on 30 Oct: halkyon.eu's DNS is at Infomaniak; `jellyfin.halkyon.eu` already points
|
||
at learn.halkyon.eu's server (`A` 89.58.53.154 and an `AAAA`), but nothing answers there,
|
||
as the sidecar isn't switched on. With `JELLYFIN_URL` unset, the instance suggests
|
||
`https://jellyfin.halkyon.eu` to its people as soon as a Jellyfin server answers there.
|
||
|
||
## Decisions
|
||
|
||
- **D1 A WireGuard tunnel from home to the instance's server, which publishes Jellyfin**
|
||
(the developer's choice, 30 Oct, as recommended). Alternatives weighed:
|
||
- *Cloudflare Tunnel*: hides the address too, but needs all of halkyon.eu's DNS moved
|
||
to Cloudflare (a tunnel needs a zone on Cloudflare's nameservers on the free plan),
|
||
lets Cloudflare read every password, token and video (it ends TLS), conflicts with
|
||
Cloudflare's terms for serving mostly video outside its paid video products, and
|
||
leaves `jellyfin.caddy`'s rules to be rebuilt as Cloudflare's.
|
||
- *An SSH reverse tunnel* (`ssh -R` as `deploy`, kept up by autossh): no new software
|
||
on the server, but TCP over TCP stalls a stream on a lossy line, and binding where a
|
||
container reaches it needs sshd's `GatewayPorts` changed.
|
||
- *The sidecar as built*, the media copied to the server: no tunnel at all, but the
|
||
server's disk holds the library.
|
||
|
||
The public sees only the instance's server, TLS ends on a machine the operator holds,
|
||
`jellyfin.caddy` and the sidecar README's settings hold unchanged, and the DNS record
|
||
is already right. Streams are bounded by the home line's upload speed and count
|
||
towards the server's traffic.
|
||
|
||
- **D2 The home end runs in Compose, Jellyfin inside the tunnel's network namespace**
|
||
(recommended; over WireGuard on the home host with Jellyfin's port bound to the
|
||
tunnel's address, which Docker can't bind before the interface is up, and over
|
||
firewall rules on the home host). A `wireguard` container (`cap_add: NET_ADMIN`, the
|
||
host's kernel module) holds the tunnel; Jellyfin joins it with
|
||
`network_mode: service:wireguard`, so it listens on the tunnel and on that namespace's
|
||
loopback alone, and is not on the home network at all (no DLNA, no discovery on UDP
|
||
7359). The admin port is published on the home host's loopback only
|
||
(`127.0.0.1:8096`), where Learn on the same machine may reach it over plain HTTP
|
||
(`isLoopback`). The home end dials out with `PersistentKeepalive = 25`, so the FRITZ!Box
|
||
needs no open port, and the server's config holds no endpoint for it. The files live
|
||
beside the deploy as `deploy/home-jellyfin/` (compose file, WireGuard config without
|
||
its key; the steps in the sidecar README); the key stays out of the repository.
|
||
|
||
- **D3 The tunnel's addresses are 10.61.96.1 (the server) and 10.61.96.2 (home), a /30,
|
||
IPv4 alone** (recommended): outside Docker's address pools (172.17.0.0/16 to
|
||
172.31.0.0/16 and 192.168.0.0/16 in /20s, which a new network could take), the edge
|
||
network (172.30.80.0/24), the home network and Multipass' bridges on the laptop
|
||
(10.97.x.0/24). Caddy reaches Jellyfin over IPv4; visitors keep coming over either.
|
||
|
||
- **D4 WireGuard on the server's host, as `wg-quick@wg-jellyfin`, on UDP 51820**
|
||
(recommended, over a WireGuard container on `edge`, which would need `NET_ADMIN` beside
|
||
the instance and routes into it from Caddy). The port is opened in the host's
|
||
firewall; WireGuard answers nothing that isn't signed by a known key, so it shows no
|
||
service to a scan. The home peer's `AllowedIPs` on the server is `10.61.96.2/32`, so
|
||
nothing it sends under another source address gets in. The server's key is kept in
|
||
`/etc/wireguard/` (root, 0600) and in the operator's password manager, as a new
|
||
server needs it again.
|
||
|
||
- **D5 The tunnel reaches Jellyfin from Caddy alone, and reaches nothing on the server**
|
||
(recommended), by rules that `wg-quick`'s `PostUp` sets and `PreDown` removes, so they
|
||
live exactly as long as the interface:
|
||
- in `DOCKER-USER`: `172.30.80.2` (Caddy) to `10.61.96.2` on TCP 8096 accepted; every
|
||
other new connection into or out of `wg-jellyfin` dropped, replies
|
||
(`ESTABLISHED,RELATED`) passing;
|
||
- in `INPUT`: nothing new from `wg-jellyfin` accepted, so home can't reach sshd,
|
||
published ports or anything else listening on the host.
|
||
|
||
The rules match Caddy's own address, as `DOCKER-USER` sees packets before Docker
|
||
masquerades them; home then sees them from `10.61.96.1` (Docker's `MASQUERADE` for
|
||
`edge`, which takes every packet leaving the network; found while building, the
|
||
proposal had them arrive as `172.30.80.2`), so home's `AllowedIPs` is
|
||
`10.61.96.1/32` alone. A compromised home machine thus gets replies to Caddy's
|
||
requests and nothing else of the production server.
|
||
|
||
- **D6 The CI runner can't reach the tunnel either** (recommended): `10.61.96.0/30` is
|
||
added to `IPAddressDeny` in `deploy/runner/forgejo-runner.service`, beside Docker's
|
||
ranges. Without it, a job could talk to the home Jellyfin through the host's route.
|
||
|
||
- **D7 One site, with its upstream a setting** (recommended, over a second site file
|
||
repeating the rules): `jellyfin.caddy` proxies to `{$JELLYFIN_UPSTREAM:jellyfin:8096}`.
|
||
A new `deploy/compose.jellyfin-home.yaml` mounts the same site into Caddy, sets
|
||
`JELLYFIN_DOMAIN` and `JELLYFIN_UPSTREAM=10.61.96.2:8096`, leaves the server's
|
||
`JELLYFIN_URL` unset (D9) and runs no Jellyfin container; `COMPOSE_FILE` takes either it or
|
||
`compose.jellyfin.yaml`, never both. `redeploy.sh` needs no change: WireGuard is the
|
||
host's, set up once by root, as the sidecars' users are.
|
||
|
||
- **D8 Jellyfin's own settings for a server seen through a proxy** (recommended; the
|
||
sidecar README's step 6, with two additions): `JELLYFIN_PublishedServerUrl =
|
||
https://jellyfin.halkyon.eu`, so Jellyfin never hands out a home address; **LAN
|
||
networks** set to `172.29.96.1/32` alone (the gateway of the home end's Compose
|
||
network, its subnet fixed for it, through which Docker hands on what reaches the
|
||
machine's loopback; the proposal said `127.0.0.1/32`, which no request comes from),
|
||
as Jellyfin counts every private range as local when it's left empty, which would
|
||
count the tunnel too and let an admin without remote access sign in through it;
|
||
**Known proxies** `10.61.96.1`, as which Caddy's requests arrive (D5). The admin works
|
||
from the home machine's `http://localhost:8096`, not through an SSH tunnel; the
|
||
wizard is finished there before the tunnel is first brought up.
|
||
|
||
- **D9 learn.halkyon.eu suggests it to all its people** (the developer's, 30 Oct, over
|
||
`JELLYFIN_URL=none`), while it answers: the compose file leaves `JELLYFIN_URL` unset,
|
||
so the instance looks at `https://jellyfin.` on its own domain (`JellyfinSuggestion`)
|
||
and suggests it only once it answers as a Jellyfin server. Set, the instance would
|
||
suggest it whether it answers or not, which D10 makes the wrong default. Its login
|
||
screen shows only the accounts meant for everyone; the admin and every personal
|
||
account are hidden from it (D8, the sidecar README's step 6).
|
||
- **D10 The home end is the developer's own machine, this laptop** (the developer's, 30
|
||
Oct, over an always-on machine at home): Jellyfin is reachable while it runs and away
|
||
otherwise, which is accepted. It dials out, so it serves from wherever the laptop is
|
||
online, not only at home, as long as outgoing UDP passes. Asleep, the tunnel stops; on
|
||
waking, the next keepalive brings it back within 25 s.
|
||
|
||
- **D11 The home end dials the server by its address** (the developer's, 31 Oct, found
|
||
setting the home end up): `Endpoint = 89.58.53.154:51820` in `wg0.conf`, not
|
||
`learn.halkyon.eu`. Whenever `wg0.conf` fails to come up, linuxserver/wireguard's client
|
||
mode deletes the namespace's default route (`ip route del default` in its
|
||
`svc-wireguard/run`, ls126: a kill switch, for the VPN clients it is made for), so
|
||
Jellyfin, in that namespace, reaches neither TMDb nor anything else, and a library
|
||
scanned then gets frames of its videos for posters, until both containers are
|
||
recreated. Bringing an interface up needs no network, but resolving a name does
|
||
(`wg` retries it for some seconds, then fails), so on the home end (D10), starting or
|
||
waking before its network and DNS, the name made that failure recur. By address it
|
||
comes up whatever the network does, the handshake following once it is back; what
|
||
is left is a placeholder or a wrong key, met once while setting it up, which the
|
||
sidecar README's step 4 says how to recover from. IPv4, which every home network has.
|
||
A new address of the server's has to be written in by hand. Over a custom service
|
||
in the image's `/custom-services.d` putting the route back (working against the
|
||
image's kill switch and resting on its scripts as ls126 has them, which an update
|
||
may change unnoticed), and over turning the namespace round, Jellyfin owning it and
|
||
a WireGuard container without a kill switch joining it (a new D2 for a failure the
|
||
address all but removes).
|
||
|
||
## Consequences
|
||
|
||
- People reaching `jellyfin.halkyon.eu` see the instance's server alone. The instance's
|
||
server knows the home address (the tunnel's endpoint, in `wg show`), as its operator
|
||
does anyway; nothing it serves names it.
|
||
- Every byte streamed crosses the home line's upload and the server twice (in through
|
||
the tunnel, out to the visitor). Learn streams as stored (`static=true`), so a video's
|
||
bitrate must fit the upload; Train's smaller copies (D52) are transcoded at home.
|
||
- The production server gains a listening UDP port, a kernel interface and firewall
|
||
rules outside Compose; they belong in the sidecar README's steps, with how to switch
|
||
them off (`systemctl disable --now wg-quick@wg-jellyfin`, the port closed).
|
||
- With the laptop off, asleep or offline, Caddy answers 502 at `jellyfin.halkyon.eu`;
|
||
Learn and Train show the server as unreachable, as for any Jellyfin that is away. The
|
||
instance stops suggesting it at its first look after that: a look counts for an hour,
|
||
and the first `instance.info()` after the hour starts the next one in the background,
|
||
that answer still giving the old suggestion. Apps that took the suggestion already
|
||
keep it (the app asks once), and devices signed in to it keep their token.
|
||
- The instance's own look reaches `jellyfin.halkyon.eu` at its public address from
|
||
inside its container, through the host back to Caddy. With the sidecar this was never
|
||
used in production, as its compose file sets `JELLYFIN_URL`; the check after setup
|
||
confirms that `instance.info()` names it.
|
||
|
||
## As built (30 Oct 2026, the agent's calls, each with its reason)
|
||
|
||
- **Files**: `deploy/compose.jellyfin-home.yaml` and `jellyfin.caddy`'s upstream
|
||
(`{$JELLYFIN_UPSTREAM:jellyfin:8096}`, D7); `deploy/home-jellyfin/` with the home end's
|
||
`compose.yaml` and `wg0.conf.example`, and under `server/` the instance's
|
||
`wg-jellyfin.conf`, its rules (`wg-jellyfin-rules.sh`, POSIX sh as the host's
|
||
scripts are: the host has no Dart) and a systemd drop-in; the steps in
|
||
`deploy/sidecars/README.md`, *Jellyfin at home*. `test/deploy/jellyfin_home_test.dart`
|
||
holds each rule above to the files, refusals first: no `JELLYFIN_URL`, the home peer's
|
||
one address, the drops, the runner's deny, Jellyfin inside the tunnel's namespace,
|
||
its port on the loopback alone.
|
||
- **The rules in chains of their own** (`halkyon-jellyfin-forward`, `-input`), jumped to
|
||
from `DOCKER-USER` and `INPUT` for the tunnel's interface alone, so their order
|
||
against Docker's rules is fixed and `down` removes all of them, a half-made set too.
|
||
`RETURN` rather than `ACCEPT` for what passes, so Docker's own rules still apply.
|
||
- **The tunnel starts after Docker and restarts with it** (`After`, `Requires`,
|
||
`PartOf=docker.service`): its rules go into Docker's chain, which exists only once
|
||
Docker runs.
|
||
- **The server's private key is a file beside the config** (`PostUp = wg set %i
|
||
private-key …`), so `wg-jellyfin.conf` is copied from the repository as it is, but
|
||
for the home end's public key.
|
||
- **The server looks at Jellyfin through Caddy on `edge`** (`extra_hosts`, the domain at
|
||
172.30.80.2), not at the public address from inside its container, which needs
|
||
Docker's hairpin NAT and, by the `AAAA` record, IPv6 to work for it. The certificate
|
||
is the domain's, so the look is as anyone's.
|
||
- **CI's Sandbox step also fails if a job reaches 10.61.96.2:8096**, as it does for
|
||
Docker's networks: the deny in the runner's unit is only on the server once installed
|
||
again (step 3).
|
||
- **The home end's image** is `linuxserver/wireguard` 1.0.20260223-r0-ls126, of 15 Oct
|
||
2026, pinned by its digest: two weeks old, rather than the build of the day before;
|
||
its client mode brings up `wg0.conf` with the host's kernel module, and its `wg`
|
||
makes the home end's keys, so nothing is installed on the machine. Jellyfin's image
|
||
is the sidecar's, and so is its user ID, 61096.
|