Decided 2026-10-01. CI run #32 died in its analysis: a cold `dart analyze`
of the workspace with the lint plugin now peaks at about 3.7 GB (2.1 GB
warm), measured with an empty analysis cache, while the runner's unit
allowed 3 GB. The kernel killed it, systemd's default OOMPolicy (stop) then
stopped the whole unit, and the job's remaining processes got SIGTERM: the
"signal: terminated" in the log. The packages this branch added tipped it
over.
The runner may now use 5 GB of sahaquiel's 8, and above 4 GB the kernel
reclaims from CI first (MemoryHigh), so the instance keeps at least 3 GB;
it uses under 1 GB today. Installed by hand on sahaquiel, like the rest of
the unit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
Installing the runner on sahaquiel, `useradd --system` gave `runner` user ID 999: the ID
under which PostgreSQL and the server run inside their containers, which Docker doesn't
remap. To the kernel, those processes were the runner's own, so a job could have read the
server's environment (database password, session pepper, GitHub secret) and, through
/proc, PostgreSQL's files and its local socket, which the official image trusts. The
sandbox's other walls don't help there: ProtectProc hides only other users' processes,
IPAddressDeny doesn't cover Unix sockets. Only our own check ran as 999; `runner` now has
user and group ID 61000.
The check job now starts with a Sandbox step that fails unless only the runner's own
processes are visible, all in one process namespace, /home/deploy is invisible, the
Docker socket is unusable and Docker's bridge network unreachable. It would have caught
this; outside a sandbox it fails as it should. The unit and AGENTS.md say how to create
the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
