build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# The Forgejo Actions runner of this repository's CI, on halkyon.li's host at home (ADR
|
|
|
|
|
# 0039, D6, D16; on learn.halkyon.eu's server until it retired in November 2026).
|
|
|
|
|
# Installed as /etc/systemd/system/forgejo-runner.service; its configuration is
|
|
|
|
|
# config.yml, next to this file. It runs in the nightly window alone: started at 02:00 by
|
|
|
|
|
# forgejo-runner.timer, stopped three hours later (RuntimeMaxSec), while night.sh rests
|
|
|
|
|
# the instance's heavy sidecars (halkyon-night.timer, halkyon-morning.timer). Jobs pushed
|
|
|
|
|
# during the day wait in Forgejo's queue.
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
#
|
|
|
|
|
# Jobs run directly on the host as the unprivileged user `runner` ("without any
|
|
|
|
|
# isolation" from that user, as Forgejo's security guide puts it), and they run our
|
|
|
|
|
# dependencies' build code. On the production server, so everything below keeps them
|
|
|
|
|
# away from production and makes sure CI can't starve the instance. Docker is out of
|
|
|
|
|
# reach entirely: `runner` isn't in the docker group, and the socket is hidden too.
|
ci: stop a job whose sandbox leaks, first thing
Installing the runner on sahaquiel, `useradd --system` gave `runner` user ID 999: the ID
under which PostgreSQL and the server run inside their containers, which Docker doesn't
remap. To the kernel, those processes were the runner's own, so a job could have read the
server's environment (database password, session pepper, GitHub secret) and, through
/proc, PostgreSQL's files and its local socket, which the official image trusts. The
sandbox's other walls don't help there: ProtectProc hides only other users' processes,
IPAddressDeny doesn't cover Unix sockets. Only our own check ran as 999; `runner` now has
user and group ID 61000.
The check job now starts with a Sandbox step that fails unless only the runner's own
processes are visible, all in one process namespace, /home/deploy is invisible, the
Docker socket is unusable and Docker's bridge network unreachable. It would have caught
this; outside a sandbox it fails as it should. The unit and AGENTS.md say how to create
the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 22:12:31 +02:00
|
|
|
#
|
|
|
|
|
# `runner` has user and group ID 61000, and must never share a user ID with a
|
|
|
|
|
# container's processes (PostgreSQL and the server run as 999; Docker doesn't remap
|
|
|
|
|
# them): processes of one user ID can read each other's environment and, through
|
|
|
|
|
# /proc, their files. `useradd --system` picks 999 when it is free, so create it with
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# useradd --system --uid 61000 --user-group --no-create-home \
|
ci: stop a job whose sandbox leaks, first thing
Installing the runner on sahaquiel, `useradd --system` gave `runner` user ID 999: the ID
under which PostgreSQL and the server run inside their containers, which Docker doesn't
remap. To the kernel, those processes were the runner's own, so a job could have read the
server's environment (database password, session pepper, GitHub secret) and, through
/proc, PostgreSQL's files and its local socket, which the official image trusts. The
sandbox's other walls don't help there: ProtectProc hides only other users' processes,
IPAddressDeny doesn't cover Unix sockets. Only our own check ran as 999; `runner` now has
user and group ID 61000.
The check job now starts with a Sandbox step that fails unless only the runner's own
processes are visible, all in one process namespace, /home/deploy is invisible, the
Docker socket is unusable and Docker's bridge network unreachable. It would have caught
this; outside a sandbox it fails as it should. The unit and AGENTS.md say how to create
the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 22:12:31 +02:00
|
|
|
# --home-dir /var/lib/forgejo-runner --shell /usr/sbin/nologin runner
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# Its home is /data/forgejo-runner on the HDD (the root disk has no room for Flutter and
|
|
|
|
|
# the caches), seen by the runner as /var/lib/forgejo-runner (BindPaths, below):
|
|
|
|
|
# install -d -o runner -g runner -m 700 /data/forgejo-runner /var/lib/forgejo-runner
|
ci: stop a job whose sandbox leaks, first thing
Installing the runner on sahaquiel, `useradd --system` gave `runner` user ID 999: the ID
under which PostgreSQL and the server run inside their containers, which Docker doesn't
remap. To the kernel, those processes were the runner's own, so a job could have read the
server's environment (database password, session pepper, GitHub secret) and, through
/proc, PostgreSQL's files and its local socket, which the official image trusts. The
sandbox's other walls don't help there: ProtectProc hides only other users' processes,
IPAddressDeny doesn't cover Unix sockets. Only our own check ran as 999; `runner` now has
user and group ID 61000.
The check job now starts with a Sandbox step that fails unless only the runner's own
processes are visible, all in one process namespace, /home/deploy is invisible, the
Docker socket is unusable and Docker's bridge network unreachable. It would have caught
this; outside a sandbox it fails as it should. The unit and AGENTS.md say how to create
the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 22:12:31 +02:00
|
|
|
# CI's first step checks this sandbox and fails if it leaks.
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
[Unit]
|
|
|
|
|
Description=Forgejo Actions runner (CI of halkyon_learn; jobs on the host)
|
|
|
|
|
Documentation=https://forgejo.org/docs/latest/admin/actions/security/
|
|
|
|
|
After=network-online.target
|
|
|
|
|
Wants=network-online.target
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# Its folder is on the HDD, which its LUKS key file unlocks once the root disk is open.
|
|
|
|
|
RequiresMountsFor=/data
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
|
|
|
|
|
[Service]
|
|
|
|
|
User=runner
|
|
|
|
|
Group=runner
|
|
|
|
|
WorkingDirectory=/var/lib/forgejo-runner
|
|
|
|
|
ExecStart=/usr/local/bin/forgejo-runner daemon --config /etc/forgejo-runner/config.yml
|
|
|
|
|
# The runner's token, from a root-only file.
|
|
|
|
|
LoadCredential=token:/etc/forgejo-runner/token
|
|
|
|
|
Restart=on-failure
|
|
|
|
|
RestartSec=30
|
|
|
|
|
# The runner waits up to 10 minutes for a running job (shutdown_timeout).
|
|
|
|
|
TimeoutStopSec=11min
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# The nightly window's end: three hours after forgejo-runner.timer started it at 02:00. A
|
|
|
|
|
# job still running is stopped, and the next one's Leftovers step clears what it left.
|
|
|
|
|
RuntimeMaxSec=3h
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
|
|
|
|
|
# Away from production: /home (deploy's .env, the checkout, the database dumps) is
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# invisible, and so is /data, where /home really lies, with Docker's data and the media,
|
|
|
|
|
# but for the runner's own folder; the rest of the system is read-only, other users'
|
|
|
|
|
# processes are hidden, and Docker and containerd out of reach.
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
ProtectHome=yes
|
|
|
|
|
ProtectSystem=strict
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
TemporaryFileSystem=/data:ro
|
|
|
|
|
BindPaths=/data/forgejo-runner:/var/lib/forgejo-runner
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
ReadWritePaths=/var/lib/forgejo-runner
|
|
|
|
|
PrivateTmp=yes
|
|
|
|
|
ProtectProc=invisible
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
InaccessiblePaths=-/run/docker.sock -/run/containerd -/var/lib/docker -/var/lib/containerd
|
|
|
|
|
# The network: the internet over IPv4 alone. No private IPv4 network (the home network
|
|
|
|
|
# and its router, Docker's networks, where the database listens, the tunnel to ramiel,
|
|
|
|
|
# Multipass' bridges, CGNAT and link-local addresses), and no IPv6 at all: the home
|
|
|
|
|
# network's IPv6 addresses are public ones, under a prefix the provider changes, on the
|
|
|
|
|
# same interface as the internet, so no address or interface could tell them apart. All
|
|
|
|
|
# CI fetches (pub.dev, GitHub, code.halkyon.app) answers over IPv4.
|
|
|
|
|
IPAddressDeny=10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16 ::/0
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
NoNewPrivileges=yes
|
|
|
|
|
CapabilityBoundingSet=
|
|
|
|
|
AmbientCapabilities=
|
|
|
|
|
SystemCallArchitectures=native
|
2026-11-11 23:27:35 +01:00
|
|
|
# No RestrictSUIDSGID=: its filter can't see the mode inside openat2's argument, so it
|
|
|
|
|
# refuses openat2 altogether (ENOSYS), and GNU tar 1.35 enters every folder it unpacks
|
|
|
|
|
# through openat2 without falling back: Flutter's Gradle wrapper failed to unpack (found
|
|
|
|
|
# 12 Nov 2026). NoNewPrivileges= and the empty capability set keep a set-user-ID file
|
|
|
|
|
# from raising anything here, and one the runner made is its own, in its 0700 folder.
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
ProtectKernelTunables=yes
|
|
|
|
|
ProtectKernelModules=yes
|
|
|
|
|
ProtectKernelLogs=yes
|
|
|
|
|
ProtectControlGroups=yes
|
|
|
|
|
ProtectClock=yes
|
|
|
|
|
ProtectHostname=yes
|
|
|
|
|
LockPersonality=yes
|
|
|
|
|
RestrictRealtime=yes
|
|
|
|
|
RestrictNamespaces=yes
|
|
|
|
|
RemoveIPC=yes
|
|
|
|
|
# Netlink stays: Dart lists network interfaces through it.
|
|
|
|
|
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
|
|
|
|
|
# System calls no CI job needs fail with an error rather than killing the job.
|
|
|
|
|
SystemCallFilter=~@clock @cpu-emulation @debug @module @mount @obsolete @raw-io @reboot @swap
|
|
|
|
|
SystemCallErrorNumber=EPERM
|
|
|
|
|
UMask=0077
|
|
|
|
|
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# Never at the instance's expense, which keeps serving through the night: three of the
|
|
|
|
|
# four cores (its servers idle then), at most 5 GB of the 7.2 (the kernel reclaims from CI
|
|
|
|
|
# first above 4.5 GB), and swap beyond, the lowest priority for CPU and disk, and CI's
|
|
|
|
|
# processes the first the kernel kills should the whole machine run out of memory. A
|
|
|
|
|
# cold `dart analyze` of the workspace with its lint plugin needs about 5.4 GB (3.7 GB
|
|
|
|
|
# warm), so it swaps a little rather than being killed: slower, within the window.
|
|
|
|
|
CPUQuota=300%
|
|
|
|
|
MemoryHigh=4.5G
|
|
|
|
|
MemoryMax=5G
|
2026-10-22 19:04:57 +02:00
|
|
|
OOMScoreAdjust=1000
|
ci: check every push on sahaquiel and deploy a green main
Pushes to main were deployed by hand. Now Forgejo Actions runs tool/check.sh on every
push, and a green main deploys learn.halkyon.eu by itself.
The runner (forgejo-runner v13.1.0) runs on sahaquiel as the unprivileged user `runner`,
with jobs directly on the host: containers would take Docker access, which is root on the
production server, and Forgejo warns that host jobs have no isolation from the runner's
user. So its systemd unit (deploy/runner/forgejo-runner.service) provides it: /home and
the Docker socket invisible, Docker's container networks unreachable, no capabilities,
two cores and 3 GB at most, lowest priority. It is registered to this repository only,
with the UUID and token of Forgejo's "Create new runner" (the old `register` is
deprecated); the token reaches it as a systemd credential. The configuration
(deploy/runner/config.yml) was dry-run against v13.1.0, the unit rates 1.8 in
`systemd-analyze security`. CI runs on pushes only: a pull request from a fork would run
its code on the production server.
The deploy job calls deploy/redeploy.sh as `deploy`, through a key that authorized_keys
pins to that script and to connections from sahaquiel itself. The script deploys the
commit CI checked while it is still main's tip, as a fast-forward, only if every new
commit is signed by a key in deploy/allowed-signers (as the deployed checkout lists
them), after dumping the database (the last ten are kept), and fails unless the server
answers afterwards; a failure puts the checkout back. Tested against the local stack:
refusals of a malformed argument, a stale commit, a forged unsigned commit and a
non-fast-forward, a real deploy with its dump, a repeat, and a failed build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 21:49:04 +02:00
|
|
|
TasksMax=2048
|
|
|
|
|
Nice=10
|
|
|
|
|
IOSchedulingClass=best-effort
|
|
|
|
|
IOSchedulingPriority=7
|
|
|
|
|
|
build(ci,deploy)!: the runner at home in a nightly window, deploys by hand (ADR 0039, D6, D13, D16)
The CI runner moves to halkyon.li's host at home and runs from 02:00 to
05:00 alone: a timer starts it, its unit's RuntimeMaxSec ends it, and
night.sh rests the instance's heavy sidecars around it as deploy, while
Caddy, PostgreSQL and the apps' servers keep serving. Jobs pushed during the
day wait in Forgejo's queue.
Its sandbox fits the new host: its folder on the HDD, the rest of /data
hidden, containerd's store out of reach, no private IPv4 network and no
IPv6 at all (the home network's IPv6 addresses are public ones under a
changing prefix, on the internet's own interface), 5 GB of memory with swap
beyond. The Sandbox step checks the tunnel, the home router and IPv6.
BREAKING CHANGE: CI deploys nothing any more: the deploy job goes with
learn.halkyon.eu's server, and halkyon.li deploys by hand with redeploy.sh
once a commit's night's run passed. Jobs run on the runner labelled
balthasar; sahaquiel's runner picks up none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-11-08 19:20:11 +01:00
|
|
|
# No [Install]: never started at boot, only by forgejo-runner.timer.
|