feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.
An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 16:52:37 +02:00
|
|
|
# The Halkyon Learn instance: Caddy in front, the Serverpod server, PostgreSQL, and
|
|
|
|
|
# the dictionaries kept current beside them.
|
feat(deploy): Docker Compose stack for the instance
Caddy, the Serverpod server and PostgreSQL 16 in deploy/. Only Caddy publishes ports; the server and Postgres are reachable only inside the Compose network (Docker bypasses ufw, and the server listens on every interface). Secrets come from deploy/.env as environment variables; there is no production passwords.yaml, and .dockerignore keeps local secrets out of the build context.
The server image builds on Serverpod's template with an Ubuntu 24.04 runtime: complete glibc incl. libgcc_s for the SQLite libraries Serverpod 4 bundles, CA certificates, a non-root user, and exec so SIGTERM reaches the server. The scaffold's dev-only docker-compose.yaml is removed.
Verified locally: image builds, all native libraries resolve, no passwords.yaml in the image, migrations apply at start, Caddy serves 200, port 8080 unreachable from the host, Postgres in UTC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-27 17:18:31 +02:00
|
|
|
#
|
|
|
|
|
# cd deploy
|
|
|
|
|
# cp .env.example .env && chmod 600 .env # then fill in the secrets
|
|
|
|
|
# docker compose up -d --build
|
|
|
|
|
#
|
|
|
|
|
# Only Caddy publishes ports. Docker opens published ports directly in the kernel's
|
|
|
|
|
# packet filter, past ufw, and the Serverpod server listens on every interface, so the
|
|
|
|
|
# server and Postgres publish nothing: they're reachable only inside this project's
|
|
|
|
|
# network, where Caddy finds the server by its service name.
|
|
|
|
|
|
|
|
|
|
name: halkyon-learn
|
|
|
|
|
|
|
|
|
|
# Logs go to the host's journal, whose size and age limits apply (journald.conf).
|
|
|
|
|
x-logging: &logging
|
|
|
|
|
driver: journald
|
|
|
|
|
|
|
|
|
|
services:
|
|
|
|
|
caddy:
|
|
|
|
|
image: caddy:2
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
ports:
|
|
|
|
|
- "80:80"
|
|
|
|
|
- "443:443"
|
|
|
|
|
- "443:443/udp" # HTTP/3
|
|
|
|
|
environment:
|
|
|
|
|
# `learn.halkyon.eu` on sahaquiel (automatic Let's Encrypt certificate);
|
|
|
|
|
# `http://localhost` for the local rehearsal (plain HTTP, no certificate).
|
|
|
|
|
DOMAIN: ${DOMAIN:?set DOMAIN in deploy/.env}
|
|
|
|
|
volumes:
|
|
|
|
|
- ./Caddyfile:/etc/caddy/Caddyfile:ro
|
|
|
|
|
- caddy_data:/data # certificates: must survive restarts (rate limits)
|
|
|
|
|
- caddy_config:/config
|
|
|
|
|
depends_on:
|
|
|
|
|
- server
|
|
|
|
|
logging: *logging
|
|
|
|
|
|
|
|
|
|
server:
|
|
|
|
|
build:
|
|
|
|
|
context: ..
|
|
|
|
|
dockerfile: halkyon_learn_server/Dockerfile
|
|
|
|
|
image: halkyon-learn-server
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
# Applies pending migrations at every start, then serves. One instance only, so
|
ci: check the generated code; deploy: migrate before the switch
Two of Serverpod's own conventions, as its v4 project template and roles have them.
CI now runs `serverpod generate` (CLI pinned to the server's Serverpod version, like
the template's workflow) and fails if the committed generated code differs from its
output, so a model or endpoint changed without regenerating can't reach main. Checked in
a clean clone: today's code passes, a committed stale generated file fails.
redeploy.sh now applies the migrations in a run of their own, with Serverpod's
maintenance role (--role=maintenance --apply-migrations), after the dump and before
switching to the new server. That run exits with 0 only if the migrations worked, and
each migration is one transaction, so a failing one stops the deploy while the old
server keeps serving; before, the new server applied them at start and would have
crash-looped. Tested on the local stack: a deploy through the new step, and a failing
run (wrong database password) that exits with 1 after six attempts. AGENTS.md says
migrations must keep the old server working until the switch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 22:55:24 +02:00
|
|
|
# no two servers ever migrate at once. (On learn.halkyon.eu, deploy/redeploy.sh has
|
|
|
|
|
# applied them in a run of their own before, so nothing is left to do here.)
|
feat(deploy): Docker Compose stack for the instance
Caddy, the Serverpod server and PostgreSQL 16 in deploy/. Only Caddy publishes ports; the server and Postgres are reachable only inside the Compose network (Docker bypasses ufw, and the server listens on every interface). Secrets come from deploy/.env as environment variables; there is no production passwords.yaml, and .dockerignore keeps local secrets out of the build context.
The server image builds on Serverpod's template with an Ubuntu 24.04 runtime: complete glibc incl. libgcc_s for the SQLite libraries Serverpod 4 bundles, CA certificates, a non-root user, and exec so SIGTERM reaches the server. The scaffold's dev-only docker-compose.yaml is removed.
Verified locally: image builds, all native libraries resolve, no passwords.yaml in the image, migrations apply at start, Caddy serves 200, port 8080 unreachable from the host, Postgres in UTC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-27 17:18:31 +02:00
|
|
|
command: ["--apply-migrations"]
|
|
|
|
|
environment:
|
|
|
|
|
SERVERPOD_DATABASE_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in deploy/.env}
|
|
|
|
|
SERVERPOD_PASSWORD_serverSideSessionKeyHashPepper: ${SESSION_KEY_HASH_PEPPER:?set SESSION_KEY_HASH_PEPPER in deploy/.env}
|
feat(server): GitHub sign-in that keeps only the GitHub ID
As decided, a GitHub sign-in keeps nothing but the numeric GitHub ID, so the login is our own, on Serverpod's GitHub helpers (lib/src/auth/github.dart): Serverpod's stock login also stores the name and email in a profile and downloads the avatar, and its endpoint is never exposed. The server exchanges the authorization code with the client secret, reads the ID, and drops the access token.
GitHubUpgradeEndpoint links the GitHub ID to the signed-in anonymous person, grants identified (to their existing sessions too) and removes the anonymous identity, in one transaction: the person, their spaces and their studysets stay the same. GitHubSignInEndpoint gives a session for a person who has the GitHub account already; it's called without a session, since Serverpod issues no session for another person inside a signed-in call, and it never creates people. InstanceEndpoint.info() tells the app the instance's GitHub client ID. Refusals are GitHubSignInRefusedException with a reason: GitHub sign-in off here, the account taken by another person, another GitHub account linked already, an unknown account.
GitHub sign-in is on only where the passwords hold githubClientId and githubClientSecret (empty counts as missing, as Compose passes unset variables on as empty strings); production takes them from GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET in deploy/.env. grantIdentified can now join a caller's transaction. No migration: the GitHub accounts table comes with the auth module.
Tests, refusals first: upgrading when not signed in (checked by switching the guard off: the test fails), both ways without a GitHub app, an account taken by someone else with nothing changed, a second GitHub account, an unknown account with nobody created; then linking keeps the person and reaches their sessions and stores no email, linking twice changes nothing, signing in reaches the right person.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 13:35:51 +02:00
|
|
|
# GitHub sign-in, only with this instance's GitHub OAuth app; empty: off.
|
|
|
|
|
SERVERPOD_PASSWORD_githubClientId: ${GITHUB_CLIENT_ID:-}
|
|
|
|
|
SERVERPOD_PASSWORD_githubClientSecret: ${GITHUB_CLIENT_SECRET:-}
|
2026-09-30 16:19:46 +02:00
|
|
|
# Where the source of what this instance runs is; empty: the project's.
|
|
|
|
|
SOURCE_URL: ${SOURCE_URL:-}
|
Limit how much room people's files take
An anonymous person may keep 100 MB of images and sounds, an identified one
500 MB, and one person more by a file_allowance row the operator adds; all
of them together 15 GB, since every file is in the database and in each dump
the server keeps (decided 2026-10-02). Each comes from the environment
(FILE_LIMIT_*_MB), with these defaults, documented in deploy/.env.example.
The server checks when an upload starts, counting unfinished uploads so they
can't fill the disk unnoticed, and again when it finishes, counting what is
kept, so uploads started side by side can't overshoot; a refusal is a
FileLimitExceededException, and an upload refused as it finishes is deleted.
deletePerson deletes the allowance with the person. Migration: the new
table file_allowance, only added.
The app keeps a refused file waiting on the device and stops that round of
uploads; the Anki import warns when a deck's files may not fit in the room
left (asset.usage()).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 05:57:57 +02:00
|
|
|
# How much room people's files may take, in MiB; empty: the defaults.
|
|
|
|
|
FILE_LIMIT_ANONYMOUS_MB: ${FILE_LIMIT_ANONYMOUS_MB:-}
|
|
|
|
|
FILE_LIMIT_IDENTIFIED_MB: ${FILE_LIMIT_IDENTIFIED_MB:-}
|
|
|
|
|
FILE_LIMIT_INSTANCE_MB: ${FILE_LIMIT_INSTANCE_MB:-}
|
feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.
An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 16:52:37 +02:00
|
|
|
# The dictionary files it loads at start and rereads daily, those that changed
|
|
|
|
|
# (see the server's README): those the dictionaries service keeps current, read
|
|
|
|
|
# only. To supply files yourself instead, remove that service and mount
|
|
|
|
|
# ./dictionaries here (`- ./dictionaries:/app/dictionaries:ro`).
|
feat(server): dictionaries, loaded from files and searched
The server now holds dictionaries for everyone: `dictionary_entry` and
`dictionary_source`, reference data owned by no one, never synced. It
loads the dictionary files in its `dictionaries` folder at every start,
those whose SHA-256 changed: in the background, each source replaced in
one transaction, so searches meanwhile find the old entries; a file it
can't read is logged and left out. All of JMdict loads in under half a
minute. On an instance the folder is deploy/dictionaries, mounted
read-only (compose.yaml); the server's README says how.
DictionaryEndpoint (signed in): the dictionaries with their credits; a
search (exact key, exact loose key, key starting with the query, loose
key starting with it, each in the source's order; katakana as hiragana,
macrons written out), some 150-450 ms over all of JMdict; and the
entries with a label, counted and a page at a time (all JLPT N5 words).
The curator now keeps spellings that aren't quite right apart, as
`looseKeys` (romaji with long vowels typed once): "sushi" finds 寿司
before 数詞 (すうし).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 00:51:27 +02:00
|
|
|
volumes:
|
feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.
An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 16:52:37 +02:00
|
|
|
- dictionaries:/app/dictionaries:ro
|
feat(deploy): Docker Compose stack for the instance
Caddy, the Serverpod server and PostgreSQL 16 in deploy/. Only Caddy publishes ports; the server and Postgres are reachable only inside the Compose network (Docker bypasses ufw, and the server listens on every interface). Secrets come from deploy/.env as environment variables; there is no production passwords.yaml, and .dockerignore keeps local secrets out of the build context.
The server image builds on Serverpod's template with an Ubuntu 24.04 runtime: complete glibc incl. libgcc_s for the SQLite libraries Serverpod 4 bundles, CA certificates, a non-root user, and exec so SIGTERM reaches the server. The scaffold's dev-only docker-compose.yaml is removed.
Verified locally: image builds, all native libraries resolve, no passwords.yaml in the image, migrations apply at start, Caddy serves 200, port 8080 unreachable from the host, Postgres in UTC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-27 17:18:31 +02:00
|
|
|
depends_on:
|
|
|
|
|
postgres:
|
|
|
|
|
condition: service_healthy
|
|
|
|
|
logging: *logging
|
|
|
|
|
|
feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.
An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 16:52:37 +02:00
|
|
|
# Keeps the dictionary files current from EDRDG's files of the day, once they are a
|
|
|
|
|
# week old (EDRDG's licence asks it of servers showing its dictionaries at least
|
|
|
|
|
# monthly; the curator's README): in a container of its own, the curator never in
|
|
|
|
|
# the server's image or process. It only downloads (some 25 MB a week) and writes
|
|
|
|
|
# into the volume; the files are never backed up, as they can be had again.
|
|
|
|
|
dictionaries:
|
|
|
|
|
build:
|
|
|
|
|
context: ..
|
|
|
|
|
dockerfile: halkyon_learn_curator/Dockerfile
|
|
|
|
|
image: halkyon-learn-dictionaries
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
# Docker's stop signal reaches the curator through a small init.
|
|
|
|
|
init: true
|
|
|
|
|
# A conversion takes some 400 MB for a minute a week.
|
|
|
|
|
mem_limit: 1g
|
|
|
|
|
volumes:
|
|
|
|
|
- dictionaries:/dictionaries
|
|
|
|
|
logging: *logging
|
|
|
|
|
|
feat(deploy): Docker Compose stack for the instance
Caddy, the Serverpod server and PostgreSQL 16 in deploy/. Only Caddy publishes ports; the server and Postgres are reachable only inside the Compose network (Docker bypasses ufw, and the server listens on every interface). Secrets come from deploy/.env as environment variables; there is no production passwords.yaml, and .dockerignore keeps local secrets out of the build context.
The server image builds on Serverpod's template with an Ubuntu 24.04 runtime: complete glibc incl. libgcc_s for the SQLite libraries Serverpod 4 bundles, CA certificates, a non-root user, and exec so SIGTERM reaches the server. The scaffold's dev-only docker-compose.yaml is removed.
Verified locally: image builds, all native libraries resolve, no passwords.yaml in the image, migrations apply at start, Caddy serves 200, port 8080 unreachable from the host, Postgres in UTC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-27 17:18:31 +02:00
|
|
|
postgres:
|
|
|
|
|
image: postgres:16
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
# The server's session is UTC anyway (the driver sets it); this makes the
|
|
|
|
|
# database's own default UTC too, for psql, COPY loads and future sidecars.
|
|
|
|
|
command: ["postgres", "-c", "timezone=UTC", "-c", "log_timezone=UTC"]
|
|
|
|
|
environment:
|
|
|
|
|
POSTGRES_USER: halkyon_learn
|
|
|
|
|
POSTGRES_DB: halkyon_learn
|
|
|
|
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in deploy/.env}
|
|
|
|
|
volumes:
|
|
|
|
|
- postgres_data:/var/lib/postgresql/data
|
|
|
|
|
healthcheck:
|
|
|
|
|
test: ["CMD-SHELL", "pg_isready -U halkyon_learn -d halkyon_learn"]
|
|
|
|
|
interval: 5s
|
|
|
|
|
timeout: 5s
|
|
|
|
|
retries: 12
|
|
|
|
|
logging: *logging
|
|
|
|
|
|
|
|
|
|
volumes:
|
|
|
|
|
caddy_data:
|
|
|
|
|
caddy_config:
|
feat(deploy): instances keep their dictionaries current, weekly, by themselves
EDRDG's licence asks of a server showing its dictionaries that it update
them at least monthly. The instance now does it alone: a `dictionaries`
service in compose, built from the curator (halkyon_learn_curator/
Dockerfile, its own unprivileged user 10001, 1 GB at most), runs `curate
update --every 7`, downloading EDRDG's files of the day once its files are
a week old and replacing them in a volume the server mounts read-only.
The curator never enters the server's image or process (decided
2026-10-09: a container of its own, weekly). The server rereads the
folder daily at 04:07 (DictionaryReloadFutureCall), loading what changed,
and loads now run one at a time. redeploy.sh builds both images.
An operator who'd rather supply the files removes the service and mounts
deploy/dictionaries/ into the server, as compose.yaml says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 16:52:37 +02:00
|
|
|
dictionaries:
|
feat(deploy): Docker Compose stack for the instance
Caddy, the Serverpod server and PostgreSQL 16 in deploy/. Only Caddy publishes ports; the server and Postgres are reachable only inside the Compose network (Docker bypasses ufw, and the server listens on every interface). Secrets come from deploy/.env as environment variables; there is no production passwords.yaml, and .dockerignore keeps local secrets out of the build context.
The server image builds on Serverpod's template with an Ubuntu 24.04 runtime: complete glibc incl. libgcc_s for the SQLite libraries Serverpod 4 bundles, CA certificates, a non-root user, and exec so SIGTERM reaches the server. The scaffold's dev-only docker-compose.yaml is removed.
Verified locally: image builds, all native libraries resolve, no passwords.yaml in the image, migrations apply at start, Caddy serves 200, port 8080 unreachable from the host, Postgres in UTC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-27 17:18:31 +02:00
|
|
|
postgres_data:
|