halkyon/deploy/compose.yaml

174 lines
7.6 KiB
YAML
Raw Normal View History

# The Halkyon Learn instance: Caddy in front, the Serverpod server, PostgreSQL, and
# the dictionaries kept current beside them.
#
# cd deploy
# cp .env.example .env && chmod 600 .env # then fill in the secrets
# docker compose up -d --build
#
# Only Caddy publishes ports. Docker opens published ports directly in the kernel's
# packet filter, past ufw, and the Serverpod server listens on every interface, so the
# server and Postgres publish nothing: they're reachable only inside this project's
# networks, where Caddy finds the server by its service name.
#
# Two networks (decided 2026-10-14): `edge`, Caddy's, which the server shares with it,
# and the internal default one, where the server reaches Postgres. Caddy never reaches
# Postgres, nor does a sidecar behind Caddy.
#
# Sidecars (decided 2026-10-14) are each a compose file of their own beside this one,
# switched on in .env (COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml): each adds its
# service and mounts its site into Caddy's sidecars/ folder (sidecars/README.md).
name: halkyon-learn
# Logs go to the host's journal, whose size and age limits apply (journald.conf).
x-logging: &logging
driver: journald
services:
caddy:
image: caddy:2
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp" # HTTP/3
environment:
# `learn.halkyon.eu` on its server (automatic Let's Encrypt certificate);
# `http://localhost` for the local rehearsal (plain HTTP, no certificate).
DOMAIN: ${DOMAIN:?set DOMAIN in deploy/.env}
feat(profile): profile pages on the account domain, and links verified by rel="me" A profile now has a page on the web, at the root of the account domain (https://halkyon.eu/@raisondetredev where the operator set ACCOUNT_DOMAIN, else on the instance's web server), so the address belongs to the identity rather than to Learn and stays when other services share it (ADR 0011). The page shows what the app's profile sheet shows, in English or German, without JavaScript or cookies, asks search engines not to list it, marks every link rel="me" so other sites can verify links to it, and sends a visitor who asked on another host to its own address. Caddy forwards /@… to the web server, and serves nothing else on the account domain; compose now passes ACCOUNT_DOMAIN on, which it never did. A link is verified as Mastodon verifies one: its page links back to the profile's with rel="me". profile.checkLinks fetches each link's page, at most ten times an hour, over HTTPS on port 443 and to public addresses only, connecting to the address it checked and verifying the certificate for the host, with three redirects, 1 MiB and ten seconds at most, and keeps what links back on the profile (verifiedLinks, a column of a table no device syncs). A changed link loses its check, a new handle all of them. The app shows the page's address to copy, checks a link saved by itself and on Check links, and marks verified links in the form and in others' profiles; a check answered after a newer save only adds its checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-15 20:29:30 +02:00
# The account domain whose profile pages this instance serves, where it isn't
# DOMAIN; empty for none.
ACCOUNT_SITE: ${ACCOUNT_DOMAIN:-}
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data # certificates: must survive restarts (rate limits)
- caddy_config:/config
networks:
edge:
# Fixed, so containers behind Caddy can trust it as their proxy (the
# visitor's own address comes in X-Forwarded-For).
ipv4_address: 172.30.80.2
depends_on:
- server
logging: *logging
server:
build:
context: ..
dockerfile: apps/learn/halkyon_learn_server/Dockerfile
image: halkyon-learn-server
restart: unless-stopped
# Applies pending migrations at every start, then serves. One instance only, so
# no two servers ever migrate at once. (On learn.halkyon.eu, deploy/redeploy.sh has
# applied them in a run of their own before, so nothing is left to do here.)
command: ["--apply-migrations"]
environment:
SERVERPOD_DATABASE_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in deploy/.env}
SERVERPOD_PASSWORD_serverSideSessionKeyHashPepper: ${SESSION_KEY_HASH_PEPPER:?set SESSION_KEY_HASH_PEPPER in deploy/.env}
feat(server): GitHub sign-in that keeps only the GitHub ID As decided, a GitHub sign-in keeps nothing but the numeric GitHub ID, so the login is our own, on Serverpod's GitHub helpers (lib/src/auth/github.dart): Serverpod's stock login also stores the name and email in a profile and downloads the avatar, and its endpoint is never exposed. The server exchanges the authorization code with the client secret, reads the ID, and drops the access token. GitHubUpgradeEndpoint links the GitHub ID to the signed-in anonymous person, grants identified (to their existing sessions too) and removes the anonymous identity, in one transaction: the person, their spaces and their studysets stay the same. GitHubSignInEndpoint gives a session for a person who has the GitHub account already; it's called without a session, since Serverpod issues no session for another person inside a signed-in call, and it never creates people. InstanceEndpoint.info() tells the app the instance's GitHub client ID. Refusals are GitHubSignInRefusedException with a reason: GitHub sign-in off here, the account taken by another person, another GitHub account linked already, an unknown account. GitHub sign-in is on only where the passwords hold githubClientId and githubClientSecret (empty counts as missing, as Compose passes unset variables on as empty strings); production takes them from GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET in deploy/.env. grantIdentified can now join a caller's transaction. No migration: the GitHub accounts table comes with the auth module. Tests, refusals first: upgrading when not signed in (checked by switching the guard off: the test fails), both ways without a GitHub app, an account taken by someone else with nothing changed, a second GitHub account, an unknown account with nobody created; then linking keeps the person and reaches their sessions and stores no email, linking twice changes nothing, signing in reaches the right person. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-09-30 13:35:51 +02:00
# GitHub sign-in, only with this instance's GitHub OAuth app; empty: off.
SERVERPOD_PASSWORD_githubClientId: ${GITHUB_CLIENT_ID:-}
SERVERPOD_PASSWORD_githubClientSecret: ${GITHUB_CLIENT_SECRET:-}
# Email sign-in (ADR 0010), only with the pepper and an SMTP service; empty: off.
SERVERPOD_PASSWORD_emailSignInPepper: ${EMAIL_SIGN_IN_PEPPER:-}
SERVERPOD_PASSWORD_emailSmtpHost: ${EMAIL_SMTP_HOST:-}
SERVERPOD_PASSWORD_emailSmtpPort: ${EMAIL_SMTP_PORT:-}
SERVERPOD_PASSWORD_emailSmtpUser: ${EMAIL_SMTP_USER:-}
SERVERPOD_PASSWORD_emailSmtpPassword: ${EMAIL_SMTP_PASSWORD:-}
SERVERPOD_PASSWORD_emailFrom: ${EMAIL_FROM:-}
# Where the source of what this instance runs is; empty: the project's.
SOURCE_URL: ${SOURCE_URL:-}
feat(profile): profile pages on the account domain, and links verified by rel="me" A profile now has a page on the web, at the root of the account domain (https://halkyon.eu/@raisondetredev where the operator set ACCOUNT_DOMAIN, else on the instance's web server), so the address belongs to the identity rather than to Learn and stays when other services share it (ADR 0011). The page shows what the app's profile sheet shows, in English or German, without JavaScript or cookies, asks search engines not to list it, marks every link rel="me" so other sites can verify links to it, and sends a visitor who asked on another host to its own address. Caddy forwards /@… to the web server, and serves nothing else on the account domain; compose now passes ACCOUNT_DOMAIN on, which it never did. A link is verified as Mastodon verifies one: its page links back to the profile's with rel="me". profile.checkLinks fetches each link's page, at most ten times an hour, over HTTPS on port 443 and to public addresses only, connecting to the address it checked and verifying the certificate for the host, with three redirects, 1 MiB and ten seconds at most, and keeps what links back on the profile (verifiedLinks, a column of a table no device syncs). A changed link loses its check, a new handle all of them. The app shows the page's address to copy, checks a link saved by itself and on Check links, and marks verified links in the form and in others' profiles; a check answered after a newer save only adds its checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-15 20:29:30 +02:00
# The domain people's handles are on; empty: DOMAIN's host.
ACCOUNT_DOMAIN: ${ACCOUNT_DOMAIN:-}
# The Jellyfin server suggested to people; empty: jellyfin. on the
# instance's own domain, if one answers there; `none`: none.
JELLYFIN_URL: ${JELLYFIN_URL:-}
# Where people's data and the backups are, which the app shows: `EU`, a
# country (`DE`) or a subdivision (`US-CA`); empty: it says nothing.
DATA_LOCATION: ${DATA_LOCATION:-}
BACKUP_LOCATION: ${BACKUP_LOCATION:-}
# How much room people's files may take, in MiB; empty: the defaults.
FILE_LIMIT_ANONYMOUS_MB: ${FILE_LIMIT_ANONYMOUS_MB:-}
FILE_LIMIT_IDENTIFIED_MB: ${FILE_LIMIT_IDENTIFIED_MB:-}
FILE_LIMIT_INSTANCE_MB: ${FILE_LIMIT_INSTANCE_MB:-}
# The git repositories of published studysets (ADR 0005, rule 3).
REPOSITORIES_DIR: /app/repositories
feat: a dictionary's corrections reach the cards made from it, by offer Every dictionary entry now has a version: 16 hex digits of the SHA-256 of what cards take of it (headword, reading, entry), made by the loader and carried on every hit. A card keeps the version it took (card schema 8). DictionaryEndpoint.changed returns, of the entries named with the versions cards keep (1,000 at most a call; the app asks 500 at a time), those whose version is another now. The card browser says how many cards are behind their dictionary and takes the corrections at a tap (decided 2026-10-09: by offer, never by themselves). Taking one writes the entry as it is now over the revision in which the card last took it, beside whatever came after, so the revisions merge per side as two devices' edits do (ADR 0003): a side only the dictionary changed takes the correction, a side the person changed too is a conflict settled in the composer, whose words now name that case. Cards made before versions are never offered one. The server rereads its dictionary folder hourly now, not daily: the dictionaries service fills its volume after an instance's first start, and an hour with nothing changed costs a hash of each file. Migration 20261009145434785-dictionary-entry-version: adds dictionary_entry.version, text NOT NULL DEFAULT '' (rows loaded before have none and are never reported as corrected), and by hand empties dictionary_source."fileHash", so every source loads once more and its entries get their versions; the schema stays as definition.sql says. The old server goes on working until the switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-09 17:32:00 +02:00
# The dictionary files it loads at start and rereads hourly, those that changed
# (see the server's README): those the dictionaries service keeps current, read
# only. To supply files yourself instead, remove that service and mount
# ./dictionaries here (`- ./dictionaries:/app/dictionaries:ro`).
volumes:
- dictionaries:/app/dictionaries:ro
# Every release of every published studyset: backed up beside the database
# (backup.sh), never deleted with `down -v`.
- repositories:/app/repositories
networks:
- default
- edge
depends_on:
postgres:
condition: service_healthy
logging: *logging
# Keeps the dictionary files current from EDRDG's files of the day, once they are a
# week old (EDRDG's licence asks it of servers showing its dictionaries at least
feat(dictionary): German, Greek, English, Chinese and Korean words A studyset in German, Greek, English, Chinese or Korean finds its dictionary as one in Japanese does (ADR 0008), and its subtitles are cut into words by its language (`WordCutting`): Chinese by its longest words as they stand, the longer words over each cut offered as for Japanese; Korean at its spaces, a word the dictionary lacks as it stands taken without the particle it ends with (집에서 is 집, 아이 stays a child); every other language at its spaces, a word written exactly so first (the noun Essen before the verb essen). A studyset finds the dictionary of its language tag, else of its language alone (zh for zh-Hans). A Wiktionary word shows its genders, principal forms, IPA, romanisation and Hanja; a German or Greek noun of one gender goes onto its card with its article (das Haus, το σπίτι); its tag is its word class, named in full (Noun, Adjective, Preposition), as Wiktionary's adj and adv would read as JMdict's codes. Articles, pronouns, prepositions and conjunctions are left out of a video's words. Chinese words show their pinyin over each character on the Watch screen, carry their HSK level as a tag (HSK-1 … HSK-7-9), and the dictionary adds a level whole; the Watch screen filters by HSK levels for Chinese, JLPT's for Japanese, and by none for a language without. A reading is typed in a session only when it is kana, so pinyin isn't. Words are looked up on the web on Jisho in Japanese, on Wiktionary at the language's section otherwise, and a search's hint speaks of kana and romaji only in Japanese. `DICTIONARY_LANGUAGES` in deploy/.env names the languages an instance keeps (unset: de,el,en,ko,zh; empty: none). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 07:45:15 +02:00
# monthly; the curator's README), and the languages of DICTIONARY_LANGUAGES: Chinese
# from MDBG's CC-CEDICT as often, the rest from Wiktionary's dump whenever kaikki.org
# makes a new one (about monthly, some 3 GB, converted as it downloads): in a
# container of its own, the curator never in the server's image or process. It only
# downloads and writes into the volume; the files are never backed up, as they can be
# had again.
dictionaries:
build:
context: ..
dockerfile: apps/learn/halkyon_learn_curator/Dockerfile
image: halkyon-learn-dictionaries
restart: unless-stopped
# Docker's stop signal reaches the curator through a small init.
init: true
feat(dictionary): German, Greek, English, Chinese and Korean words A studyset in German, Greek, English, Chinese or Korean finds its dictionary as one in Japanese does (ADR 0008), and its subtitles are cut into words by its language (`WordCutting`): Chinese by its longest words as they stand, the longer words over each cut offered as for Japanese; Korean at its spaces, a word the dictionary lacks as it stands taken without the particle it ends with (집에서 is 집, 아이 stays a child); every other language at its spaces, a word written exactly so first (the noun Essen before the verb essen). A studyset finds the dictionary of its language tag, else of its language alone (zh for zh-Hans). A Wiktionary word shows its genders, principal forms, IPA, romanisation and Hanja; a German or Greek noun of one gender goes onto its card with its article (das Haus, το σπίτι); its tag is its word class, named in full (Noun, Adjective, Preposition), as Wiktionary's adj and adv would read as JMdict's codes. Articles, pronouns, prepositions and conjunctions are left out of a video's words. Chinese words show their pinyin over each character on the Watch screen, carry their HSK level as a tag (HSK-1 … HSK-7-9), and the dictionary adds a level whole; the Watch screen filters by HSK levels for Chinese, JLPT's for Japanese, and by none for a language without. A reading is typed in a session only when it is kana, so pinyin isn't. Words are looked up on the web on Jisho in Japanese, on Wiktionary at the language's section otherwise, and a search's hint speaks of kana and romaji only in Japanese. `DICTIONARY_LANGUAGES` in deploy/.env names the languages an instance keeps (unset: de,el,en,ko,zh; empty: none). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 07:45:15 +02:00
# JMdict's conversion takes some 400 MB for a minute a week, Wiktionary's some
feat(curator): characters of Chinese and Korean, the Greek alphabet, Korean words from KRDict For parity with Japanese's KANJIDIC2 and kana: - Unihan (Unicode License v3) makes characters' dictionaries: the 8,105 standard simplified characters (unihan-hans) and the traditional ones in use (unihan-hant), told apart and ranked by CC-CEDICT's words (Unihan calls 这 its own traditional variant; 後 ranks as the 后 of 以后), with pinyin, meanings, strokes, Kangxi radical, HSK 2025 character levels, the 3,500 common ones and Hong Kong's grades; and Korean Hanja (unihan-ko) with their sounds, the 1,800 of education, and their native readings from Wiktionary (물 수), with the 40 Hangul letters as kana are. - The Greek alphabet, the curator's own (CC0): letters with names, transliteration and sound. - KRDict, the Basic Korean Dictionary for learners (CC BY-SA 2.0 KR, downloaded without an account), replaces Wiktionary for Korean's words: learners' levels, pronunciation where it differs from the spelling (같이 [가치]), Hanja, English senses; Wiktionary's update keeps the words' forms (가다's 99) and the Hanja's readings beside it for both. - The update converts KRDict when its page offers another export and the characters weekly, and removes the files of languages no longer kept or of sources replaced. Measured on the real data: Unihan 3 s; KRDict 54,870 entries in 24 s and under 600 MB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 12:08:10 +02:00
# 300 MB for ten minutes a month, KRDict's some 600 MB for half a minute.
mem_limit: 1g
feat(dictionary): German, Greek, English, Chinese and Korean words A studyset in German, Greek, English, Chinese or Korean finds its dictionary as one in Japanese does (ADR 0008), and its subtitles are cut into words by its language (`WordCutting`): Chinese by its longest words as they stand, the longer words over each cut offered as for Japanese; Korean at its spaces, a word the dictionary lacks as it stands taken without the particle it ends with (집에서 is 집, 아이 stays a child); every other language at its spaces, a word written exactly so first (the noun Essen before the verb essen). A studyset finds the dictionary of its language tag, else of its language alone (zh for zh-Hans). A Wiktionary word shows its genders, principal forms, IPA, romanisation and Hanja; a German or Greek noun of one gender goes onto its card with its article (das Haus, το σπίτι); its tag is its word class, named in full (Noun, Adjective, Preposition), as Wiktionary's adj and adv would read as JMdict's codes. Articles, pronouns, prepositions and conjunctions are left out of a video's words. Chinese words show their pinyin over each character on the Watch screen, carry their HSK level as a tag (HSK-1 … HSK-7-9), and the dictionary adds a level whole; the Watch screen filters by HSK levels for Chinese, JLPT's for Japanese, and by none for a language without. A reading is typed in a session only when it is kana, so pinyin isn't. Words are looked up on the web on Jisho in Japanese, on Wiktionary at the language's section otherwise, and a search's hint speaks of kana and romaji only in Japanese. `DICTIONARY_LANGUAGES` in deploy/.env names the languages an instance keeps (unset: de,el,en,ko,zh; empty: none). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-14 07:45:15 +02:00
# Unset: German, Greek, English, Korean and Chinese; empty: none.
command: ["--every", "7", "--languages", "${DICTIONARY_LANGUAGES-de,el,en,ko,zh}"]
volumes:
- dictionaries:/dictionaries
logging: *logging
postgres:
image: postgres:16
restart: unless-stopped
# The server's session is UTC anyway (the driver sets it); this makes the
# database's own default UTC too, for psql, COPY loads and future sidecars.
command: ["postgres", "-c", "timezone=UTC", "-c", "log_timezone=UTC"]
environment:
POSTGRES_USER: halkyon_learn
POSTGRES_DB: halkyon_learn
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in deploy/.env}
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U halkyon_learn -d halkyon_learn"]
interval: 5s
timeout: 5s
retries: 12
logging: *logging
networks:
edge:
name: halkyon-learn-edge
ipam:
config:
- subnet: 172.30.80.0/24
gateway: 172.30.80.1
# Containers without an address of their own (the server) get one from the
# upper half, so the fixed ones below it (Caddy .2, sidecars from .10) are
# never taken: the server started first once and took Caddy's (2026-10-14).
# (With a range alone, Docker would move the gateway to its first address.)
ip_range: 172.30.80.128/25
volumes:
caddy_data:
caddy_config:
dictionaries:
postgres_data:
repositories: