build(train): Train's server and BRouter deployable beside the instance (ADR 0030, D46)
deploy/compose.train.yaml adds Train's server at train.halkyon.eu, a member of Learn's register, with its database halkyon_train in the instance's PostgreSQL, and BRouter beside it alone (v1.7.10, checked by SHA-256, no log), its segments of the operator's region kept current monthly. Nothing changes until deploy/.env names the compose file: redeploy.sh and backup.sh take Train in only then, and the operator's steps are in sidecars/README.md. restic now forgets snapshots grouped by host alone, so adding Train's dump leaves no group behind for good. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
781576726a
commit
a8f7c26e69
13 changed files with 483 additions and 36 deletions
|
|
@ -3,7 +3,7 @@
|
||||||
What holds for every app and package of this repository is in the [root AGENTS.md](../../AGENTS.md); it applies here in full, and Learn's patterns (`apps/learn/AGENTS.md`) wherever Train needs the same thing.
|
What holds for every app and package of this repository is in the [root AGENTS.md](../../AGENTS.md); it applies here in full, and Learn's patterns (`apps/learn/AGENTS.md`) wherever Train needs the same thing.
|
||||||
|
|
||||||
- **A shell on the shared packages**, Todo's copied and renamed (decided 2026-10-28, ADR 0030, D12; what Train is to do: ADR 0030): signing in, the instances, the account and the look work, and since step 5a (28 Oct 2026) a person's workouts are kept and synced with `serverpod_offline_sync`, every row sealed (ADR 0030, D7, D14, D15), and on Android a workout is recorded (5b: ADR 0030, D13, D16, D18 to D21), and since 5c home lists them and each has its page with its map (D22 to D24). Packages: `halkyon_train_server`, `halkyon_train_client` (generated, never edited), `halkyon_train_flutter`; app ID `app.halkyon.train` on every platform; no web build. Licences as Learn's (server SSPL, client and app AGPL; `LICENSE.md`).
|
- **A shell on the shared packages**, Todo's copied and renamed (decided 2026-10-28, ADR 0030, D12; what Train is to do: ADR 0030): signing in, the instances, the account and the look work, and since step 5a (28 Oct 2026) a person's workouts are kept and synced with `serverpod_offline_sync`, every row sealed (ADR 0030, D7, D14, D15), and on Android a workout is recorded (5b: ADR 0030, D13, D16, D18 to D21), and since 5c home lists them and each has its page with its map (D22 to D24). Packages: `halkyon_train_server`, `halkyon_train_client` (generated, never edited), `halkyon_train_flutter`; app ID `app.halkyon.train` on every platform; no web build. Licences as Learn's (server SSPL, client and app AGPL; `LICENSE.md`).
|
||||||
- **The server** includes `halkyon_base` (ADR 0014) and configures it in `server.dart` (`trainBase()`, `lib/src/train_base.dart`): its basics say it serves `train` (never change it once apps use it), sign-in mails name Halkyon Train, the source is this repository, `minimumAppBuild` 1, and `deleteTrainData` (`lib/src/train_data.dart`) as its `deletePersonData` hook, which deletes the person's sync spaces and so, by the sync engine's cascades, every row of theirs (`test/integration/workout/person_deletion_test.dart`). Authentication through the module's builders: a password, SimpleLogin on the register, email codes proving an address, server-side sessions; no anonymous accounts (ADR 0020, D25), nor GitHub (D36). **Accounts** (ADR 0016): Train holds its own register when it runs alone, and is a member of the instance's register (Learn's server on learn.halkyon.eu) where the operator sets `ACCOUNT_REGISTER_URL` (`trainBase(accounts: AccountsMode.fromEnvironment())`) with the instance's shared `INSTANCE_SECRET` (`instanceSecret`); then sign-in, handles and profiles are the register's, the email settings are the register's alone, and an identified person's deletion goes to the register first. `addInstanceRoutes` serves the calls between the services on its web server, and the server doesn't start without the secret where it needs it. `schedulePruning` (`lib/src/pruning.dart`) keeps the module's pruning scheduled, one entry each however often the server starts (and cancels the one under the pruning's name before ADR 0020, D25): the anonymous accounts left from before daily, sign-up attempts and codes hourly, as the register the deletions its members haven't confirmed hourly, as a member the people the register no longer knows daily (each a no-op in the other mode). Session logs are never persisted, in any run mode. **The Loop's firmware relay** (ADR 0030, D10, D40, `lib/src/firmware/`): the `firmware` endpoint asks Polar's firmware service in the person's place and hands on the file, kept in memory a day per hardware and version (`FirmwareRelay`); on by default, off where the operator sets `LOOP_FIRMWARE_RELAY=off`, the app then offering the direct way alone; tested on a fake Polar (`test/integration/firmware/`). **Routes** (ADR 0030, D11, D44, `lib/src/routing/`): the `routing` endpoint plans a route through the instance's BRouter (`BROUTER_URL`; none: `offered` is false and plans are refused) for the people signed in, at most 50 waypoints and 30 plans a person a minute, nothing kept or logged; BRouter's GeoJSON read in `brouter.dart` (points, elevations, climbs above 3 m, turn hints, stretches by surface), tested on its format (`test/unit/routing/`) and on a fake BRouter (`test/integration/routing/`). Train's running profile is `apps/train/brouter/halkyon-run.brf`, which the sidecar's profiles hold beside BRouter's own. Migrations (2026-10-28): an initial one with Serverpod's tables, its auth modules' and the module's as they stand (Todo's grew one by one with the module; Train starts where they ended), then `workouts`, purely additive: the sync engine's tables and Train's first three, then `profile`, `days` (Train's `day`, `day_series` and `night`) and `plans` (`plan`, D42), each purely additive. **Synced models** (`database: sync`, the module `offline_sync`, `experimental_features: databaseSync` in `config/generator.yaml`): `content_key` (the person's key sealed to the account), `workout` and `workout_series` (a workout's series in chunks of at most ten minutes, deleted with it), `profile` (the body's figures, one row a person), `day`, `day_series` (a day's 24/7 heart rate in chunks of three hours, deleted with it) and `night` (one row a person and date each, ADR 0030, D37) and `plan` (an interval plan, D42), each a sealed `payload` with its `keyId` and `nonce`, nothing else readable. Tests (`test/integration/train_server_test.dart`) call the module's endpoints through the server's own dispatch (`halkyon_base.<endpoint>`), since the generated test tools expose the project's endpoints alone.
|
- **The server** includes `halkyon_base` (ADR 0014) and configures it in `server.dart` (`trainBase()`, `lib/src/train_base.dart`): its basics say it serves `train` (never change it once apps use it), sign-in mails name Halkyon Train, the source is this repository, `minimumAppBuild` 1, and `deleteTrainData` (`lib/src/train_data.dart`) as its `deletePersonData` hook, which deletes the person's sync spaces and so, by the sync engine's cascades, every row of theirs (`test/integration/workout/person_deletion_test.dart`). Authentication through the module's builders: a password, SimpleLogin on the register, email codes proving an address, server-side sessions; no anonymous accounts (ADR 0020, D25), nor GitHub (D36). **Accounts** (ADR 0016): Train holds its own register when it runs alone, and is a member of the instance's register (Learn's server on learn.halkyon.eu) where the operator sets `ACCOUNT_REGISTER_URL` (`trainBase(accounts: AccountsMode.fromEnvironment())`) with the instance's shared `INSTANCE_SECRET` (`instanceSecret`); then sign-in, handles and profiles are the register's, the email settings are the register's alone, and an identified person's deletion goes to the register first. `addInstanceRoutes` serves the calls between the services on its web server, and the server doesn't start without the secret where it needs it. `schedulePruning` (`lib/src/pruning.dart`) keeps the module's pruning scheduled, one entry each however often the server starts (and cancels the one under the pruning's name before ADR 0020, D25): the anonymous accounts left from before daily, sign-up attempts and codes hourly, as the register the deletions its members haven't confirmed hourly, as a member the people the register no longer knows daily (each a no-op in the other mode). Session logs are never persisted, in any run mode. **The Loop's firmware relay** (ADR 0030, D10, D40, `lib/src/firmware/`): the `firmware` endpoint asks Polar's firmware service in the person's place and hands on the file, kept in memory a day per hardware and version (`FirmwareRelay`); on by default, off where the operator sets `LOOP_FIRMWARE_RELAY=off`, the app then offering the direct way alone; tested on a fake Polar (`test/integration/firmware/`). **Routes** (ADR 0030, D11, D44, `lib/src/routing/`): the `routing` endpoint plans a route through the instance's BRouter (`BROUTER_URL`; none: `offered` is false and plans are refused) for the people signed in, at most 50 waypoints and 30 plans a person a minute, nothing kept or logged; BRouter's GeoJSON read in `brouter.dart` (points, elevations, climbs above 3 m, turn hints, stretches by surface), tested on its format (`test/unit/routing/`) and on a fake BRouter (`test/integration/routing/`). Train's running profile is `apps/train/brouter/halkyon-run.brf`, which the sidecar's profiles hold beside BRouter's own. **Deployed** (D46) beside learn.halkyon.eu's server once `deploy/.env` names `deploy/compose.train.yaml`: at train.halkyon.eu (`config/production.yaml`, the image `Dockerfile` as Learn's), a member of Learn's register, its database `halkyon_train` in the instance's PostgreSQL, BRouter's image `deploy/brouter/` (its segments kept current by `segments.sh`); `redeploy.sh` and `backup.sh` take Train's server and database in only where it is switched on, and the operator's steps are `deploy/sidecars/README.md`, *Train*. Migrations (2026-10-28): an initial one with Serverpod's tables, its auth modules' and the module's as they stand (Todo's grew one by one with the module; Train starts where they ended), then `workouts`, purely additive: the sync engine's tables and Train's first three, then `profile`, `days` (Train's `day`, `day_series` and `night`) and `plans` (`plan`, D42), each purely additive. **Synced models** (`database: sync`, the module `offline_sync`, `experimental_features: databaseSync` in `config/generator.yaml`): `content_key` (the person's key sealed to the account), `workout` and `workout_series` (a workout's series in chunks of at most ten minutes, deleted with it), `profile` (the body's figures, one row a person), `day`, `day_series` (a day's 24/7 heart rate in chunks of three hours, deleted with it) and `night` (one row a person and date each, ADR 0030, D37) and `plan` (an interval plan, D42), each a sealed `payload` with its `keyId` and `nonce`, nothing else readable. Tests (`test/integration/train_server_test.dart`) call the module's endpoints through the server's own dispatch (`halkyon_base.<endpoint>`), since the generated test tools expose the project's endpoints alone.
|
||||||
- **The app** is halkyon_app's `InstanceGate` and `SessionGate` with Train's frame (`trainFrame`: "Train" under "Halkyon", its icon, `train.example.org`, its own sentence) and `ServerpodInstanceChecker(app: 'train')`; `TrainDependencies` (`AppDependencies<TrainPerson>`) on one instance, `TrainPerson` per person: their own sync database in their folder (`sync.sqlite`, halkyon_app's `PersonFolders`, one person's at a time), halkyon_app's `SyncService` behind `AppCompatibility` (the module's basics' `minimumAppBuild`, against the build `package_info_plus` reads), their key (`TrainKeys`, `lib/src/core/privacy/`: one key per person, IDs `train1.<ms>.<nonce>`, the oldest-made writing, kept in `content_key` and opened only in Train's form, ADR 0030, D15) and their workouts (`WorkoutRepository`, `lib/src/feature/workout/`: halkyon_app's `PayloadEnvelope` seals a workout for its row and a series chunk for its row and its workout; payloads are versioned JSON, a series' columns delta-coded, gzipped before sealing; a deletion blanks every payload first); the gateways' tests run on a sync database opened without a server, the repository's on memory gateways and real libsodium; a person's app whose home lists their workouts (`WorkoutsCubit`, with `TrainPerson.deletions`, halkyon_app's `DelayedDeletions`) and opens each one's page (`WorkoutCubit`: `analyse` in `workout_analysis.dart` reads its series into the track's pieces, splits, elevation and charts, drawn by `workout_chart.dart`; the map with flutter_map on the instance's source, `MapSourceRepository` asking `map.source`, its tiles from `TrainPerson.mapTiles`, vector_map_tiles in the app and none in widget tests), and the account sheet: halkyon_app's `AccountTiles` (whose account on which instance, Switch account, Sign out, Delete account, which asks first) in Train's words, and halkyon_app's About (`showAppAbout`: the source of what the instance runs, from `TrainDependencies.sourceCodeUrl`, and the licence list). Texts through gen-l10n in British English (the template), German (informal du), an empty `en_US` and `en_GB` (`test/l10n/arb_files_test.dart` keeps them in step). The look follows the system's theme mode and language until Train has settings. Countries in where an instance keeps its data are named by halkyon_app's CLDR names (`AssetCountryNames`), its language picked by halkyon_app's `resolveAppLocale`. Widget tests start Train's whole tree with halkyon_app's fakes (`test/app/train_app_test.dart`).
|
- **The app** is halkyon_app's `InstanceGate` and `SessionGate` with Train's frame (`trainFrame`: "Train" under "Halkyon", its icon, `train.example.org`, its own sentence) and `ServerpodInstanceChecker(app: 'train')`; `TrainDependencies` (`AppDependencies<TrainPerson>`) on one instance, `TrainPerson` per person: their own sync database in their folder (`sync.sqlite`, halkyon_app's `PersonFolders`, one person's at a time), halkyon_app's `SyncService` behind `AppCompatibility` (the module's basics' `minimumAppBuild`, against the build `package_info_plus` reads), their key (`TrainKeys`, `lib/src/core/privacy/`: one key per person, IDs `train1.<ms>.<nonce>`, the oldest-made writing, kept in `content_key` and opened only in Train's form, ADR 0030, D15) and their workouts (`WorkoutRepository`, `lib/src/feature/workout/`: halkyon_app's `PayloadEnvelope` seals a workout for its row and a series chunk for its row and its workout; payloads are versioned JSON, a series' columns delta-coded, gzipped before sealing; a deletion blanks every payload first); the gateways' tests run on a sync database opened without a server, the repository's on memory gateways and real libsodium; a person's app whose home lists their workouts (`WorkoutsCubit`, with `TrainPerson.deletions`, halkyon_app's `DelayedDeletions`) and opens each one's page (`WorkoutCubit`: `analyse` in `workout_analysis.dart` reads its series into the track's pieces, splits, elevation and charts, drawn by `workout_chart.dart`; the map with flutter_map on the instance's source, `MapSourceRepository` asking `map.source`, its tiles from `TrainPerson.mapTiles`, vector_map_tiles in the app and none in widget tests), and the account sheet: halkyon_app's `AccountTiles` (whose account on which instance, Switch account, Sign out, Delete account, which asks first) in Train's words, and halkyon_app's About (`showAppAbout`: the source of what the instance runs, from `TrainDependencies.sourceCodeUrl`, and the licence list). Texts through gen-l10n in British English (the template), German (informal du), an empty `en_US` and `en_GB` (`test/l10n/arb_files_test.dart` keeps them in step). The look follows the system's theme mode and language until Train has settings. Countries in where an instance keeps its data are named by halkyon_app's CLDR names (`AssetCountryNames`), its language picked by halkyon_app's `resolveAppLocale`. Widget tests start Train's whole tree with halkyon_app's fakes (`test/app/train_app_test.dart`).
|
||||||
- **Recording** (ADR 0030, D13, D16, D18 to D21, D26 to D28) is Android's alone for now (iOS waits for a Mac, D25) (`TrainPerson.recording` null elsewhere): `RecorderService` (Kotlin, `android/app/src/main/kotlin/app/halkyon/train/`), a foreground service of the types `location` and `health` started from the open app, takes fixes from LocationManager's GPS (no Play services), steps from the step detector the heart rate from the strap and a row from the rower the person chose (ADR 0030, D26 to D28: `NotifyLink.kt` on Android's own Bluetooth API, no package, `HeartRateMeasurement.kt` parsing 0x2A37 and `RowerData.kt` FTMS's Rower Data 0x2AD1; `DeviceScanner.kt` finds them for the sheets over `…/recorder/straps` and `…/recorder/rowers`; each remembered in the person's `strap.json` and `rower.json`, `DeviceStore`), decides auto-pause (`AutoPause.kt`) and appends each sample as a line to the person's `recording.txt` (format in `recording_file.dart`), flushed every 5 s; `MainActivity` asks the permissions (never background location) and bridges `app.halkyon.train/recorder` (methods) and `…/recorder/lines` (the file so far, then each line). The app (`lib/src/feature/recording/`) follows those lines with `RecordingBloc`, works out what the screen and the notification show (`liveOf`, the saved totals' functions), and saves a recording as a workout and its series (`recordedWorkout`) or throws it away; one cut off, or stopped and not saved, is found as the person's app opens. Tests: the Dart side with `test/support/fake_recorder_platform.dart`; the Kotlin auto-pause and the strap's and the rower's parsers on the JVM, `JAVA_HOME=<Android Studio's jbr> ./gradlew :app:testDebugUnitTest` in `android/` after a first `flutter build apk` (which writes `gradlew`), not part of `tool/check.dart`, as CI's runner has no Android SDK.
|
- **Recording** (ADR 0030, D13, D16, D18 to D21, D26 to D28) is Android's alone for now (iOS waits for a Mac, D25) (`TrainPerson.recording` null elsewhere): `RecorderService` (Kotlin, `android/app/src/main/kotlin/app/halkyon/train/`), a foreground service of the types `location` and `health` started from the open app, takes fixes from LocationManager's GPS (no Play services), steps from the step detector the heart rate from the strap and a row from the rower the person chose (ADR 0030, D26 to D28: `NotifyLink.kt` on Android's own Bluetooth API, no package, `HeartRateMeasurement.kt` parsing 0x2A37 and `RowerData.kt` FTMS's Rower Data 0x2AD1; `DeviceScanner.kt` finds them for the sheets over `…/recorder/straps` and `…/recorder/rowers`; each remembered in the person's `strap.json` and `rower.json`, `DeviceStore`), decides auto-pause (`AutoPause.kt`) and appends each sample as a line to the person's `recording.txt` (format in `recording_file.dart`), flushed every 5 s; `MainActivity` asks the permissions (never background location) and bridges `app.halkyon.train/recorder` (methods) and `…/recorder/lines` (the file so far, then each line). The app (`lib/src/feature/recording/`) follows those lines with `RecordingBloc`, works out what the screen and the notification show (`liveOf`, the saved totals' functions), and saves a recording as a workout and its series (`recordedWorkout`) or throws it away; one cut off, or stopped and not saved, is found as the person's app opens. Tests: the Dart side with `test/support/fake_recorder_platform.dart`; the Kotlin auto-pause and the strap's and the rower's parsers on the JVM, `JAVA_HOME=<Android Studio's jbr> ./gradlew :app:testDebugUnitTest` in `android/` after a first `flutter build apk` (which writes `gradlew`), not part of `tool/check.dart`, as CI's runner has no Android SDK.
|
||||||
- **Health Connect** (ADR 0030, D30, D31) is Android's (`TrainPerson.health` null elsewhere): `HealthConnectBridge.kt` on AndroidX's `connect-client` 1.1.0 over `app.halkyon.train/health` (its records' mapping in `HealthRecords.kt`, tested on the JVM; the rationale's links from Health Connect in the manifest, `…/health/rationale` to the app), `HealthPlatform` in Dart (`lib/src/feature/health/`), `HealthConnect` per person: with writing on, every workout written as `healthRecordsOf` makes it (client IDs from the workout's ID, a version counted up as it changes, a fingerprint deciding whether it did), written again or deleted as the workouts change and as the app opens, what was written kept in the person's `health_connect.json` (`HealthStore`, never synced); with reading on, the sessions of the apps the person turned on brought in as workouts through the changes token (`importedWorkout` in `health_import.dart`, the workout's `origin` in its payload; a double of one Train has brings at most its heart rate, `isDouble`; one deleted in Train is remembered so it stays out; a track only when asked for on its page, `bringInTrack`), as the app opens and comes to the front (`HealthConnectListener`, which opens the rationale too); the screen from the account sheet (`HealthScreen`, `HealthCubit`). Tests on `test/support/fake_health_platform.dart`. Train needs Android 8 (API 26) for it.
|
- **Health Connect** (ADR 0030, D30, D31) is Android's (`TrainPerson.health` null elsewhere): `HealthConnectBridge.kt` on AndroidX's `connect-client` 1.1.0 over `app.halkyon.train/health` (its records' mapping in `HealthRecords.kt`, tested on the JVM; the rationale's links from Health Connect in the manifest, `…/health/rationale` to the app), `HealthPlatform` in Dart (`lib/src/feature/health/`), `HealthConnect` per person: with writing on, every workout written as `healthRecordsOf` makes it (client IDs from the workout's ID, a version counted up as it changes, a fingerprint deciding whether it did), written again or deleted as the workouts change and as the app opens, what was written kept in the person's `health_connect.json` (`HealthStore`, never synced); with reading on, the sessions of the apps the person turned on brought in as workouts through the changes token (`importedWorkout` in `health_import.dart`, the workout's `origin` in its payload; a double of one Train has brings at most its heart rate, `isDouble`; one deleted in Train is remembered so it stays out; a track only when asked for on its page, `bringInTrack`), as the app opens and comes to the front (`HealthConnectListener`, which opens the rationale too); the screen from the account sheet (`HealthScreen`, `HealthCubit`). Tests on `test/support/fake_health_platform.dart`. Train needs Android 8 (API 26) for it.
|
||||||
|
|
|
||||||
60
apps/train/halkyon_train_server/Dockerfile
Normal file
60
apps/train/halkyon_train_server/Dockerfile
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
# Train's server image (ADR 0030, D46). Built from the repository root (the build
|
||||||
|
# context), because the workspace's root pubspec and lockfile live there:
|
||||||
|
#
|
||||||
|
# docker build -f apps/train/halkyon_train_server/Dockerfile .
|
||||||
|
#
|
||||||
|
# Learn's server image (apps/learn/halkyon_learn_server/Dockerfile) for Train: the
|
||||||
|
# same Dart, the same Ubuntu runtime and start command; no git, no repositories
|
||||||
|
# folder, no data of its own.
|
||||||
|
|
||||||
|
# ── Build stage: compile the server to a native executable ────────────────────────
|
||||||
|
FROM dart:3.13.4 AS build
|
||||||
|
WORKDIR /app
|
||||||
|
COPY pubspec.yaml pubspec.lock ./
|
||||||
|
COPY tool/workspace_only.sh tool/
|
||||||
|
COPY apps/train/halkyon_train_server apps/train/halkyon_train_server
|
||||||
|
COPY packages/halkyon_base_server packages/halkyon_base_server
|
||||||
|
COPY packages/halkyon_base_client packages/halkyon_base_client
|
||||||
|
COPY packages/halkyon_core packages/halkyon_core
|
||||||
|
COPY packages/halkyon_llm packages/halkyon_llm
|
||||||
|
COPY packages/halkyon_oidc_server packages/halkyon_oidc_server
|
||||||
|
COPY packages/halkyon_oidc_client packages/halkyon_oidc_client
|
||||||
|
|
||||||
|
# The workspace root with only the server and the modules it builds on, as Learn's
|
||||||
|
# image has it (the reasons there).
|
||||||
|
RUN sh tool/workspace_only.sh apps/train/halkyon_train_server packages/halkyon_base_server packages/halkyon_base_client packages/halkyon_core packages/halkyon_llm packages/halkyon_oidc_server packages/halkyon_oidc_client
|
||||||
|
RUN dart pub get
|
||||||
|
WORKDIR /app/apps/train/halkyon_train_server
|
||||||
|
RUN dart build cli --target bin/main.dart --output build
|
||||||
|
RUN mv build/bundle/bin/main build/bundle/bin/server
|
||||||
|
RUN mkdir -p config migrations
|
||||||
|
|
||||||
|
# ── Runtime stage ─────────────────────────────────────────────────────────────────
|
||||||
|
# Ubuntu 24.04 as Learn's; CA certificates for outgoing TLS (Polar's firmware
|
||||||
|
# service, D40).
|
||||||
|
FROM ubuntu:24.04
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
ENV runmode=production
|
||||||
|
ENV serverid=default
|
||||||
|
ENV logging=normal
|
||||||
|
ENV role=monolith
|
||||||
|
|
||||||
|
COPY --from=build /app/apps/train/halkyon_train_server/build/bundle/ ./
|
||||||
|
COPY --from=build /app/apps/train/halkyon_train_server/config/ config/
|
||||||
|
COPY --from=build /app/apps/train/halkyon_train_server/migrations/ migrations/
|
||||||
|
COPY --from=build /app/apps/train/halkyon_train_server/lib/src/generated/protocol.yaml lib/src/generated/protocol.yaml
|
||||||
|
|
||||||
|
# No root rights needed: it listens on 8080 and 8082 and writes nothing.
|
||||||
|
RUN useradd --system --no-create-home --shell /usr/sbin/nologin app
|
||||||
|
USER app
|
||||||
|
|
||||||
|
# The API server, and the web server the register tells of accounts gone (ADR
|
||||||
|
# 0016), reached on the instance's own network alone.
|
||||||
|
EXPOSE 8080 8082
|
||||||
|
|
||||||
|
ENTRYPOINT ["/bin/sh", "-c", "exec ./bin/server --mode=$runmode --server-id=$serverid --logging=$logging --role=$role \"$@\"", "--"]
|
||||||
|
CMD []
|
||||||
|
|
@ -1,27 +1,20 @@
|
||||||
# This is the configuration file for your production environment.
|
# Train's server on learn.halkyon.eu's server, at train.halkyon.eu (ADR 0030, D46),
|
||||||
# Typically, you will want to route the traffic through a load balancer
|
# behind the instance's Caddy (deploy/compose.train.yaml): its API and its web server
|
||||||
# which adds SSL security through https. If you use Serverpod's standard
|
# on the one domain, as Learn's; its database in the instance's PostgreSQL.
|
||||||
# Terraform scripts to deploy your server, all you need to change in
|
|
||||||
# this file is the examplepod.com domain name.
|
|
||||||
|
|
||||||
# Configuration for the main API server.
|
# Configuration for the main API server.
|
||||||
apiServer:
|
apiServer:
|
||||||
port: 8080
|
port: 8080
|
||||||
publicHost: api.examplepod.com
|
publicHost: train.halkyon.eu
|
||||||
publicPort: 443
|
publicPort: 443
|
||||||
publicScheme: https
|
publicScheme: https
|
||||||
|
|
||||||
# Configuration for the Insights server.
|
# No insightsServer, as Learn's: without the block Serverpod doesn't start Insights.
|
||||||
insightsServer:
|
|
||||||
port: 8081
|
|
||||||
publicHost: insights.examplepod.com
|
|
||||||
publicPort: 443
|
|
||||||
publicScheme: https
|
|
||||||
|
|
||||||
# Configuration for the web server.
|
# Configuration for the web server.
|
||||||
webServer:
|
webServer:
|
||||||
port: 8082
|
port: 8082
|
||||||
publicHost: app.examplepod.com
|
publicHost: train.halkyon.eu
|
||||||
publicPort: 443
|
publicPort: 443
|
||||||
publicScheme: https
|
publicScheme: https
|
||||||
|
|
||||||
|
|
@ -30,11 +23,12 @@ webServer:
|
||||||
# If you are connecting on a public IP (e.g. on AWS or Google Cloud Run), you
|
# If you are connecting on a public IP (e.g. on AWS or Google Cloud Run), you
|
||||||
# connect on the public IP of the database e.g. database.examplepod.com.
|
# connect on the public IP of the database e.g. database.examplepod.com.
|
||||||
database:
|
database:
|
||||||
host: database.private-production.examplepod.com
|
host: postgres
|
||||||
port: 5432
|
port: 5432
|
||||||
name: serverpod
|
name: halkyon_train
|
||||||
user: postgres
|
user: halkyon_train
|
||||||
requireSsl: true
|
# On the instance's own network, never leaving the server.
|
||||||
|
requireSsl: false
|
||||||
#isUnixSocket: true # defaults to false
|
#isUnixSocket: true # defaults to false
|
||||||
#maxConnectionCount: 10 # Defaults to 10, a negative or null value removes the limit
|
#maxConnectionCount: 10 # Defaults to 10, a negative or null value removes the limit
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -37,13 +37,35 @@ DOMAIN=http://localhost
|
||||||
|
|
||||||
# Sidecars switched on beside the instance (sidecars/README.md), each by its compose file
|
# Sidecars switched on beside the instance (sidecars/README.md), each by its compose file
|
||||||
# after the instance's own: compose.jellyfin.yaml, a Jellyfin people watch and listen to
|
# after the instance's own: compose.jellyfin.yaml, a Jellyfin people watch and listen to
|
||||||
# learn from. Unset: the instance alone.
|
# learn from; compose.languagetool.yaml, spelling and grammar checked; compose.train.yaml,
|
||||||
|
# Halkyon Train's server with its BRouter. Unset: the instance alone.
|
||||||
#COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml
|
#COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml
|
||||||
|
|
||||||
# With the Jellyfin sidecar: the domain it answers at, which the instance then suggests
|
# With the Jellyfin sidecar: the domain it answers at, which the instance then suggests
|
||||||
# to its people (the app's Jellyfin sign-in fills it in).
|
# to its people (the app's Jellyfin sign-in fills it in).
|
||||||
#JELLYFIN_DOMAIN=jellyfin.example.org
|
#JELLYFIN_DOMAIN=jellyfin.example.org
|
||||||
|
|
||||||
|
# With Halkyon Train beside the instance (compose.train.yaml, sidecars/README.md,
|
||||||
|
# Train): the domain its apps reach it at, its database's password, its own session
|
||||||
|
# pepper (never change it once people sign in), INSTANCE_SECRET above set, and the
|
||||||
|
# region of BRouter's segments, `west,south,east,north` in whole degrees, multiples of
|
||||||
|
# 5 (empty: Europe's, -25,35,40,70, about 3.3 GB). Generate each secret:
|
||||||
|
# openssl rand -hex 32. Its server is told of accounts gone without INSTANCE_MEMBERS;
|
||||||
|
# setting it, name Train's too (`http://train-server:8082/`).
|
||||||
|
#TRAIN_DOMAIN=train.example.org
|
||||||
|
#TRAIN_POSTGRES_PASSWORD=
|
||||||
|
#TRAIN_SESSION_KEY_HASH_PEPPER=
|
||||||
|
#BROUTER_REGION=
|
||||||
|
# The Loop's firmware through the instance (Train's ADR 0030, D40): `off` to stop
|
||||||
|
# relaying it; empty: on.
|
||||||
|
#LOOP_FIRMWARE_RELAY=
|
||||||
|
# Train's map (ADR 0030, D22): a MapLibre style over HTTPS, with its name, credit and
|
||||||
|
# page; empty: OpenFreeMap's.
|
||||||
|
#MAP_STYLE_URL=
|
||||||
|
#MAP_SOURCE_NAME=
|
||||||
|
#MAP_ATTRIBUTION=
|
||||||
|
#MAP_ABOUT_URL=
|
||||||
|
|
||||||
# Postgres password, shared by the postgres and server services.
|
# Postgres password, shared by the postgres and server services.
|
||||||
# Generate: openssl rand -base64 32
|
# Generate: openssl rand -base64 32
|
||||||
POSTGRES_PASSWORD=
|
POSTGRES_PASSWORD=
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,8 @@
|
||||||
# how it was set up and how to restore.
|
# how it was set up and how to restore.
|
||||||
#
|
#
|
||||||
# Everything people keep is in the database, their files too (Serverpod's database
|
# Everything people keep is in the database, their files too (Serverpod's database
|
||||||
# cloud storage), but for the releases of published studysets, which are git
|
# cloud storage), and in Train's where .env switches Train on (compose.train.yaml,
|
||||||
|
# Halkyon Train's ADR 0030, D46), but for the releases of published studysets, which are git
|
||||||
# repositories in the server's volume: a dump and a bundle of each repository cover
|
# repositories in the server's volume: a dump and a bundle of each repository cover
|
||||||
# the instance; the secrets in deploy/.env are kept in Proton Pass instead. restic stores the dump encrypted in a Hetzner Storage Box
|
# the instance; the secrets in deploy/.env are kept in Proton Pass instead. restic stores the dump encrypted in a Hetzner Storage Box
|
||||||
# and keeps 24 hourly, 7 daily, 4 weekly and 3 monthly snapshots (decided 2026-10-03).
|
# and keeps 24 hourly, 7 daily, 4 weekly and 3 monthly snapshots (decided 2026-10-03).
|
||||||
|
|
@ -35,6 +36,11 @@ cd "$(dirname "$0")"
|
||||||
# the last hour and stores it only once; restic compresses what it stores.
|
# the last hour and stores it only once; restic compresses what it stores.
|
||||||
mkdir -p -m 700 "$HOME/backups"
|
mkdir -p -m 700 "$HOME/backups"
|
||||||
dump="$HOME/backups/latest.dump"
|
dump="$HOME/backups/latest.dump"
|
||||||
|
train_dump="$HOME/backups/latest-train.dump"
|
||||||
|
train=
|
||||||
|
if docker compose config --services | grep -qx train-server; then
|
||||||
|
train=yes
|
||||||
|
fi
|
||||||
(
|
(
|
||||||
umask 077
|
umask 077
|
||||||
# Never during a deploy (redeploy.sh holds this lock): its migrations would wait
|
# Never during a deploy (redeploy.sh holds this lock): its migrations would wait
|
||||||
|
|
@ -42,8 +48,14 @@ dump="$HOME/backups/latest.dump"
|
||||||
exec 9>"$HOME/.redeploy.lock"
|
exec 9>"$HOME/.redeploy.lock"
|
||||||
flock 9
|
flock 9
|
||||||
docker compose exec -T postgres pg_dump -U halkyon_learn -Fc -Z 0 halkyon_learn >"$dump.partial"
|
docker compose exec -T postgres pg_dump -U halkyon_learn -Fc -Z 0 halkyon_learn >"$dump.partial"
|
||||||
|
if [ -n "$train" ]; then
|
||||||
|
docker compose exec -T postgres pg_dump -U halkyon_learn -Fc -Z 0 halkyon_train >"$train_dump.partial"
|
||||||
|
fi
|
||||||
)
|
)
|
||||||
mv "$dump.partial" "$dump"
|
mv "$dump.partial" "$dump"
|
||||||
|
if [ -n "$train" ]; then
|
||||||
|
mv "$train_dump.partial" "$train_dump"
|
||||||
|
fi
|
||||||
|
|
||||||
# Each repository as a git bundle, after the dump: a release the dump lacks is never
|
# Each repository as a git bundle, after the dump: a release the dump lacks is never
|
||||||
# numbered again (the server numbers past the highest tag), while one the bundles
|
# numbered again (the server numbers past the highest tag), while one the bundles
|
||||||
|
|
@ -67,8 +79,15 @@ releases="$HOME/backups/repositories.tar"
|
||||||
)
|
)
|
||||||
mv "$releases.partial" "$releases"
|
mv "$releases.partial" "$releases"
|
||||||
|
|
||||||
restic backup --quiet "$dump" "$releases"
|
if [ -n "$train" ]; then
|
||||||
restic forget --quiet --keep-hourly 24 --keep-daily 7 --keep-weekly 4 --keep-monthly 3
|
restic backup --quiet "$dump" "$train_dump" "$releases"
|
||||||
|
else
|
||||||
|
restic backup --quiet "$dump" "$releases"
|
||||||
|
fi
|
||||||
|
# By host alone: restic groups by the paths backed up too, so a path added (Train's
|
||||||
|
# dump) would leave the snapshots before it a group of their own, never forgotten, and
|
||||||
|
# a deleted account in them for good.
|
||||||
|
restic forget --quiet --group-by host --keep-hourly 24 --keep-daily 7 --keep-weekly 4 --keep-monthly 3
|
||||||
|
|
||||||
# Once a day, in the run at 03:00: free the space of forgotten snapshots, and read the
|
# Once a day, in the run at 03:00: free the space of forgotten snapshots, and read the
|
||||||
# repository's structure; on Sundays also a tenth of the stored data, chosen at random,
|
# repository's structure; on Sundays also a tenth of the stored data, chosen at random,
|
||||||
|
|
|
||||||
|
|
@ -196,6 +196,21 @@ the way back from a restore that went wrong.
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Train's database, where `.env` switches Train on (`compose.train.yaml`, Halkyon Train's
|
||||||
|
ADR 0030, D46), is in the same snapshots as `latest-train.dump`, and is restored the
|
||||||
|
same way, its server stopped meanwhile (`docker compose stop train-server`), from the
|
||||||
|
same snapshot as Learn's, so that the register's accounts and Train's people agree:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
restic dump <snapshot> /home/deploy/backups/latest-train.dump >~/backups/restore-train.dump
|
||||||
|
docker compose exec -T postgres \
|
||||||
|
pg_restore -U halkyon_learn -d postgres --clean --create <~/backups/restore-train.dump
|
||||||
|
```
|
||||||
|
|
||||||
|
`pg_restore --create` makes the database anew owned as it was, by `halkyon_train`,
|
||||||
|
whose role is the cluster's and so survives the restore (on a new server, make it first:
|
||||||
|
`sidecars/README.md`, *Train*, step 2).
|
||||||
|
|
||||||
To check a backup without touching the instance, restore it into a database of its own
|
To check a backup without touching the instance, restore it into a database of its own
|
||||||
next to the real one, look, and drop it:
|
next to the real one, look, and drop it:
|
||||||
|
|
||||||
|
|
|
||||||
32
deploy/brouter/Dockerfile
Normal file
32
deploy/brouter/Dockerfile
Normal file
|
|
@ -0,0 +1,32 @@
|
||||||
|
# BRouter for Train's routes (ADR 0030, D11, D44, D46): its release, checked against
|
||||||
|
# GitHub's SHA-256 of it, on Eclipse Temurin's Java 17 as BRouter's own image, with
|
||||||
|
# its profiles and Train's running profile. Built from the repository root:
|
||||||
|
#
|
||||||
|
# docker build -f deploy/brouter/Dockerfile .
|
||||||
|
#
|
||||||
|
# The same image refreshes the segments (segments.sh), so it carries curl.
|
||||||
|
FROM eclipse-temurin:17-jre-noble
|
||||||
|
|
||||||
|
# BRouter v1.7.10, released 17 Jul 2026.
|
||||||
|
ADD --checksum=sha256:023fec3ba997758e8cd7ab9e1bae52e962af3f00b57683e3de86b84ffad01532 \
|
||||||
|
https://github.com/abrensch/brouter/releases/download/v1.7.10/brouter-1.7.10.zip /tmp/brouter.zip
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends unzip curl ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
|
&& unzip -q /tmp/brouter.zip -d /tmp \
|
||||||
|
&& mkdir -p /app \
|
||||||
|
&& mv /tmp/brouter-1.7.10/brouter-1.7.10-all.jar /app/brouter.jar \
|
||||||
|
&& mv /tmp/brouter-1.7.10/profiles2 /profiles2 \
|
||||||
|
&& rm -rf /tmp/brouter.zip /tmp/brouter-1.7.10
|
||||||
|
COPY apps/train/brouter/halkyon-run.brf /profiles2/halkyon-run.brf
|
||||||
|
COPY --chmod=755 deploy/brouter/segments.sh /app/segments.sh
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
EXPOSE 17777
|
||||||
|
|
||||||
|
# Segments in /segments4 (a volume), profiles in the image, no custom profiles; four
|
||||||
|
# routes at a time, each stopped after a minute. -Xmx as BRouter's script raised: the
|
||||||
|
# memory a route takes grows with its length, and Europe's long routes need more than
|
||||||
|
# its 128 MB.
|
||||||
|
CMD ["java", "-Xmx384M", "-Xms128M", "-Xmn16M", "-DmaxRunningTime=60", "-DuseRFCMimeType=false", "-DusePOSTRequests=false", \
|
||||||
|
"-cp", "/app/brouter.jar", "btools.server.RouteServer", "/segments4", "/profiles2", "/tmp", "17777", "4", "0.0.0.0"]
|
||||||
58
deploy/brouter/segments.sh
Executable file
58
deploy/brouter/segments.sh
Executable file
|
|
@ -0,0 +1,58 @@
|
||||||
|
#!/bin/sh
|
||||||
|
# Keeps BRouter's segments of the operator's region current (ADR 0030, D11, D44):
|
||||||
|
# every 5°×5° segment of BRouter's project whose south-west corner lies in
|
||||||
|
# BROUTER_REGION (`west,south,east,north` in whole degrees, multiples of 5; Europe's
|
||||||
|
# `-25,35,40,70` by default), downloaded where the project has a newer one than the
|
||||||
|
# volume holds, then again every BROUTER_REFRESH_DAYS days (30). Each is written beside
|
||||||
|
# its old self and moved over it, so BRouter never reads half a file. Segments of
|
||||||
|
# the sea, which the project doesn't have, are skipped.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
base=https://brouter.de/brouter/segments4
|
||||||
|
region=${BROUTER_REGION:--25,35,40,70}
|
||||||
|
days=${BROUTER_REFRESH_DAYS:-30}
|
||||||
|
cd /segments4
|
||||||
|
|
||||||
|
IFS=, read -r west south east north <<END
|
||||||
|
$region
|
||||||
|
END
|
||||||
|
|
||||||
|
name() {
|
||||||
|
lon=$1 lat=$2
|
||||||
|
if [ "$lon" -lt 0 ]; then ew="W$((-lon))"; else ew="E$lon"; fi
|
||||||
|
if [ "$lat" -lt 0 ]; then ns="S$((-lat))"; else ns="N$lat"; fi
|
||||||
|
echo "${ew}_${ns}.rd5"
|
||||||
|
}
|
||||||
|
|
||||||
|
refresh() {
|
||||||
|
listing=$(curl -fsS "$base/") || { echo "The segments' listing didn't come." >&2; return 0; }
|
||||||
|
fetched=0 kept=0
|
||||||
|
lon=$west
|
||||||
|
while [ "$lon" -le "$east" ]; do
|
||||||
|
lat=$south
|
||||||
|
while [ "$lat" -le "$north" ]; do
|
||||||
|
file=$(name "$lon" "$lat")
|
||||||
|
if echo "$listing" | grep -q "href=\"$file\""; then
|
||||||
|
# Only where the project's is newer than ours (If-Modified-Since): a
|
||||||
|
# current one comes back empty.
|
||||||
|
if [ -e "$file" ]; then set -- -z "$file"; else set --; fi
|
||||||
|
if curl -fsS -R --retry 3 "$@" -o "$file.part" "$base/$file" &&
|
||||||
|
[ -s "$file.part" ]; then
|
||||||
|
mv "$file.part" "$file"
|
||||||
|
fetched=$((fetched + 1))
|
||||||
|
else
|
||||||
|
rm -f "$file.part"
|
||||||
|
kept=$((kept + 1))
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
lat=$((lat + 5))
|
||||||
|
done
|
||||||
|
lon=$((lon + 5))
|
||||||
|
done
|
||||||
|
echo "Segments: $fetched fetched, $kept current, $(du -sh . | cut -f1) in all."
|
||||||
|
}
|
||||||
|
|
||||||
|
while :; do
|
||||||
|
refresh
|
||||||
|
sleep $((days * 86400))
|
||||||
|
done
|
||||||
114
deploy/compose.train.yaml
Normal file
114
deploy/compose.train.yaml
Normal file
|
|
@ -0,0 +1,114 @@
|
||||||
|
# Halkyon Train's server beside the instance (ADR 0030, D46), with the BRouter that
|
||||||
|
# plans its routes (D11, D44). Switched on in .env, with Train's domain and secrets:
|
||||||
|
#
|
||||||
|
# COMPOSE_FILE=compose.yaml:compose.train.yaml
|
||||||
|
# TRAIN_DOMAIN=train.halkyon.eu
|
||||||
|
#
|
||||||
|
# Train is a member of the instance's register, which Learn's server holds (ADR 0016):
|
||||||
|
# one account a person, signing in through the register, the shared INSTANCE_SECRET
|
||||||
|
# on every call between them, which reach each other on the instance's own networks
|
||||||
|
# alone. Its database is `halkyon_train` in the instance's PostgreSQL, under a role of
|
||||||
|
# its own that can't connect to Learn's. BRouter has no site and no sign-in: only
|
||||||
|
# Train's server reaches it, on a network of their own, and asks it nothing that is
|
||||||
|
# logged. Its segments (Europe's, about 3.3 GB) are in ./brouter/segments, which the
|
||||||
|
# refresher keeps current monthly, owned by a user ID of their own, which no person,
|
||||||
|
# no other container and not the CI runner uses. Setting it up: sidecars/README.md.
|
||||||
|
|
||||||
|
services:
|
||||||
|
train-server:
|
||||||
|
build:
|
||||||
|
context: ..
|
||||||
|
dockerfile: apps/train/halkyon_train_server/Dockerfile
|
||||||
|
image: halkyon-train-server
|
||||||
|
restart: unless-stopped
|
||||||
|
# As Learn's: migrations at every start (redeploy.sh has applied them before).
|
||||||
|
command: ["--apply-migrations"]
|
||||||
|
environment:
|
||||||
|
SERVERPOD_DATABASE_PASSWORD: ${TRAIN_POSTGRES_PASSWORD:?set TRAIN_POSTGRES_PASSWORD in deploy/.env}
|
||||||
|
SERVERPOD_PASSWORD_serverSideSessionKeyHashPepper: ${TRAIN_SESSION_KEY_HASH_PEPPER:?set TRAIN_SESSION_KEY_HASH_PEPPER in deploy/.env}
|
||||||
|
# The register (ADR 0016): Learn's web server, on the instance's own network,
|
||||||
|
# and the secret every call between them carries.
|
||||||
|
ACCOUNT_REGISTER_URL: http://server:8082/
|
||||||
|
SERVERPOD_PASSWORD_instanceSecret: ${INSTANCE_SECRET:?set INSTANCE_SECRET in deploy/.env}
|
||||||
|
# Caddy, as for Learn's server (compose.yaml).
|
||||||
|
TRUSTED_PROXIES: 172.30.80.2 fd4a:6c6b:7e80::2
|
||||||
|
SOURCE_URL: ${SOURCE_URL:-}
|
||||||
|
DATA_LOCATION: ${DATA_LOCATION:-}
|
||||||
|
BACKUP_LOCATION: ${BACKUP_LOCATION:-}
|
||||||
|
# Routes (D44), from the BRouter below.
|
||||||
|
BROUTER_URL: http://brouter:17777/
|
||||||
|
# The Loop's firmware through the instance (D40): on unless `off`.
|
||||||
|
LOOP_FIRMWARE_RELAY: ${LOOP_FIRMWARE_RELAY:-}
|
||||||
|
# The map source (D22); empty: OpenFreeMap.
|
||||||
|
MAP_STYLE_URL: ${MAP_STYLE_URL:-}
|
||||||
|
MAP_SOURCE_NAME: ${MAP_SOURCE_NAME:-}
|
||||||
|
MAP_ATTRIBUTION: ${MAP_ATTRIBUTION:-}
|
||||||
|
MAP_ABOUT_URL: ${MAP_ABOUT_URL:-}
|
||||||
|
networks:
|
||||||
|
- default
|
||||||
|
- edge
|
||||||
|
- routing
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
logging:
|
||||||
|
driver: journald
|
||||||
|
|
||||||
|
brouter:
|
||||||
|
build:
|
||||||
|
context: ..
|
||||||
|
dockerfile: deploy/brouter/Dockerfile
|
||||||
|
image: halkyon-brouter
|
||||||
|
restart: unless-stopped
|
||||||
|
user: "61177:61177"
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /tmp:size=16m,uid=61177,gid=61177
|
||||||
|
volumes:
|
||||||
|
- ./brouter/segments:/segments4:ro
|
||||||
|
# Beside Train's server alone: no site, no published port, never beside Caddy or
|
||||||
|
# Postgres.
|
||||||
|
networks:
|
||||||
|
- routing
|
||||||
|
mem_limit: 640m
|
||||||
|
# No log at all: BRouter writes each route's waypoints to its output, and where
|
||||||
|
# people run must never be kept. To look into a failure, switch to journald for
|
||||||
|
# a while.
|
||||||
|
logging:
|
||||||
|
driver: none
|
||||||
|
|
||||||
|
brouter-segments:
|
||||||
|
image: halkyon-brouter
|
||||||
|
restart: unless-stopped
|
||||||
|
user: "61177:61177"
|
||||||
|
read_only: true
|
||||||
|
command: ["/app/segments.sh"]
|
||||||
|
environment:
|
||||||
|
# `west,south,east,north` of the segments' south-west corners; Europe's.
|
||||||
|
BROUTER_REGION: ${BROUTER_REGION:--25,35,40,70}
|
||||||
|
volumes:
|
||||||
|
- ./brouter/segments:/segments4
|
||||||
|
# Out to brouter.de alone; on a network of its own, beside nothing.
|
||||||
|
networks:
|
||||||
|
- downloads
|
||||||
|
mem_limit: 64m
|
||||||
|
logging:
|
||||||
|
driver: journald
|
||||||
|
|
||||||
|
caddy:
|
||||||
|
environment:
|
||||||
|
TRAIN_DOMAIN: ${TRAIN_DOMAIN:?set TRAIN_DOMAIN in deploy/.env}
|
||||||
|
volumes:
|
||||||
|
- ./sidecars/train.caddy:/etc/caddy/sidecars/train.caddy:ro
|
||||||
|
|
||||||
|
server:
|
||||||
|
environment:
|
||||||
|
# Train's web server, told when an account goes (ADR 0016); set
|
||||||
|
# INSTANCE_MEMBERS in .env to name more members, Train's among them.
|
||||||
|
INSTANCE_MEMBERS: ${INSTANCE_MEMBERS:-http://train-server:8082/}
|
||||||
|
|
||||||
|
networks:
|
||||||
|
# Train's server and BRouter, without a way out.
|
||||||
|
routing:
|
||||||
|
internal: true
|
||||||
|
downloads:
|
||||||
|
|
@ -10,7 +10,8 @@
|
||||||
# - as a fast-forward of the deployed commit,
|
# - as a fast-forward of the deployed commit,
|
||||||
# - if every new commit is signed by a key in deploy/allowed-signers, as the deployed
|
# - if every new commit is signed by a key in deploy/allowed-signers, as the deployed
|
||||||
# checkout lists them (so no commit can add its own key),
|
# checkout lists them (so no commit can add its own key),
|
||||||
# dumps the database first (the last 3 dumps stay in ~/backups), applies the migrations
|
# dumps the database first (the last 3 dumps stay in ~/backups; Train's too where .env
|
||||||
|
# switches Train on, compose.train.yaml), applies the migrations
|
||||||
# before switching to the new server, starts it once more after a migration (see below),
|
# before switching to the new server, starts it once more after a migration (see below),
|
||||||
# and fails unless that server answers afterwards. A failure leaves the checkout at the deployed commit, so
|
# and fails unless that server answers afterwards. A failure leaves the checkout at the deployed commit, so
|
||||||
# running the CI job again retries. By hand, with your own key, which passes the commit as CI does:
|
# running the CI job again retries. By hand, with your own key, which passes the commit as CI does:
|
||||||
|
|
@ -59,15 +60,30 @@ main() {
|
||||||
git merge -q --ff-only "$commit"
|
git merge -q --ff-only "$commit"
|
||||||
trap back_out EXIT
|
trap back_out EXIT
|
||||||
cd deploy
|
cd deploy
|
||||||
|
# Train's server, where .env switches it on (compose.train.yaml): built, dumped,
|
||||||
|
# migrated and waited for as Learn's.
|
||||||
|
train=
|
||||||
|
if docker compose config --services | grep -qx train-server; then
|
||||||
|
train=yes
|
||||||
|
fi
|
||||||
# Build while the running server keeps serving; then dump, migrate, switch.
|
# Build while the running server keeps serving; then dump, migrate, switch.
|
||||||
docker compose build -q server dictionaries
|
docker compose build -q server dictionaries
|
||||||
dump
|
if [ -n "$train" ]; then
|
||||||
|
docker compose build -q train-server brouter
|
||||||
|
fi
|
||||||
|
dump halkyon_learn
|
||||||
|
if [ -n "$train" ]; then
|
||||||
|
dump halkyon_train
|
||||||
|
fi
|
||||||
# Migrations as a run of their own, before the switch: Serverpod's maintenance role
|
# Migrations as a run of their own, before the switch: Serverpod's maintenance role
|
||||||
# applies them and exits with 0 only if that worked. Each migration is one
|
# applies them and exits with 0 only if that worked. Each migration is one
|
||||||
# transaction, so a failing one stops the deploy here and the running server keeps
|
# transaction, so a failing one stops the deploy here and the running server keeps
|
||||||
# serving the schema it knows. (Until the switch, the old server serves the new
|
# serving the schema it knows. (Until the switch, the old server serves the new
|
||||||
# schema; migrations that only add, as they should, keep that harmless.)
|
# schema; migrations that only add, as they should, keep that harmless.)
|
||||||
docker compose run --rm --no-deps -e role=maintenance server --apply-migrations
|
docker compose run --rm --no-deps -e role=maintenance server --apply-migrations
|
||||||
|
if [ -n "$train" ]; then
|
||||||
|
docker compose run --rm --no-deps -e role=maintenance train-server --apply-migrations
|
||||||
|
fi
|
||||||
switched=yes
|
switched=yes
|
||||||
docker compose up -d --remove-orphans
|
docker compose up -d --remove-orphans
|
||||||
# Caddy reads its Caddyfile when it starts, and `up` doesn't restart it for a
|
# Caddy reads its Caddyfile when it starts, and `up` doesn't restart it for a
|
||||||
|
|
@ -79,7 +95,10 @@ main() {
|
||||||
if ! git diff --quiet "$deployed" "$commit" -- Caddyfile 'sidecars/*.caddy'; then
|
if ! git diff --quiet "$deployed" "$commit" -- Caddyfile 'sidecars/*.caddy'; then
|
||||||
docker compose up -d --force-recreate --no-deps caddy
|
docker compose up -d --force-recreate --no-deps caddy
|
||||||
fi
|
fi
|
||||||
wait_until_answering
|
wait_until_answering server
|
||||||
|
if [ -n "$train" ]; then
|
||||||
|
wait_until_answering train-server
|
||||||
|
fi
|
||||||
# After a migration the new server is started once more. The sync engine
|
# After a migration the new server is started once more. The sync engine
|
||||||
# (serverpod_offline_sync v0.0.10) reconciles its registry of synced tables as a
|
# (serverpod_offline_sync v0.0.10) reconciles its registry of synced tables as a
|
||||||
# server first uses the database; when a migration changed a synced table, that
|
# server first uses the database; when a migration changed a synced table, that
|
||||||
|
|
@ -90,42 +109,50 @@ main() {
|
||||||
# finds the registry as the first start left it.
|
# finds the registry as the first start left it.
|
||||||
if ! git diff --quiet "$deployed" "$commit" -- ../apps/learn/halkyon_learn_server/migrations; then
|
if ! git diff --quiet "$deployed" "$commit" -- ../apps/learn/halkyon_learn_server/migrations; then
|
||||||
docker compose restart server
|
docker compose restart server
|
||||||
wait_until_answering
|
wait_until_answering server
|
||||||
|
fi
|
||||||
|
# Train's server too: it syncs through the same engine.
|
||||||
|
if [ -n "$train" ] &&
|
||||||
|
! git diff --quiet "$deployed" "$commit" -- ../apps/train/halkyon_train_server/migrations; then
|
||||||
|
docker compose restart train-server
|
||||||
|
wait_until_answering train-server
|
||||||
fi
|
fi
|
||||||
trap - EXIT
|
trap - EXIT
|
||||||
docker image prune -f >/dev/null
|
docker image prune -f >/dev/null
|
||||||
echo "Deployed $commit."
|
echo "Deployed $commit."
|
||||||
}
|
}
|
||||||
|
|
||||||
# Dumps the database, as it is before the new server applies its migrations.
|
# Dumps the database $1 (halkyon_learn, halkyon_train), as it is before the new server
|
||||||
|
# applies its migrations; as the instance's superuser, halkyon_learn.
|
||||||
dump() {
|
dump() {
|
||||||
mkdir -p "$HOME/backups"
|
mkdir -p "$HOME/backups"
|
||||||
file="$HOME/backups/halkyon_learn-$(git rev-parse --short "$deployed")-$(date -u +%Y%m%dT%H%M%SZ).dump"
|
file="$HOME/backups/$1-$(git rev-parse --short "$deployed")-$(date -u +%Y%m%dT%H%M%SZ).dump"
|
||||||
(
|
(
|
||||||
umask 077
|
umask 077
|
||||||
docker compose exec -T postgres pg_dump -U halkyon_learn -Fc halkyon_learn >"$file"
|
docker compose exec -T postgres pg_dump -U halkyon_learn -Fc "$1" >"$file"
|
||||||
)
|
)
|
||||||
echo "Database dumped to $file ($(du -h "$file" | cut -f1))."
|
echo "Database dumped to $file ($(du -h "$file" | cut -f1))."
|
||||||
# Only the newest 3 (decided 2026-10-02): they are the deploys' way back, not backups;
|
# Only the newest 3 (decided 2026-10-02): they are the deploys' way back, not backups;
|
||||||
# those go off-site. Every file people keep is in each of them, so fewer leave room.
|
# those go off-site. Every file people keep is in each of them, so fewer leave room.
|
||||||
# (ls is safe here: the names are ours, letters, digits and dashes.)
|
# (ls is safe here: the names are ours, letters, digits and dashes.)
|
||||||
# shellcheck disable=SC2012
|
# shellcheck disable=SC2012
|
||||||
ls -1t "$HOME"/backups/halkyon_learn-*.dump | tail -n +4 | xargs -r rm --
|
ls -1t "$HOME"/backups/"$1"-*.dump | tail -n +4 | xargs -r rm --
|
||||||
}
|
}
|
||||||
|
|
||||||
# Waits up to a minute for the new server to answer inside the stack's network.
|
# Waits up to a minute for the new server $1 (server, train-server) to answer inside the
|
||||||
|
# stack's network.
|
||||||
wait_until_answering() {
|
wait_until_answering() {
|
||||||
tries=0
|
tries=0
|
||||||
while [ $tries -lt 30 ]; do
|
while [ $tries -lt 30 ]; do
|
||||||
if docker compose exec -T caddy wget -q -O /dev/null http://server:8080/ 2>/dev/null; then
|
if docker compose exec -T caddy wget -q -O /dev/null "http://$1:8080/" 2>/dev/null; then
|
||||||
echo 'The server answers.'
|
echo "The $1 answers."
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
tries=$((tries + 1))
|
tries=$((tries + 1))
|
||||||
sleep 2
|
sleep 2
|
||||||
done
|
done
|
||||||
docker compose logs --tail 40 server >&2
|
docker compose logs --tail 40 "$1" >&2
|
||||||
fail "the server doesn't answer a minute after the switch"
|
fail "the $1 doesn't answer a minute after the switch"
|
||||||
}
|
}
|
||||||
|
|
||||||
# A failed deploy puts the checkout back at the deployed commit, so running the CI job
|
# A failed deploy puts the checkout back at the deployed commit, so running the CI job
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,7 @@ site changes.
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Jellyfin, which people watch and listen to learn from | `compose.jellyfin.yaml` | `jellyfin.caddy` |
|
| Jellyfin, which people watch and listen to learn from | `compose.jellyfin.yaml` | `jellyfin.caddy` |
|
||||||
| LanguageTool, which checks spelling and grammar | `compose.languagetool.yaml` | none |
|
| LanguageTool, which checks spelling and grammar | `compose.languagetool.yaml` | none |
|
||||||
|
| Halkyon Train's server, with BRouter for its routes | `compose.train.yaml` | `train.caddy` |
|
||||||
|
|
||||||
Both together: `COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml:compose.languagetool.yaml`.
|
Both together: `COMPOSE_FILE=compose.yaml:compose.jellyfin.yaml:compose.languagetool.yaml`.
|
||||||
|
|
||||||
|
|
@ -151,3 +152,76 @@ Steps, in order; ✱ marks a step as root.
|
||||||
Switching it off: the compose file out of `COMPOSE_FILE`, then `docker compose up -d
|
Switching it off: the compose file out of `COMPOSE_FILE`, then `docker compose up -d
|
||||||
--remove-orphans`; the app's Writing settings then say the instance checks nothing, and
|
--remove-orphans`; the app's Writing settings then say the instance checks nothing, and
|
||||||
`deploy/languagetool/` stays until removed by hand.
|
`deploy/languagetool/` stays until removed by hand.
|
||||||
|
|
||||||
|
## Train
|
||||||
|
|
||||||
|
Halkyon Train's server beside the instance (Halkyon Train's ADR 0030, D46), at its own
|
||||||
|
domain, a member of the register Learn's server holds (ADR 0016), its database
|
||||||
|
`halkyon_train` in the instance's PostgreSQL, and BRouter planning its routes (D11, D44)
|
||||||
|
from the segments of the operator's region (Europe's by default, about 3.3 GB on
|
||||||
|
28 Oct 2026), refreshed monthly. `redeploy.sh` builds, dumps, migrates and waits for it
|
||||||
|
as for Learn's server once `.env` names its compose file, and `backup.sh` backs its
|
||||||
|
database up beside Learn's. The map stays OpenFreeMap's until the instance serves its
|
||||||
|
own tiles (D22, D23).
|
||||||
|
|
||||||
|
Steps, in order; ✱ marks a step as root.
|
||||||
|
|
||||||
|
1. **Room on the disk** (the operator, on the instance's server): `df -h ~deploy`.
|
||||||
|
BRouter's segments take about 3.3 GB for Europe, and a refresh writes each one beside
|
||||||
|
the old before replacing it; Train's dumps go to `~/backups` with Learn's.
|
||||||
|
2. **Its database** (the operator, on the instance's server, as `deploy`, in
|
||||||
|
`~/halkyon-learn/deploy`): a role of Train's own, with the password that goes into
|
||||||
|
`.env` as `TRAIN_POSTGRES_PASSWORD` (`openssl rand -hex 32`), and its database:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose exec -T postgres psql -U halkyon_learn -d postgres -v ON_ERROR_STOP=1 \
|
||||||
|
-v password="<TRAIN_POSTGRES_PASSWORD>" <<'SQL'
|
||||||
|
CREATE ROLE halkyon_train LOGIN PASSWORD :'password';
|
||||||
|
CREATE DATABASE halkyon_train OWNER halkyon_train;
|
||||||
|
REVOKE CONNECT ON DATABASE halkyon_train FROM PUBLIC;
|
||||||
|
REVOKE CONNECT ON DATABASE halkyon_learn FROM PUBLIC;
|
||||||
|
SQL
|
||||||
|
```
|
||||||
|
|
||||||
|
`CREATE ROLE` makes the role Train's server signs in as, which may log in and is
|
||||||
|
nothing more (no superuser, no creating roles or databases); `CREATE DATABASE … OWNER`
|
||||||
|
gives it its own database, where its migrations make the tables. The two `REVOKE`s
|
||||||
|
take away the right every role has by default to connect to a database: only its
|
||||||
|
owner (and the instance's superuser, `halkyon_learn`, which Learn's server signs in
|
||||||
|
as) connects to `halkyon_train`, and `halkyon_train` no longer connects to Learn's.
|
||||||
|
Nothing of Learn's changes. (Learn's server, signing in as the superuser, could still
|
||||||
|
read Train's; giving it a role of its own is a step of its own.)
|
||||||
|
3. ✱ **BRouter's user and folder** (the operator, as a user with sudo): the user and
|
||||||
|
group ID 61177 are BRouter's alone, as the compose file runs it; the folder its
|
||||||
|
segments go to is theirs.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo groupadd --system --gid 61177 brouter
|
||||||
|
sudo useradd --system --uid 61177 --gid 61177 --no-create-home --shell /usr/sbin/nologin brouter
|
||||||
|
sudo mkdir -p ~deploy/halkyon-learn/deploy/brouter/segments
|
||||||
|
sudo chown -R brouter:brouter ~deploy/halkyon-learn/deploy/brouter/segments
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **DNS** (the operator, at the domain's DNS): Train's domain, `train.halkyon.eu`,
|
||||||
|
pointing at the instance's server (`A`, and `AAAA` if it has an IPv6 address).
|
||||||
|
5. **Switch it on** (the operator, as `deploy`): in `deploy/.env`, `compose.train.yaml`
|
||||||
|
added to `COMPOSE_FILE`, and Train's settings (`.env.example`, *With Halkyon Train*):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
COMPOSE_FILE=compose.yaml:compose.languagetool.yaml:compose.train.yaml
|
||||||
|
TRAIN_DOMAIN=train.halkyon.eu
|
||||||
|
TRAIN_POSTGRES_PASSWORD=<as in step 2>
|
||||||
|
TRAIN_SESSION_KEY_HASH_PEPPER=<openssl rand -hex 32>
|
||||||
|
```
|
||||||
|
|
||||||
|
`INSTANCE_SECRET` must be set (Learn's register takes no member without it). Where
|
||||||
|
`INSTANCE_MEMBERS` is set, add `http://train-server:8082/` to it; unset, the compose
|
||||||
|
file names Train.
|
||||||
|
6. **Start it** (the operator, as `deploy`): `docker compose up -d --build`. Caddy and
|
||||||
|
Learn's server are recreated with it (Train's site, and the register's member); the
|
||||||
|
instance is away for seconds. Train's server applies its migrations as it starts.
|
||||||
|
The segments come in the first hour or so (`docker compose logs brouter-segments`
|
||||||
|
says when); until then BRouter finds no route.
|
||||||
|
7. **Check it**: `https://train.halkyon.eu/` answers; in Train, signed in with a Learn
|
||||||
|
account, a route plans between two points of the region.
|
||||||
|
|
||||||
|
|
|
||||||
10
deploy/sidecars/train.caddy
Normal file
10
deploy/sidecars/train.caddy
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
# Halkyon Train's site (compose.train.yaml, ADR 0030, D46): its API server alone. Its
|
||||||
|
# web server (8082) takes only the register's calls between the instance's services
|
||||||
|
# (`/halkyon-base/…`, ADR 0016), on the instance's own network, never through here.
|
||||||
|
{$TRAIN_DOMAIN} {
|
||||||
|
encode zstd gzip
|
||||||
|
|
||||||
|
# Serverpod's streaming methods use WebSockets; reverse_proxy upgrades them as-is.
|
||||||
|
# The visitor's address comes last in X-Forwarded-For, as for Learn's server.
|
||||||
|
reverse_proxy train-server:8080
|
||||||
|
}
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# ADR 0030: Halkyon Train, what it does and where its data comes from
|
# ADR 0030: Halkyon Train, what it does and where its data comes from
|
||||||
|
|
||||||
- **Status:** decided 27–28 Oct 2026, by the developer (D1–D45 below). The shell (D12)
|
- **Status:** decided 27–28 Oct 2026, by the developer (D1–D46 below). The shell (D12)
|
||||||
is built; recording (D13–D28) is built on Android, iOS waiting for a Mac; Health
|
is built; recording (D13–D28) is built on Android, iOS waiting for a Mac; Health
|
||||||
Connect (D29–D31) is built; the Loop's setup and keeping (7a, D34–D36) and its
|
Connect (D29–D31) is built; the Loop's setup and keeping (7a, D34–D36) and its
|
||||||
daily data (7b, D37, D38), its sessions (7c, D39) and its firmware (D10, D40) are
|
daily data (7b, D37, D38), its sessions (7c, D39) and its firmware (D10, D40) are
|
||||||
|
|
@ -881,6 +881,28 @@ Strava. What was found, 27 Oct:
|
||||||
(to CoMaps or anything, with waypoints and turn hints) and come in as GPX shared to
|
(to CoMaps or anything, with waypoints and turn hints) and come in as GPX shared to
|
||||||
Train or opened in it, snapped to a profile or kept as drawn (over out alone, and
|
Train or opened in it, snapped to a profile or kept as drawn (over out alone, and
|
||||||
over neither yet).
|
over neither yet).
|
||||||
|
- **D46 Train's server on the instance** (recommended, the developer's; 28 Oct): it
|
||||||
|
joins learn.halkyon.eu's server, as BRouter needs a server of Train's to ask it
|
||||||
|
(D11, D22), at **train.halkyon.eu** (over a path on learn.halkyon.eu, which would
|
||||||
|
carry Learn's name for good); its data in the instance's PostgreSQL as a database of
|
||||||
|
its own, `halkyon_train`, under a role that can't connect to Learn's (over a second
|
||||||
|
PostgreSQL, twice the memory and a second backup); a **member of the instance's
|
||||||
|
register** on Learn's server (ADR 0016: one account a person across the operator's
|
||||||
|
services, over a register of its own). It is a compose file of its own,
|
||||||
|
`deploy/compose.train.yaml`, switched on in `.env` as the sidecars are, so nothing
|
||||||
|
changes on the instance until the operator does (the steps in
|
||||||
|
`deploy/sidecars/README.md`, *Train*); `redeploy.sh` then builds, dumps, migrates
|
||||||
|
and waits for it as for Learn's server, and `backup.sh` backs its database up in the
|
||||||
|
same snapshots. BRouter (v1.7.10, its release checked by SHA-256) runs beside Train's
|
||||||
|
server alone, on a network without a way out, with no log (it writes the waypoints),
|
||||||
|
its segments refreshed monthly by a container of its own; Europe's took 3.3 GB on
|
||||||
|
28 Oct. Whether the server's disk has room is the developer's to check before
|
||||||
|
switching it on. The map stays OpenFreeMap's: the PMTiles sidecar needs a style for
|
||||||
|
Protomaps' schema, a step of its own (about 40 GB for Europe, estimated from the
|
||||||
|
planet's 120 GB). Learn's server signs in to PostgreSQL as its superuser, so it could
|
||||||
|
read Train's database; a role of its own for it is left for later. restic now forgets
|
||||||
|
snapshots grouped by host alone, so that the snapshots before Train's dump joined
|
||||||
|
them aren't kept for good as a group of their own.
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue