halkyon/docs/adr/0030-halkyon-train.md

350 lines
26 KiB
Markdown
Raw Normal View History

# ADR 0030: Halkyon Train, what it does and where its data comes from
- **Status:** decided 27–28 Oct 2026, by the developer (D1–D21 below). The shell (D12)
is built; recording (D13–D21) is being built.
- **Numbering:** 0025 is held by the federation draft (stashed), 0026–0029 by the .NET
rewrite on the local branch `rewrite-dotnet`, abandoned on 27 Oct; Train takes the next
free number so that neither is ever read as this one.
- **Followed by:** [ADR 0031](0031-the-apps-icons.md) (the apps' icons, Train's among them).
- **Relates to:** ADR 0014 (halkyon_base, which Train's server includes like every other),
ADR 0016 (one account across an operator's services; Train is one of them).
## Context
Train is the third of Halkyon's apps after Learn and Todo, so far only named. On 27 Oct
the developer asked for it to "encompass advanced features for running and cycling", to
connect to a Polar Loop, and for rowing machines' APIs to be looked into (WaterRower,
Kettler, Christopeit, BH Fitness, Tunturi), the Christopeit WP 5000 in detail. For running
they asked for "GPS tracking, automatic stop (*at least* the functionality of e.g.
F-Droid's OpenTracks)", HIIT, music in tandem with the pace, and planning tracks as on
Strava. What was found, 27 Oct:
- **Open Bluetooth profiles carry most of it.** Heart rate (`0x180D`, measurement `0x2A37`
with RR intervals, hence HRV), running speed and cadence (`0x1814`), cycling speed,
cadence and power, and the Fitness Machine Service (FTMS, `0x1826`; Rower Data
`0x2AD1`, Indoor Bike Data, Treadmill Data; resistance set through its control point).
Any strap, footpod, power meter, smart trainer or FTMS rower works without a vendor's
account or SDK, on every platform with Bluetooth LE. Pixels have no ANT+ radio, so on
GrapheneOS Train is Bluetooth LE only.
- **The Polar Loop** (Sept 2025, no screen, no GPS) is no Bluetooth standard: it speaks
Polar's own protocol, and doesn't broadcast its heart rate as a `0x180D` sensor (Polar:
it "cannot be used as a heart rate sensor for other devices"). Polar's BLE SDK
(Android and iOS, its own permissive licence, source-available) covers it: first-time
setup from one's own app (`doFirstTimeUse()`, no Polar account named anywhere), live
HR, acceleration, PPI, skin temperature and PPG, and the export of sleep, sessions,
daily activity, 24/7 HR and PPI and Nightly Recharge. It pairs with one peer only;
re-pairing needs a factory reset. Firmware updates fetch from Polar (a check sending
the device's UUID, firmware version and hardware code, no health data). Polar's cloud
(AccessLink, OAuth) gives much the same through Polar Flow, free for now and revocable.
- **Rowing machines**: none of the five brands publishes an API. WaterRower documents
its S4/S5 monitor's USB serial protocol and its ComModule speaks FTMS; BH's current
i.Concept range speaks FTMS; Kettler's Coach 2 very probably, Christopeit's and
Tunturi's unconfirmed. Concept2's PM5 is documented and speaks FTMS. Older machines on
Classic Bluetooth (SPP) are reachable neither on iOS nor well from Flutter.
- **The Christopeit WP 5000** (manual of 6 Nov 2025): Bluetooth 4.0, a built-in Bluetooth
heart-rate receiver, water with a 16-level magnetic brake the console moves itself (its
12 programmes change it), mains-powered. Its manual names Kinomap, picked by brand, not
as a generic FTMS device; an owner reports (mid-2025) that neither a Garmin's FTMS
rowing profile nor EXR could pair. Christopeit's own app, Christopeit Connect (Vescape,
`com.vescape.christopeit`), supports it, declares no data collected (App Store), and
writes workouts to Health Connect or Apple Health. Whether the console speaks FTMS is
unknown until it is scanned.
- **Health Connect** is part of Android since 14, so GrapheneOS has it without Google Play;
data stays on the device and each app is granted each type. Gadgetbridge (Garmin,
Huawei, Colmi rings, Amazfit and others, no account) and Christopeit Connect write to
it; OpenTracks writes sessions, distance, speed and routes there, but no heart rate.
- **OpenTracks** is Apache-2.0, so its logic may be taken into Train (AGPL) with its
notice; the open projects talking to rowers and trainers (qdomyos-zwift,
OpenRowingMonitor, pirowflo) are GPL, references for the protocols only.
- **Music by tempo**: Spotify closed its tempo data (audio features) to new apps on 27 Nov
2024; Qobuz grants its API to commercial partners only and refuses open-source
integrations. Another Android app's title can be read from its media session and its
tempo looked up (Deezer's `bpm`, often missing; AcousticBrainz, frozen since 2022), but
only shown, never chosen by. Jellyfin's API is open but has no tempo field.
## Decisions (27 Oct 2026, the developer's)
- **D1 What Train does**: running and cycling with "advanced features", rowing machines,
the Polar Loop. For running at least what OpenTracks does (GPS tracking, pausing by
itself), structured intervals (HIIT), music matching the pace, and planning routes as
on Strava.
- **D2 In this order** (recommended): recording first (GPS, auto-pause, a heart-rate
strap), then intervals, then route planning, then music. Recording is what everything
else builds on; route planning is the largest piece (maps and a router); music needs
the intervals' targets to follow.
- **D3 Train takes over the Polar Loop** through Polar's BLE SDK, with no Polar Flow and no
Polar account (recommended, over AccessLink, Polar's cloud, as the main way in): the
SDK gives nearly all AccessLink would, on the device and without Polar in between. Since
the Loop pairs with one peer, the band is Train's alone; the phone holding it syncs it,
and other devices see its data through Train's server, as the SDK runs on Android and
iOS only.
- **D4 Music comes from Jellyfin, with Train as the player** (recommended, over reading
what Spotify or Qobuz play, which the developer judged "a rather stale feature"): Train
chooses the next track by the cadence or pace an interval asks for, may speed a track
up or slow it down a few per cent, and downloads the chosen tracks before a run for
playing offline. Each track's tempo is worked out by Train from its audio, once, and
kept (Jellyfin has no tempo field; its MusicTags plugin only passes on a BPM tag files
already have). A metronome at the target cadence works where no library is set.
- **D5 Devices connect to phones only, through `flutter_reactive_ble`** (BSD-3, Android
and iOS, Philips Hue's; recommended once the desktop was out, over `universal_ble`,
BSD-3 on every platform but younger and less used, and over `flutter_blue_plus`, whose
own licence asks a fee for commercial use, which an AGPL app can't pass on): "I find it
exceedingly unlikely one owns or purchases a fitness band or smartwatch without
intending or even owning a smartphone", and "the open license is also of paramount
importance". Train talks to its own small interface (scan, connect, subscribe, write),
the profiles' parsers are pure Dart, tested on recorded frames, so another library can
take its place behind it.
- **D6 Health Connect both ways** (recommended): Train writes its workouts there (the
session with its type, route, heart rate, distance, speed and calories), off until the
person turns it on, its own store staying the source of truth; and reads from there
other apps' workout sessions (their heart rate, distance, power) and the person's weight
and resting heart rate, each source turned on by itself. Sources Train can't speak to
(Garmin and Colmi through Gadgetbridge, the WP 5000 through Christopeit Connect) come
in so; Train tells its own records by their origin and takes a session another app
wrote of a workout it recorded itself for the same one. Through the `health` package
(MIT, CARP's, Android and iOS; recommended was a Kotlin channel of our own): it has no
types for power or cadence, which Train therefore neither writes nor reads there, but
records from the sensors directly; they can be added upstream or by a channel of our
own when a source needs them.
- **D7 Train's data is sealed, always** (recommended): the payloads of its rows (sessions,
routes, every series, what is imported) are ADR 0017's sealed content under the
person's data key, as ADR 0019 seals a private studyset, so the instance syncs what it
can't read; imported data syncs as Train's own does. Health data is a special category
of the GDPR (Art. 9), and nothing on the server needs to read it. The instance still
sees that a row exists, its size, and when it changed; what Train works out (training
load, zones, records) is worked out on the device; a route shared later is a copy made
readable on purpose, as publishing is in Learn.
- **D8 The WP 5000 is an FTMS rower, reached by Train's FTMS client and nothing of its
own** (found on the developer's machine, 27 Oct, with nRF Connect): its console
(`MRK-950R-8021`) advertises the Fitness Machine service, with its service data saying
"rower, available" (`0x1826`, `01 10 00`), and offers Fitness Machine Feature, Status,
Training Status, the Control Point (write, indicate), Rower Data (notify) and Supported
Resistance Level Range. Its features: cadence, total distance, pace, resistance level,
stroke count, energy, heart rate, elapsed time, power, and setting a resistance target.
A frame while rowing: 25 strokes a minute, 6 strokes, 18 m, pace 439 s, 100 W,
resistance 1, 1 kcal, heart rate 0 (no strap), 17 s. The forum's report that Garmin and
EXR couldn't pair isn't explained by the machine as it is now (an older firmware, or a
console already connected elsewhere, which most allow one of, may be). Its other
services (`0xFFF0`, `0xF100`, `0xAE00`, and one whose UUID spells "YULU" and "MERACH",
presumably the console's maker's own protocol, which Kinomap and Christopeit Connect
may use) are left alone; Health Connect and working out a protocol aren't needed.
- **D9 No WaterRower USB driver for now** (recommended): it would serve S4 monitors
without the ComModule, on Android only (iOS allows no USB serial device outside Apple's
accessory programme), through an OTG cable during the workout, as a second transport
beside Bluetooth, with no WaterRower at hand to test on; a WaterRower with the
ComModule speaks FTMS. Its protocol is published, so a driver can be added behind
Train's sensor interface once someone asks for it with one at hand.
- **D10 The Loop's firmware: from Polar directly by default, or through the instance**
(the developer's choice, after option 1 alone was recommended for phones always on a
VPN; over updating never). Polar's SDK checks with Polar's Firmware Management API
(sending a UUID made from the band's ID, its firmware version, its hardware code and
a fixed client name, no health data) and flashes either what it fetched or a file on
the phone (`updateFirmware(identifier, firmwareUrl)` takes `file://`, read in its
source, 27 Oct); the band's settings are backed up over Bluetooth into the phone's
memory and written back, never sent anywhere. So:
- **Directly** (the default): the phone asks Polar and fetches the file. Polar learns
the phone's address, a VPN's exit if one is always on (on GrapheneOS with "Block
connections without VPN").
- **Through the instance** (a setting): the phone sends the band's UUID, firmware
version and hardware code to Train's server, which asks Polar, fetches the file once
for every band of that hardware and hands it to the phone, which flashes it from a
file. Polar learns the instance's address, which on a small instance points at its
operator nearly as well as a home's does.
Train says both where the setting is, and names what Polar learns where it offers an
update. An update never starts by itself: Train first syncs what is on the band and
deletes it there, says the band is erased and set up again, and asks.
- **D11 Maps and routes** (recommended, after looking at how open apps do it; 27 Oct):
- **The map widget is `flutter_map`** (BSD-3, Fleaflet's; the developer's choice, over
`maplibre`, BSD-3 and rendering vector tiles natively but at 0.3): drawing tracks,
heart rate's colours and waypoints over the map is ordinary Flutter. Vector tiles
come through `vector_map_tiles` and its PMTiles plugin, tested on the developer's
phone with a region's file before route planning is built; should it stutter,
`maplibre` takes the map screens' place.
- **The map's source is the instance's, one of three** (an operator's setting): a PMTiles
file of its region as a sidecar (recommended, seen by no one else); OpenFreeMap (no
key, no contract; the default without a sidecar); or a provider of the operator's
choice with its address and key (MapTiler, JawgMaps and the European others), whose
key is then readable by anyone using the app. OpenStreetMap's own tile servers forbid
such use. The app names the source in use and what it sees.
- **The router is BRouter, as a sidecar** (MIT; recommended over GraphHopper, which needs
gigabytes to import a country, and Valhalla, with fewer sport profiles): its own image,
128–512 MB, the 5°×5° segments of the operator's region downloaded from BRouter's
project (built from OSM with elevation) and refreshed monthly, its profiles text files
a running one can be written for. As LanguageTool: no site of its own, reached by
Train's server only, for people signed in, limited, no waypoint kept or logged. A
route saved is Train's data, sealed (D7). Searching for places (geocoding) is
decided when planning is built; until then a route is planned by tapping the map.
- **Offline**: recording, following a route (synced beforehand with BRouter's turn
hints), noticing a detour and guiding back to the route, music. Maps offline along a
route (a corridor of a few kilometres downloaded with it), whole regions optional:
a phone with CoMaps holds them already. Planning, rerouting and searching need the
instance; BRouter on Android (its app, or its engine inside Train) may come later,
never on iOS (no Java).
- **With CoMaps** (Apache-2.0, OSM's data; read in its source at `63a53413`, 27 Oct, by
an agent, nothing tried on a device): its maps (MWM, its own format, in its private
storage) can't be read by another app, so the apps work together instead:
1. a route planned in Train goes to CoMaps as GPX (VIEW or SEND, a new list each;
tracks, routes, waypoints, elevation, times, colours kept; no GeoJSON), to be
seen there offline;
2. turn hints come from Train, which announces them with the workout's own, since
CoMaps navigates along no imported track (its issue #251, open);
3. optional: a route to a point is handed to CoMaps (`cm://route?sll=…&saddr=…&dll=…
&daddr=…&type=pedestrian|bicycle`), which builds it offline and the person starts;
on Android, Train may read along through CoMaps' navigation provider (since mid
2026; a runtime permission the person grants), for the route and what is left;
4. tracks and routes from CoMaps reach Train as GPX when the person shares them.
Train builds the `cm://` links itself: Organic Maps' API library and `om://` don't
reach CoMaps on Android. On iOS (`iphone/Maps/CoMaps.plist`, `DeepLinkHandler.swift`,
read 28 Oct): GPX, KML and KMZ open in CoMaps (1, 4), `cm://route` works with one
point between start and finish (3), its `backurl` leads back to Train; picking a
point (`crosshair`) is "not supported on iOS", and iOS has no navigation provider.
- **D12 Train's shell is Todo's** (recommended, 27 Oct; the icon the developer's choice,
the rest taken without objection on 28 Oct):
- **a server of its own**, `apps/train/` with `halkyon_train_server`, `_client`,
`_flutter`, including halkyon_base (`trainBase()`), its own register when it runs
alone and a member of the instance's where the operator names one (ADR 0016); over
Train's endpoints in Learn's server, whose deploys, migrations and failures would
then be Train's, and which an operator couldn't run Train without;
- **synced with `serverpod_offline_sync`** (ADR 0001), its rows sealed (D7), since
recording happens offline;
- **ports of its own in development**, 8180–8191 where Learn and Todo share
8080–8091, so that Train runs beside Learn's server as a member of its register;
- **Android and iOS** record; **Linux, macOS and Windows** show and plan (D5); no web,
as for Learn and Todo, whose keys (ADR 0017) need libsodium natively;
- **Halkyon Train**, `app.halkyon.train`, in English and German alike;
- **its icon is Lucide's `bike`**, in the family of [ADR 0031](0031-the-apps-icons.md)
("activity" and "heart-pulse" look like basic health apps, "a defibrillator or ADAC
health application"; "this is more about training"; Lucide has no runner);
- **deployed once recording works** and the developer wants to use it, not as an
empty shell first.
## Recording (28 Oct 2026, the developer's)
- **D13 Positions are recorded by Train's own platform code** (recommended, over the
`geolocator` package and paid background-tracking packages): on Android a foreground
service of its own, as Learn's Pomodoro timer has, reading `LocationManager`'s GPS
provider as OpenTracks does and appending every fix to a recording file in the app's
folder, which the app follows while open and takes in when it opens again; on iOS
CoreLocation with background location; both behind one interface (`TrackRecorder`),
faked in tests. So a recording goes on with the app swiped away, and nothing of Google
Play services is in the app: `geolocator`'s Android part depends on Play services'
location library even where its `forceLocationManager` keeps it from being used, a
non-free library that F-Droid's main repository refuses, and its positions arrive in
Dart, which a swipe ends. The strap and the rower join the same service later.
- **D14 A workout and its series are kept apart** (recommended, over one row holding
every sample and over sealed GPX or FIT files in the instance's storage): `workout`
(type, start, end, name, notes, totals; changed when edited, the last writer winning, as
a studyset's payload) and `workout_series` (a chunk of one series, positions, heart rate,
strokes, pauses, of up to 10 minutes; written once as the workout is saved, never
changed, deleted with its workout, as Learn's review log), every payload sealed (D7)
and nothing readable in a column, not the type nor the start. Editing a workout touches
one small row, two devices never conflict over samples, and every source is one more
kind of series. A chunk's payload is JSON as every payload, each column (time,
latitude, longitude, altitude, accuracy, …) a list of whole-number differences from the
value before, gzipped before it is sealed, since sealed bytes compress no more. The
recording file is the device's own and never synced. UUID v7 ids carry their creation
time, which the sync engine's change times tell anyway (D7's list).
- **D15 Sealing: the envelope moves into halkyon_app, Train holds its key itself**
(recommended, over moving all of Learn's key management and over a copy of the
envelope in Train): Learn's `PayloadEnvelope` (a payload sealed for its table, row and
column; readable payloads signed by the account; what doesn't open `Locked`) moves
unchanged into halkyon_app, Learn importing it from there; Train has one key per person
in a synced `content_key` row, its sealed ID saying it is the person's, and a device
without the account's keys writes nothing, as in Learn. Learn's tree (private or
readable studysets, their records, rewrites) stays Learn's. Publishing a workout later
doesn't change it (asked by the developer): a published workout is a readable copy made
on the device (D7), which may leave out the heart rate or the first and last few hundred
metres, as Strava's privacy zones do, the sealed workout staying as it is; the envelope
can sign the copy with the account's key.
- **D16 Auto-pause by each activity's best signal, never discarding samples**
(recommended, over GPS alone and over manual pauses only): a pause, automatic or by
hand, is a marker in a series of its own, and moving time, pace and the totals leave
paused stretches out, so a misjudged pause is corrected from the data afterwards.
Running and walking stop with the phone's step detector (Android's step sensor, iOS's
pedometer; its permission asked for, GPS alone without it), confirmed by GPS; cycling
by GPS speed, paused below 3 km/h for 5 s and going on above 5 km/h; rowing by the
rower's stroke rate, no stroke for 5 s. On by default for running and cycling, the
thresholds adjustable once Train has settings.
- **D17 Running, walking, hiking, cycling and indoor rowing first, in this order of
parts** (recommended): 5a the envelope moved, Train's key holder, the two models and
their migration, the chunk codec and the first synced repository; 5b recording on
Android (the service, the file taken in after a swipe, the recording screen,
auto-pause, saving); 5c the workouts and a workout's page with its track on a map
(D11's `flutter_map` and the instance's tile source brought forward from route
planning, the developer's choice: a run without its track looks unfinished), totals,
splits and charts; 5d recording on iOS; 5e the heart-rate strap; 5f the WP 5000 over
FTMS, from frames recorded with nRF Connect. Health Connect (D6) and the Loop (D3)
follow recording. Each type maps onto one of Health Connect's exercise types, and is a
name in the sealed payload, so new ones need no migration.
- **D18 The recording file is text, one sample a line, appended** (recommended, over
fixed-size binary records, smaller only while recording, as a saved workout is
delta-coded JSON anyway, and over samples handed straight to Dart, lost when Android
stops the Flutter engine while the service runs, against D13): each line a sample with
its kind (a position, a heart rate, a step, a pause's start or end) and its time,
flushed every few seconds. A line cut short is dropped as it is read; an unfinished
file the app finds when it next opens is offered to be saved up to its last sample or
thrown away. It lies in the person's folder, in the app's private storage, unsealed,
as their local database does, until it is saved as a workout and sealed (D7).
- **D19 Permissions asked when first needed, never background location**
(recommended, over background location, which would let a recording start with Train
closed and go on after Android killed its process, but asks to be allowed all the time
on a settings page of its own since Android 11 and a declaration on Google Play, more
than Train's privacy fits; OpenTracks does without it too): precise location and, on
Android 13 and later, notifications as an activity with GPS first starts, not at
install or first launch (refused notifications hide the recording's notification and
leave it running); physical activity only as running, walking or hiking first starts
with auto-pause on, for the step detector (D16), auto-pause falling back to GPS speed
for that workout where it is refused. The service is a foreground service of the types
`location` and `health`, started from the app while it is open, which keeps its access
to location with the screen off; a process Android kills leaves its file (D18) to be
found when Train next opens.
- **D20 Every fix is kept with its accuracy, and filtered as it is read**
(recommended, over dropping imprecise fixes in the service, which keeps the series
smaller and the reading simpler but can't be undone for the workouts recorded under a
threshold that proves wrong): the file and the saved series hold each fix the phone
delivered with the accuracy it reported; distance, pace, splits and the track drawn
leave out the fixes less accurate than a threshold, 20 m to start with (as OpenTracks'
default), adjustable once Train has settings. A better filter later (smoothing, a
Kalman filter) reads old workouts anew, as nothing was thrown away. An hour at a fix a
second is 3,600 fixes, tens of kilobytes delta-coded and gzipped.
- **D21 The recording screen** (proposed, the developer's): home's Record button opens a
sheet of the activities, the last one recorded chosen, with auto-pause's switch as D16
sets it; Start asks the permissions (D19). Until the first fix precise enough the
screen says it waits for GPS, showing the accuracy, while it records already, the time
from Start. Recording, it shows the moving time, the distance and the pace now (min/km
running, walking and hiking, km/h cycling; rowing time alone until 5f; heart rate once
a strap is there, 5e), an automatic pause told apart from one by hand, no map until 5c.
Stop is a long press, so that no pocket ends a run (over a tap and a confirmation,
quicker and easier to hit by mistake), then Save or Discard, Discard asked again. The
notification shows time and distance with Pause or Resume and opens the app; stopping
only in the app, so that no stray touch on the lock screen ends a workout. A recording
found cut off as Train opens (D18) is offered to be saved up to its last sample or
discarded.
Built on 28 Oct with three choices of the implementation's, for the developer to look
at: rowing indoors isn't offered yet, as a foreground service on Android 14 and later
needs a type its permissions allow and rowing without the rower has none (5f's
Bluetooth gives it `connectedDevice`); on foot without the step detector auto-pause
goes by GPS speed below 2 km/h for 5 s, going on above 3 km/h (D16 names only the
bike's thresholds); and Stop's hint shows above the button for three seconds rather
than in a snack bar, which would cover the button it speaks of. The service holds a
partial wake lock while it records (at most 12 hours), so that its clock and file run
with the screen off where no GPS fix wakes the processor.
## Consequences
- Train's sensors are one pattern: open profiles read by the app itself, on the phone;
the Loop through Polar's SDK, also on the phone.
- The recording runs in an Android foreground service from its first version, so that it
survives the screen going off.
- The Loop can't be used with Polar Flow while Train holds it; going back needs a factory
reset of the band.
- The desktop connects to no device and records nothing; it shows and plans, with what
the phones recorded coming through the server.