halkyon/packages/halkyon_base_server/README.md

# halkyon_base

What every Halkyon server has, as a Serverpod module (`halkyon_base_server`, with its
generated client `halkyon_base_client`): signing in (with a password and TOTP, ADR 0017,
and the older ways), getting back into an account, the account's ways in and its
deletion, the pruning of what nobody can return to, what an instance tells its apps
before anyone signs in, and people's public profiles with their pages on the web, which
the instance's register keeps (ADR 0016). It comes from Halkyon Learn's server, whose
rules it keeps.

## What it provides

Endpoints, under the nickname `base` in a host's client (`client.modules.base.emailCode`):

| Endpoint | Methods | Who may call |
|---|---|---|
| `anonymousIdp` | `login` (Serverpod's anonymous sign-in, at most 30 new accounts per network and hour) | anyone |
| `password` | `set`: the caller's password, with the address a mailed code proves unless they have one | signed in |
| `passwordSignIn` | `begin`, `complete`: SRP-6a's two steps with a code from an authenticator (or a recovery code); a session | without a session |
| `authenticator` | `enrol`, `confirm` (the first brings ten recovery codes; another takes a code from one confirmed before), `list`, `remove`, `newRecoveryCodes` | signed in |
| `accountRecovery` | `sendCode`, `resetAuthenticator`, `beginKeyProof`, `resetPassword` (with the recovery phrase), `startAccountReset`, `completeAccountReset` | without a session |
| `gitHubUpgrade` | `upgrade`: links a GitHub account to the caller, who becomes `identified`; refused where passwords are offered (deprecated) | signed in |
| `gitHubSignIn` | `signIn`: a session for whoever has that GitHub account, unless they sign in with a password; none once `GITHUB_SIGN_IN_ENDS` has come | without a session |
| `emailCode` | `sendCode`, `verifyCode`: a code by mail; a new address becomes the caller's, another person's gives a switch token | signed in |
| `emailSignIn` | `signIn`: a session for the person a switch token names | without a session |
| `account` | `delete`, `signInWays` | signed in |
| `instance` | `basics`: the app it serves, GitHub's client ID if any, whether email and password sign-in are on, the source link, the oldest app build, where data and backups are | anyone |
| `profile` | `mine`, `save`, `checkHandle`, `checkLinks` (at most 10 an hour), `setAvatar`, `removeAvatar`, `remove`: the caller's own public profile | `identified` |
| `profiles` | `byHandle`: anyone's public profile, whatever the handle's case | signed in |

- GitHub keeps only the numeric GitHub ID, in Serverpod's own table; email keeps only
  HMAC-SHA256 of the address, keyed with `emailSignInPepper` (`email_identity`), and its
  codes as Argon2id hashes (`email_code_request`). Codes are eight Crockford characters,
  valid 15 minutes, 3 tries; at most 3 an hour per address and 10 per network.
- Profiles: a handle as Mastodon's (1 to 30 letters, digits or underscores, unique
  whatever their case, a few reserved for the instance), fixed once the host says so
  (`fixHandle`; Halkyon Learn at the first publish); a display name, a bio, up to 4
  links verified as Mastodon verifies them (`rel="me"` back to the profile's page,
  fetched over HTTPS from public addresses only), and an avatar, a PNG of at most 1 MB
  in the public storage at `avatars/<SHA-256>.png`. The page is at
  `https://<account domain>/@handle` (`ACCOUNT_DOMAIN`, else the server's public host),
  served by `addProfilePageRoute`, in English or German, its foot what the host's
  `profilePageNote` says.
- **Per-network limits** (new anonymous accounts, codes by mail, password sign-ins,
  proofs of the recovery phrase) count by the visitor's IP address (`networkOf`): the
  connection's, or, for a connection from a proxy the operator trusts
  (`TRUSTED_PROXIES`), the last address of `X-Forwarded-For`, the one the proxy saw
  itself. Never the `Forwarded` header, which Caddy passes on as the visitor wrote it,
  so never Serverpod's `remoteInfo`. Serverpod's own limit on anonymous accounts is off
  (it counts by the connection's address, behind a proxy the proxy's for everyone); the
  endpoint counts them itself.
- Refusals are `PasswordRefusedException`, `GitHubSignInRefusedException`,
  `EmailSignInRefusedException` and `ProfileRefusedException`, each with a reason;
  `ProfileNotFoundException` where there is no profile.
- For the host's own code, `package:halkyon_base_server/base.dart`: the configuration,
  `tokenManagerBuilders` and `identityProviderBuilders`, the `identified` scope and
  `grantIdentified`, `deletePerson`, the pruning (`pruneAbandonedAnonymousPeople`,
  `forgetOldSignUpAttempts`, `forgetExpiredEmailCodes`, `forgetUnconfirmedAuthenticators`,
  `forgetGitHubAccounts`), short codes, `sourceCodeUrl`,
  `dataLocation`, `backupLocation`, people's profiles as the host's code reads them
  (`profileOf`, `profilesOf`, `personWithHandle`, `fixHandle`; `saveProfile` for its
  tests), `accountDomain`, `profilePageUrl`, `addProfilePageRoute`, the register
  (`AccountRegister`, `AccountsMode`, `addInstanceRoutes`, `deliverDeletionNotices`,
  `forgetPeopleTheRegisterForgot`, `issueSession`), and the models. A host never reads
  the profiles' table itself.

## The instance's register (ADR 0016)

One operator's services share one account per person: the same address, GitHub account
and handle reach the same person in each, under one account ID (each service's
`AuthUser` has it), and deleting the account deletes it everywhere. One service holds
the register, its database keeping the ways in (`email_identity`, GitHub's IDs), the
handles and profiles; the others are its members and ask it. Anonymous people stay each
service's own until they add a way in, which makes them an account of the register under
the same ID. Sessions stay each service's own: every app signs in itself.

- **Modes** (`AccountsMode`, `HalkyonBase.accounts`; a host reads it with
  `AccountsMode.fromEnvironment()`): `HoldsRegister` by default, also for an instance of
  one service, telling the members `INSTANCE_MEMBERS` names (web servers, separated by
  commas or spaces) when an account goes; `MemberOfRegister` where
  `ACCOUNT_REGISTER_URL` names the register's web server. Empty values count as unset.
- **Trust:** the instance's shared secret (`instanceSecret` in the passwords), sent as a
  bearer token with every call between the services and compared in constant time. A
  server without one takes no such call and makes none (a member without one fails to
  ask anything). A member is trusted as the register's own code: whoever holds the secret
  can act for any account, so it never leaves the operator's servers.
- **The interface:** `AccountRegister`, which every endpoint and host function asks:
  `LocalRegister` on the register (its own tables), `RemoteRegister` on a member (a POST
  of JSON to `/halkyon-base/register/<operation>` on the register's web server, 30 s;
  refusals come back as the exceptions the register threw, anything else as
  `RegisterUnavailableException`); tests put an in-memory one in its place
  (`AccountRegister.use`). The register answers on its web server (`addInstanceRoutes`),
  a member on `/halkyon-base/member/account-gone`; neither is reachable from outside (the
  host's Caddyfile never forwards them).
- **Signing in on a member:** the app calls the same endpoints in both modes. The member
  asks the register to mail a code (the device's network passed on, as the register
  counts codes per network) and to check it, or to say whose GitHub ID it is, and opens a
  session of its own for that account (`issueSession`, which makes them a person there
  under the account's ID the first time). Linking a way in makes the person identified on
  the register and then on the member; a switch token signs in as the account the
  address is already. Every refusal is the register's, as on the register itself.
- **Deleting:** an identified person's deletion on a member goes to the register first
  (refused while it can't be reached, deleting nothing), which deletes the account and
  keeps a notice for each member in the same transaction (`account_deletion_notice`),
  told at once and again hourly (`deliverDeletionNotices`, tried least first) until each
  confirms; notices to a member no longer named are dropped. An anonymous person goes
  from their service alone. Each member also deletes, daily, the identified people the
  register no longer knows (`forgetPeopleTheRegisterForgot`): it asks about everyone
  before deleting anyone, deletes nobody while the register knows none of them (a
  register misnamed or emptied), and fails, deleting nobody, while it can't reach it.
  After restoring the register's database from a backup, accounts made since are
  unknown to it: stop the members' pruning until those people are dealt with.
- **Privacy:** the register learns when someone signs in to which service; the
  instance's privacy page has to say so.
- **The register being down:** members can't sign anyone in, change profiles or delete
  accounts; their open sessions go on working.

## SRP-6a (ADR 0017)

The password never reaches the register: the device proves it knows it with SRP-6a
(RFC 5054), and the register keeps only the verifier v = g^x mod N, which `halkyon_keys`'
x makes on the device (Argon2id, so a stolen verifier costs an Argon2id per guess). The
two halves share the arithmetic, generic over the group and the hash (`SrpGroup`):

- the register's in `package:halkyon_base_server/base.dart`: `srpChallenge(group,
  verifier)` gives `SrpChallenge` (B for the client; b kept for this sign-in only), and
  `srpVerify(group, identity:, salt:, verifier:, challenge:, clientValue:, clientProof:)`
  gives `SrpAccepted` (M2 and K) or `SrpRefused` (`invalidClientValue`, `wrongProof`);
- the device's in `package:halkyon_base_client/srp.dart`, written by hand beside the
  generated client (the generator writes only `lib/src/protocol/`): `SrpClient(group)`
  with its `publicValue` A, `respond(identity:, salt:, privateValue:, serverValue:)`
  giving `SrpClientProof` (M1, K, `verifiesServer(M2)`) or `SrpClientRefused`, and
  `srpVerifier(group, x)` for sign-up.

Production uses `SrpGroup.rfc5054Group3072` (RFC 5054's 3072-bit group, g = 5) with
SHA-256. k = H(N | PAD(g)), u = H(PAD(A) | PAD(B)), K = H(PAD(S)),
M1 = H(H(N) xor H(g) | H(I) | s | PAD(A) | PAD(B) | K), M2 = H(PAD(A) | M1 | K), numbers as
unsigned big-endian bytes, PAD to N's length. A and B must lie in 1..N−1 (so neither is
≡ 0 mod N) and u must not be 0; a and b are 256 bits from `Random.secure()`; proofs are
compared in constant time. RFC 5054 Appendix B's vectors check both halves
(`SrpGroup.rfc5054TestVectorGroup`, SHA-1), and `test/unit/srp/srp_interop_test.dart` runs
the client against the register, every refusal included.

## TOTP and recovery codes (ADR 0017)

The second factor, in `package:halkyon_base_server/base.dart`:

- `newTotpSecret()`: 160 random bits, shown to the person in RFC 4648 Base32
  (`base32Encode`, `base32Decode`) through `otpauthUri(issuer:, account:, secret:)`, its
  label and issuer escaped (a space as `%20`).
- `checkTotp(secret:, code:, now:, lastAcceptedStep:)`: RFC 6238 with HMAC-SHA1, 6 digits,
  30-second steps, one step either side of now. The account's last accepted step is a
  required argument, and a code from that step or an earlier one is refused (`reused`),
  so no code is accepted twice; `TotpAccepted.step` is the step to keep before answering.
  Refusals: `malformed`, `wrong`, `reused`. `totpCode` makes a code (SHA-256 and SHA-512
  for RFC 6238's vectors).
- `RecoveryCodes.generate(key)`: ten codes of eight Crockford characters (`K7QX-M2PD`, as
  email sign-in's codes), shown once, kept as HMAC-SHA256 under the instance's key
  (`recoveryCodeKey`, at least 32 characters, kept outside the database: a code's 40 bits
  in a leaked database give nothing to guess against without it); `useRecoveryCode(typed,
  hashes, key)` compares every hash in constant time and gives the hashes left
  (`RecoveryCodeAccepted.remaining`), so each code works once.

## Signing in with a password (ADR 0017)

Built on the three parts above, on the register (members forward every step through
`AccountRegister`, `RemoteRegister` included; the session is the member's own).

- **Tables** (`serverOnly`, each row deleted with its person by `deletePerson`):
  `account_password` (one per account: Argon2id's salt and costs, SRP's verifier padded
  to N, the public keys, the private keys wrapped under the password and under the
  recovery phrase), `totp_authenticator` (the secret sealed with XChaCha20-Poly1305 under
  a key derived from `totpKey`, bound to the row's account and ID; `confirmedAt`;
  `lastStep`), `recovery_code` (keyed hashes), `sign_in_challenge` (a step begun: SRP's b
  and B, or a nonce to sign; two minutes, taken once by a delete), `account_reset` (one
  of each kind per account, with when it takes effect).
- **Setting a password** (`password.set`): the device sends salt, costs (`argon2id13`,
  3–16 passes, 64 MiB–1 GiB), verifier (1 < v < N, as long as N), public keys (32 bytes
  each) and both wrappings (24-byte nonce, 112 bytes). It needs an address: the
  caller's, or the one a mailed code proves, which becomes theirs as email sign-in makes
  it (anonymous → `identified`, same ID). It signs in once an authenticator is
  confirmed; until then a new setup replaces it.
- **Signing in:** `begin(address)` answers salt, costs and B; `complete(challenge, A, M1,
  totpCode | recoveryCode)` answers M2, the public keys, the password's wrapping and a
  session. SRP's identity I is the challenge's ID. An address nobody has (or whose
  account has no password) gets a salt and verifier from `signInDecoyKey` and the
  address's hash, the same queries and arithmetic, and the same refusal: a wrong
  password, an unknown address and a wrong, used or missing code are all
  `wrongCredentials`; `setupIncomplete` and `notDue` come only after a right password.
  At most 10 sign-ins begun per address and 30 per network an hour.
- **Authenticators:** a secret made on the register, shown once as an `otpauth://` URI
  (issuer: the account domain); confirmed by a code; while the account has a confirmed
  one, another counts only with a code from that one too (`wrongExistingCode`), so a
  session alone adds no second factor; the first confirmed brings ten recovery codes; at most 10 per account; the last confirmed one stays; removing a
  confirmed one, or new recovery codes, takes a code (10 tries an hour per account).
  Each step's code is accepted once, by a conditional update of `lastStep`.
- **Resets:** a lost authenticator: SRP's proof plus a mailed code or the recovery
  phrase's signature asks for it; it takes effect 72 hours later, a mail tells the
  address at once, and signing in with a code calls it off; then the password alone
  signs in until a new authenticator is confirmed. A lost password with the phrase: the
  account's Ed25519 signature over a nonce (`keyProofMessage`) and a code set a new
  password, the keys staying. A lost password without it: a mailed code asks, 72 hours
  pass (a mail at the start, a sign-in calls it off), a new code carries it out: new
  password, new keys, no authenticators. Either change of password ends the account's
  sessions on the register's server.
- **Older ways:** an account that signs in with a password (and a confirmed
  authenticator) no longer signs in by a mailed code (`usePassword`) or GitHub
  (`usePassword`); where passwords are offered, no new GitHub account is linked
  (`deprecated`); from `GITHUB_SIGN_IN_ENDS` on GitHub signs nobody in (`ended`) and
  `forgetGitHubAccounts` deletes every GitHub ID.

## Taking it into a host

1. Depend on both packages by path, within the repository's workspace (an app's
   packages sit in `apps/<app>/`):

   ```yaml
   # the server's pubspec.yaml
   halkyon_base_server:
     path: ../../../packages/halkyon_base_server
   # the client's pubspec.yaml: the same with halkyon_base_client
   ```

2. Name it in the server's `config/generator.yaml`:

   ```yaml
   modules:
     halkyon_base:
       nickname: base
   ```

3. Configure it once, before the pod starts, and run its authentication and pruning:

   ```dart
   final pod = Serverpod(args); // the host's generated Serverpod
   HalkyonBase.configure(
     pod,
     HalkyonBase(
       app: 'todo', // its apps accept only instances naming it; never change it
       appName: 'Halkyon Todo', // in sign-in mails
       upstreamSourceCodeUrl: 'https://code.halkyon.app/project-halkyon/halkyon_todo',
       minimumAppBuild: 1, // raise it in the commit that breaks older apps
       deletePersonData: [deleteTodosOf], // the host's own data of a person
       profilePageNote: ( // what profile pages say of the app at their foot
         english: 'On Halkyon Todo, a to-do list:',
         german: 'Auf Halkyon Todo, einer Aufgabenliste:',
         link: Uri.parse('https://halkyon.app/todo'),
       ),
       // The register, or a member of the one the operator names (ADR 0016).
       accounts: AccountsMode.fromEnvironment(),
     ),
   );
   pod.initializeAuthServices(
     tokenManagerBuilders: tokenManagerBuilders,
     identityProviderBuilders: identityProviderBuilders(pod),
   );
   ```

   Serve the calls between the instance's services on its web server:
   `addInstanceRoutes(pod)`; and the profile pages: `addProfilePageRoute(pod)`, after any
   literal routes of its own.

   Then schedule the pruning in a future call of the host's own: daily
   `pruneAbandonedAnonymousPeople`, `forgetGitHubAccounts` and
   `forgetPeopleTheRegisterForgot` (a member's), hourly `forgetOldSignUpAttempts`,
   `forgetExpiredEmailCodes`, `forgetUnconfirmedAuthenticators` and
   `deliverDeletionNotices` (the register's). Each does nothing in the other mode.

4. A host's deletion (`PersonDataDeletion`) runs inside `deletePerson`'s transaction,
   after the person's sessions ended and before their identities and the person go, in
   the order given; it deletes the person's rows in the host's tables, and returns what
   has to wait until the deletion is committed (files outside the database), or null.
   Pruning deletes through the same function.

5. Create the host's migration. A host taking the tables over from its own models (as
   Halkyon Learn does) finds no change once its copies are gone: Serverpod matches tables
   by name, and these are identical. `create-migration --empty` then records the module's
   version, with no SQL besides. Every service has the module's tables, whichever its
   mode: a member's stay empty but for its people's sessions.

The passwords it reads: `serverSideSessionKeyHashPepper`; for GitHub `githubClientId` and
`githubClientSecret`; for email `emailSignInPepper` and `emailSmtpHost`,
`emailSmtpPort`, `emailSmtpUser`, `emailSmtpPassword`, `emailFrom`; for passwords
`totpKey`, `recoveryCodeKey` and `signInDecoyKey` (each at least 32 characters, or
`addInstanceRoutes` stops the server; all three and email sign-in, or no password
sign-in); between the instance's services `instanceSecret`. Never change either pepper
once people sign in: every session or address hash would be lost; changing `totpKey`
voids every authenticator, `recoveryCodeKey` every recovery code. A member needs no email settings of its
own (the register mails the codes), but GitHub's (its own OAuth app's, whose callback is
its app's). The environment: `ACCOUNT_REGISTER_URL` and `INSTANCE_MEMBERS` (above);
`ACCOUNT_DOMAIN`, the domain handles are on, which can never change once anyone follows
one; `GITHUB_SIGN_IN_ENDS`, the day GitHub stops signing anyone in (`2027-01-22`);
`SOURCE_URL`, `DATA_LOCATION`, `BACKUP_LOCATION`: where the instance keeps people's data
and its backups, each `EU`, a country's code (`DE`) or a subdivision's (ISO 3166-2,
`US-CA`, `DE-BY`), the backups also `none`; where the disks are, not which law governs;
`TRUSTED_PROXIES`, the addresses of the reverse proxies in front of the server (Caddy's
on its compose network, which Halkyon Learn's `deploy/compose.yaml` sets), separated by
commas or spaces: without, every visitor behind the proxy shares each per-network limit;
anything but addresses stops the server (`addInstanceRoutes`).

## Developing it

`serverpod --no-analytics generate` here after changing a model or an endpoint, and
`serverpod --no-analytics create-migration` after changing a table. Tests need no Docker:
`dart test` here starts Serverpod's embedded PostgreSQL. They read `config/passwords.yaml`
(git-ignored) with throwaway values under `test:` for `database`, `serviceSecret`,
`serverSideSessionKeyHashPepper` and `emailSignInPepper`. Password sign-in's keys may be
there too (CI's are), but the tests set their own (`usePasswordKeys`) and switch them
off otherwise.

## Licences

The server package is under the Server Side Public License 1.0, as the Halkyon servers
are; the client package under the GNU Affero General Public License 3.0 or later, as the
Halkyon apps are. So each combines with its counterpart in a host exactly as the host's
own do.