halkyon/packages/halkyon_base_server/README.md

halkyon_base

What every Halkyon server has, as a Serverpod module (halkyon_base_server, with its generated client halkyon_base_client): signing in (with a password and TOTP, ADR 0017, and the older ways), getting back into an account, the account's ways in and its deletion, the pruning of what nobody can return to, what an instance tells its apps before anyone signs in, and people's public profiles with their pages on the web, which the instance's register keeps (ADR 0016). It comes from Halkyon Learn's server, whose rules it keeps.

What it provides

Endpoints, under the nickname base in a host's client (client.modules.base.emailCode):

Endpoint	Methods	Who may call
anonymousIdp	login (Serverpod's anonymous sign-in, at most 30 new accounts per network and hour)	anyone
password	set: the caller's password, with the address a mailed code proves unless they have one; beginChange, change: a new password with the old one's proof and a code, every other session ending; setRecoveryKeys: a new recovery phrase's wrapping, with the old password's proof and a code	signed in
passwordSignIn	begin, complete: SRP-6a's two steps with a code from an authenticator (or a recovery code; a setup never finished, a code by mail); a session. signUp: an account at the address a mailed code proves, with a password; a session	without a session
authenticator	enrol, confirm (the first brings ten recovery codes; another takes a code from one confirmed before), list, remove, newRecoveryCodes	signed in
accountRecovery	resetAuthenticator, beginKeyProof, resetPassword (with the recovery phrase), startAccountReset, completeAccountReset	without a session
gitHubUpgrade	upgrade: links a GitHub account to the caller, who becomes identified; refused where passwords are offered (deprecated)	signed in
gitHubSignIn	signIn: a session for whoever has that GitHub account, unless they sign in with a password; none once GITHUB_SIGN_IN_ENDS has come	without a session
emailCode	sendCode: a code by mail, which proves the address and signs nobody in	anyone
account	delete, signInWays	signed in
instance	basics: the app it serves, GitHub's client ID if any and when GitHub sign-in ends, whether codes by mail and password sign-in are on, the source link, the oldest app build, where data and backups are	anyone
profile	mine, save, checkHandle, checkLinks (at most 10 an hour), setAvatar, removeAvatar, remove: the caller's own public profile	identified
profiles	byHandle: anyone's public profile, whatever the handle's case	signed in

• GitHub keeps only the numeric GitHub ID, in Serverpod's own table; email keeps only HMAC-SHA256 of the address, keyed with emailSignInPepper (email_identity), and its codes as Argon2id hashes (email_code_request). Codes are eight Crockford characters, valid 15 minutes, 3 tries; up to 3 valid at once for an address (a fourth voids the oldest, so codes others ask for don't void the one its owner is typing); at most 3 an hour per address and network, 10 per address from every network (no mailbox flooded) and 10 per network.
• Profiles: a handle as Mastodon's (1 to 30 letters, digits or underscores, unique whatever their case, a few reserved for the instance), fixed once the host says so (fixHandle; Halkyon Learn at the first publish); a display name, a bio, up to 4 links verified as Mastodon verifies them (rel="me" back to the profile's page, fetched over HTTPS from public addresses only), and an avatar, a PNG of at most 1 MB in the public storage at avatars/<SHA-256>.png. The page is at https://<account domain>/@handle (ACCOUNT_DOMAIN, else the server's public host), served by addProfilePageRoute, in English or German, its foot what the host's profilePageNote says.
• Per-network limits (new anonymous accounts, codes by mail, password sign-ins, proofs of the recovery phrase) count by the visitor's IP address (networkOf): the connection's, or, for a connection from a proxy the operator trusts (TRUSTED_PROXIES), the last address of X-Forwarded-For, the one the proxy saw itself. Never the Forwarded header, which Caddy passes on as the visitor wrote it, so never Serverpod's remoteInfo. An IPv6 visitor counts by their /64 (networkKeyOf), which one line or server holds whole. Serverpod's own limit on anonymous accounts is off (it counts by the connection's address, behind a proxy the proxy's for everyone); the endpoint counts them itself.
• Refusals are PasswordRefusedException, GitHubSignInRefusedException, EmailSignInRefusedException and ProfileRefusedException, each with a reason; ProfileNotFoundException where there is no profile.
• For the host's own code, package:halkyon_base_server/base.dart: the configuration, tokenManagerBuilders and identityProviderBuilders, the identified scope and grantIdentified, deletePerson, the pruning (pruneAbandonedAnonymousPeople, forgetOldSignUpAttempts, forgetExpiredEmailCodes, forgetUnconfirmedAuthenticators, forgetGitHubAccounts), short codes, sourceCodeUrl, dataLocation, backupLocation, people's profiles as the host's code reads them (profileOf, profilesOf, personWithHandle, fixHandle; saveProfile for its tests), accountDomain, profilePageUrl, addProfilePageRoute, the register (AccountRegister, AccountsMode, addInstanceRoutes, deliverDeletionNotices, forgetPeopleTheRegisterForgot, issueSession), and the models. A host never reads the profiles' table itself.

The instance's register (ADR 0016)

One operator's services share one account per person: the same address, GitHub account and handle reach the same person in each, under one account ID (each service's AuthUser has it), and deleting the account deletes it everywhere. One service holds the register, its database keeping the ways in (email_identity, GitHub's IDs), the handles and profiles; the others are its members and ask it. Anonymous people stay each service's own until they add a way in, which makes them an account of the register under the same ID. Sessions stay each service's own: every app signs in itself.

• Modes (AccountsMode, HalkyonBase.accounts; a host reads it with AccountsMode.fromEnvironment()): HoldsRegister by default, also for an instance of one service, telling the members INSTANCE_MEMBERS names (web servers, separated by commas or spaces) when an account goes; MemberOfRegister where ACCOUNT_REGISTER_URL names the register's web server. Empty values count as unset.
• Trust: the instance's shared secret (instanceSecret in the passwords), sent as a bearer token with every call between the services and compared in constant time. A server without one takes no such call and makes none (a member without one fails to ask anything). A member is trusted as the register's own code: whoever holds the secret can act for any account, so it never leaves the operator's servers.
• The interface: AccountRegister, which every endpoint and host function asks: LocalRegister on the register (its own tables), RemoteRegister on a member (a POST of JSON to /halkyon-base/register/<operation> on the register's web server, 30 s; refusals come back as the exceptions the register threw, anything else as RegisterUnavailableException); tests put an in-memory one in its place (AccountRegister.use). The register answers on its web server (addInstanceRoutes), a member on /halkyon-base/member/account-gone; neither is reachable from outside (the host's Caddyfile never forwards them).
• Signing in on a member: the app calls the same endpoints in both modes. The member asks the register to mail a code (the device's network passed on, as the register counts codes per network) and to check it, or to say whose GitHub ID it is, and opens a session of its own for that account (issueSession, which makes them a person there under the account's ID the first time). Linking a way in makes the person identified on the register and then on the member; a switch token signs in as the account the address is already. Every refusal is the register's, as on the register itself.
• Deleting: an identified person's deletion on a member goes to the register first (refused while it can't be reached, deleting nothing), which deletes the account and keeps a notice for each member in the same transaction (account_deletion_notice), told at once and again hourly (deliverDeletionNotices, tried least first) until each confirms; notices to a member no longer named are dropped. An anonymous person goes from their service alone. Each member also deletes, daily, the identified people the register no longer knows (forgetPeopleTheRegisterForgot): it asks about everyone before deleting anyone, deletes nobody while the register knows none of them (a register misnamed or emptied), and fails, deleting nobody, while it can't reach it. After restoring the register's database from a backup, accounts made since are unknown to it: stop the members' pruning until those people are dealt with.
• Privacy: the register learns when someone signs in to which service; the instance's privacy page has to say so.
• The register being down: members can't sign anyone in, change profiles or delete accounts; their open sessions go on working.

SRP-6a (ADR 0017)

The password never reaches the register: the device proves it knows it with SRP-6a (RFC 5054), and the register keeps only the verifier v = g^x mod N, which halkyon_keys' x makes on the device (Argon2id, so a stolen verifier costs an Argon2id per guess). The two halves share the arithmetic, generic over the group and the hash (SrpGroup):

• the register's in package:halkyon_base_server/base.dart: srpChallenge(group, verifier) gives SrpChallenge (B for the client; b kept for this sign-in only), and srpVerify(group, identity:, salt:, verifier:, challenge:, clientValue:, clientProof:) gives SrpAccepted (M2 and K) or SrpRefused (invalidClientValue, wrongProof);
• the device's in package:halkyon_base_client/srp.dart, written by hand beside the generated client (the generator writes only lib/src/protocol/): SrpClient(group) with its publicValue A, respond(identity:, salt:, privateValue:, serverValue:) giving SrpClientProof (M1, K, verifiesServer(M2)) or SrpClientRefused, and srpVerifier(group, x) for sign-up.

Production uses SrpGroup.rfc5054Group3072 (RFC 5054's 3072-bit group, g = 5) with SHA-256. k = H(N | PAD(g)), u = H(PAD(A) | PAD(B)), K = H(PAD(S)), M1 = H(H(N) xor H(g) | H(I) | s | PAD(A) | PAD(B) | K), M2 = H(PAD(A) | M1 | K), numbers as unsigned big-endian bytes, PAD to N's length. A and B must lie in 1..N−1 (so neither is ≡ 0 mod N) and u must not be 0; a and b are 256 bits from Random.secure(); proofs are compared in constant time. RFC 5054 Appendix B's vectors check both halves (SrpGroup.rfc5054TestVectorGroup, SHA-1), and test/unit/srp/srp_interop_test.dart runs the client against the register, every refusal included.

TOTP and recovery codes (ADR 0017)

The second factor, in package:halkyon_base_server/base.dart:

• newTotpSecret(): 160 random bits, shown to the person in RFC 4648 Base32 (base32Encode, base32Decode) through otpauthUri(issuer:, account:, secret:), its label and issuer escaped (a space as %20).
• checkTotp(secret:, code:, now:, lastAcceptedStep:): RFC 6238 with HMAC-SHA1, 6 digits, 30-second steps, one step either side of now. The account's last accepted step is a required argument, and a code from that step or an earlier one is refused (reused), so no code is accepted twice; TotpAccepted.step is the step to keep before answering. Refusals: malformed, wrong, reused. totpCode makes a code (SHA-256 and SHA-512 for RFC 6238's vectors).
• RecoveryCodes.generate(key): ten codes of eight Crockford characters (K7QX-M2PD, as email sign-in's codes), shown once, kept as HMAC-SHA256 under the instance's key (recoveryCodeKey, at least 32 characters, kept outside the database: a code's 40 bits in a leaked database give nothing to guess against without it); useRecoveryCode(typed, hashes, key) compares every hash in constant time and gives the hashes left (RecoveryCodeAccepted.remaining), so each code works once.

Signing in with a password (ADR 0017)

Built on the three parts above, on the register (members forward every step through AccountRegister, RemoteRegister included; the session is the member's own).

• Tables (serverOnly, each row deleted with its person by deletePerson): account_password (one per account: Argon2id's salt and costs, SRP's verifier padded to N, the public keys, the private keys wrapped under the password and under the recovery phrase), totp_authenticator (the secret sealed with XChaCha20-Poly1305 under a key derived from totpKey, bound to the row's account and ID; confirmedAt; lastStep), recovery_code (keyed hashes), sign_in_challenge (a step begun: SRP's b and B, or a nonce to sign; two minutes, taken once by a delete), account_reset (one of each kind per account, with when it takes effect).
• Setting a password (password.set): the device sends salt, costs (argon2id13, 3–16 passes, 64 MiB–1 GiB), verifier (1 < v < N, as long as N), public keys (32 bytes each) and both wrappings (24-byte nonce, 112 bytes). It needs an address: the caller's, or the one a mailed code proves, which becomes theirs as email sign-in makes it (anonymous → identified, same ID). It signs in once an authenticator is confirmed; until then a new setup replaces it, never once one was, even while a reset leaves the account without one (a new password comes through the recovery phrase or the account's reset alone).
• Codes by mail (emailCode.sendCode, without a session) only prove an address: for a password's setup, a sign-up, a setup never finished, the resets and deleting an account without a working second factor. Each is asked for a purpose (EmailCodePurpose: signUp, finishSetup, authenticatorReset, accountReset, deletion; email_code_request.purpose), only that purpose takes it (any other is refused as no longer valid and leaves it unused), and its mail says what it would do, in its subject too (a notification shows the subject alone), and to type it only in the app on that instance. Since step 3 nothing turns a code into a session (emailCode.verifyCode, emailSignIn.signIn and the switch token went with build 22, minimumAppBuild 22 in Learn).
• Signing up (passwordSignIn.signUp(request, code, setup), without a session): an address nobody has becomes a new identified account, an account with the address and no password takes it (keeping its ID); refused (passwordExists) for an address whose account has a password. The session it gives sets up the authenticator.
• Signing in: begin(address) answers salt, costs and B; complete(challenge, A, M1, totpCode | recoveryCode) answers M2, the public keys, the password's wrapping and a session. A setup never finished (never a confirmed authenticator) signs in with a code mailed to the address it began with in its place (request, code), which calls off a reset of the account waiting; any other account takes no code by mail there and leaves it unused. SRP's identity I is passwordIdentity: the challenge's ID and the host the device addressed (host on every proving call: in production it must be the server's apiServer.publicHost, else otherInstance; in development any, as an emulator reaches the machine as 10.0.2.2), so a hostile instance relaying a sign-in or a phrase's proof here gets nothing taken; a member forwards its own host, and the phrase's proof for a new password signs that password's digest (newPasswordDigest). An address nobody has (or whose account has no password) gets a salt and verifier from signInDecoyKey and the address's hash, the same queries and arithmetic, and the same refusal: a wrong password, an unknown address and a wrong, used or missing code are all wrongCredentials; setupIncomplete and notDue come only after a right password. At most 10 sign-ins begun per address and network an hour (5 proofs of the phrase), 30 per network, and, from every network together, 30 proofs checked per address (proofsPerAddress: every password and phrase proof, recorded before it is checked whatever its outcome, so a right one counts as a wrong one and challenges begun ahead get no more checked; an address nobody has alike), past which no proof is checked and nothing begins for it anywhere until the hour has passed: so whoever knows an address and fills its count from one network holds no other back, and guessing from many makes at most 30 guesses an hour. A signed-in owner's proofs (a change challenge, below) never count toward the address, and no sign-in takes one. A sign-in or the phrase's reset carrying a code takes the account's address and language, refused (wrongCredentials) before any code is checked when it isn't theirs.
• The owner's proof (OwnerProof, ADR 0017's third reading, as Proton has it): a session alone never gets a code checked. Every signed-in change of how an account is secured (adding an authenticator once one is confirmed, removing a confirmed one, new recovery codes, a new phrase, a new password, deleting) takes the password's SRP proof for a challenge password.beginChange gave (30 begun an hour per session, holding back no other) and then a TOTP or recovery code (proveOwner, ownersCode). A proof given where none is taken is refused, so none passes unchecked. Each proof counts for its session alone (owner_proof_tries.dart, asOwnerProofTry, table session_code_failures, kept where the session is, on a member too), before it is checked; the tenth wrong in a row ends the session (sessionEnded), only a proof accepted clears the count, and a try that checked none is taken back. Never toward the account's waits, nor waiting for them: whoever knows the password holds no signed-in owner out of changing it, and a new password forgets those waits.
• Authenticators: a secret made on the register, shown once as an otpauth:// URI (issuer: the account domain); confirmed by a code from it; while the account has a confirmed one, another counts only with the owner's proof too, so a session alone adds no second factor; confirming one confirmed already is refused as unknown (its code is never checked for a session); the first confirmed brings ten recovery codes; at most 10 per account; the last confirmed one stays; removing a confirmed one, or new recovery codes, takes the owner's proof, removing one never confirmed nothing. Each step's code is accepted once, by a conditional update of lastStep.
• Codes tried in a row (second_factor_waits.dart, table second_factor_failures): every code tried against an account's second factor after a right password or phrase at sign-in or in the phrase's reset (never the owner's proofs while signed in) counts, per account, from every network, before it is checked, one at a time per account (an advisory lock), so codes sent at once get no more checked; every tenth in a row begins a wait as it is counted (codesWait, with when), an hour, then twice as long each time, a day at the most, and every wait mails the account's address (which those calls take); a code accepted, either reset carried out, and a new password clear it.
• Resets: a lost authenticator: SRP's proof plus a mailed code or the recovery phrase's signature asks for it; it takes effect 72 hours later, a mail tells the address at once, and signing in with a code calls it off; then the password alone signs in until a new authenticator is confirmed. A lost password with the phrase: a signature over a nonce (keyProofMessage) by the phrase's own proof key (Ed25519, derived from the phrase alone, its public half kept as recoveryProofKey; never the account's signing key, which every signed-in device keeps) and a code set a new password, the keys staying. A lost password without it: a mailed code asks, 72 hours pass (a mail at the start, a sign-in calls it off), a new code carries it out: new password, new keys, no authenticators.
• Changing while signed in: password.change takes the owner's proof (the old password's SRP proof for a password.beginChange challenge, and a TOTP or recovery code) and the same keys wrapped under the new password; password.setRecoveryKeys replaces the phrase's wrapping with the same proof (a phrase of one's own would otherwise set a new password through the phrase's reset). A session alone changes neither, nor a session and a code.
• Deleting an account with a password (account.delete(proof:), a DeletionProof) takes the same: the password's proof for a beginChange challenge and a code, bound to the host the device addressed (requireOwnerToDelete; a member passes the proof on and the register checks it), counted for the session as every owner's proof is. The code is a TOTP or recovery code wherever an authenticator is confirmed; where no second factor works (a setup never finished, or the authenticators' reset taken effect; SignInWays.secondFactorWorks tells the app) it may be one by mail asked for deleting (request, code), to the account's own address. An account without a password deletes with its session, as before.
• Sessions end everywhere: a change of password ends every other session of the account; a reset of the password or the account ends all. The register ends its own and keeps a sessions_ended_notice per member (told at once and hourly until it confirms, deliverSessionNotices, as deletions are told): it names a moment, so a session made since stays, and the session that asked for a change. A member answering one (/halkyon-base/member/sessions-ended) ends its sessions made before the moment; a member signing someone in after a reset ends its own first and makes the new session only once its clock is past the moment (issueSessionAfter), refusing with passwordChanged (sign in again with the new password) when its clock is more than maxClockSkew behind. A refusal for a reset not due says when it takes effect (takesEffectAt).
• Older ways: an account that signs in with a password (it had a confirmed authenticator, which a reset may have taken since: passwordOnly in its ways in) no longer signs in with GitHub (usePassword); where passwords are offered, no new GitHub account is linked (deprecated); from GITHUB_SIGN_IN_ENDS on GitHub signs nobody in (ended), the basics stop naming the day (gitHubSignInEnds, which apps show until then) and forgetGitHubAccounts deletes every GitHub ID.

Taking it into a host

1. Depend on both packages by path, within the repository's workspace (an app's packages sit in apps/<app>/):

   # the server's pubspec.yaml
   halkyon_base_server:
     path: ../../../packages/halkyon_base_server
   # the client's pubspec.yaml: the same with halkyon_base_client
   

2. Name it in the server's config/generator.yaml:

   modules:
     halkyon_base:
       nickname: base
   

3. Configure it once, before the pod starts, and run its authentication and pruning:

   final pod = Serverpod(args); // the host's generated Serverpod
   HalkyonBase.configure(
     pod,
     HalkyonBase(
       app: 'todo', // its apps accept only instances naming it; never change it
       appName: 'Halkyon Todo', // in sign-in mails
       upstreamSourceCodeUrl: 'https://code.halkyon.app/project-halkyon/halkyon_todo',
       minimumAppBuild: 1, // raise it in the commit that breaks older apps
       deletePersonData: [deleteTodosOf], // the host's own data of a person
       profilePageNote: ( // what profile pages say of the app at their foot
         english: 'On Halkyon Todo, a to-do list:',
         german: 'Auf Halkyon Todo, einer Aufgabenliste:',
         link: Uri.parse('https://halkyon.app/todo'),
       ),
       // The register, or a member of the one the operator names (ADR 0016).
       accounts: AccountsMode.fromEnvironment(),
     ),
   );
   pod.initializeAuthServices(
     tokenManagerBuilders: tokenManagerBuilders,
     identityProviderBuilders: identityProviderBuilders(pod),
   );
   

   Serve the calls between the instance's services on its web server: addInstanceRoutes(pod); and the profile pages: addProfilePageRoute(pod), after any literal routes of its own.

   Then schedule the pruning in a future call of the host's own: daily pruneAbandonedAnonymousPeople, forgetGitHubAccounts and forgetPeopleTheRegisterForgot (a member's), hourly forgetOldSignUpAttempts, forgetExpiredEmailCodes, forgetUnconfirmedAuthenticators and deliverDeletionNotices (the register's). Each does nothing in the other mode.

4. A host's deletion (PersonDataDeletion) runs inside deletePerson's transaction, after the person's sessions ended and before their identities and the person go, in the order given; it deletes the person's rows in the host's tables, and returns what has to wait until the deletion is committed (files outside the database), or null. Pruning deletes through the same function.

5. Create the host's migration. A host taking the tables over from its own models (as Halkyon Learn does) finds no change once its copies are gone: Serverpod matches tables by name, and these are identical. create-migration --empty then records the module's version, with no SQL besides. Every service has the module's tables, whichever its mode: a member's stay empty but for its people's sessions.

The passwords it reads: serverSideSessionKeyHashPepper; for GitHub githubClientId and githubClientSecret; for email emailSignInPepper and emailSmtpHost, emailSmtpPort, emailSmtpUser, emailSmtpPassword, emailFrom; for passwords totpKey, recoveryCodeKey and signInDecoyKey (each at least 32 characters, or addInstanceRoutes stops the server; all three and email sign-in, or no password sign-in); between the instance's services instanceSecret. Never change either pepper once people sign in: every session or address hash would be lost; changing totpKey voids every authenticator, recoveryCodeKey every recovery code. A member needs no email settings of its own (the register mails the codes), but GitHub's (its own OAuth app's, whose callback is its app's). The environment: ACCOUNT_REGISTER_URL and INSTANCE_MEMBERS (above); ACCOUNT_DOMAIN, the domain handles are on, which can never change once anyone follows one; GITHUB_SIGN_IN_ENDS, the day GitHub stops signing anyone in (2027-01-22); SOURCE_URL, DATA_LOCATION, BACKUP_LOCATION: where the instance keeps people's data and its backups, each EU, a country's code (DE) or a subdivision's (ISO 3166-2, US-CA, DE-BY), the backups also none; where the disks are, not which law governs; TRUSTED_PROXIES, the addresses of the reverse proxies in front of the server (Caddy's on its compose network, which Halkyon Learn's deploy/compose.yaml sets), separated by commas or spaces: without, every visitor behind the proxy shares each per-network limit; anything but addresses stops the server (addInstanceRoutes).

Developing it

serverpod --no-analytics generate here after changing a model or an endpoint, and serverpod --no-analytics create-migration after changing a table. Tests need no Docker: dart test here starts Serverpod's embedded PostgreSQL. They read config/passwords.yaml (git-ignored) with throwaway values under test: for database, serviceSecret, serverSideSessionKeyHashPepper and emailSignInPepper. Password sign-in's keys may be there too (CI's are), but the tests set their own (usePasswordKeys) and switch them off otherwise.

Licences

The server package is under the Server Side Public License 1.0, as the Halkyon servers are; the client package under the GNU Affero General Public License 3.0 or later, as the Halkyon apps are. So each combines with its counterpart in a host exactly as the host's own do.