halkyon/docs/adr/0021-private-instance-and-assistant.md

ADR 0021: A private instance: approved sign-ups and an assistant the operator grants

• Status: proposed 23 Oct 2026. Decided by the developer on 23 Oct: D1–D6 below; the rest under Proposal is for review before building.
• Builds on: ADR 0016 (the register), ADR 0019 (private studysets), ADR 0020 (SimpleLogin), ADR 0022 (the hand-off and the connector gone).
• Changes: the rule of 10 Oct 2026 that Halkyon calls no language model with an API key, for an instance whose operator opts in.

Context

The developer may stop the public instance and run halkyon.li for themselves and their close family instead. They don't know in advance which addresses their family will sign in with, and with SimpleLogin (ADR 0020) the instance keeps no addresses anyway.

On 10 Oct the developer ruled out API keys for language models: on a public instance anyone could drive up the bill, and each person should decide what a model sees. Assistants came through people's own subscriptions instead: the connector (ADR 0006) and the hand-off to Claude or Lumo, both gone since 23 Oct (ADR 0022). A family instance whose operator approves every account takes away the first reason; the second holds if the app shows what it sends.

People in one family may prefer different providers (the developer: "I might use myself the Claude API while other might want to use the OpenAI API, then one other the Proton Lumo API"). A plan for Claude's or ChatGPT's apps is no API access: the APIs are billed apart, by their providers' consoles.

Two protocols reach nearly every provider: Anthropic's Messages API (Claude), and the OpenAI-compatible chat completions (OpenAI, Mistral, Google Gemini's compatibility endpoint, OpenRouter, local models through Ollama, llama.cpp or vLLM). Proton announced a Lumo API for autumn or winter 2026 without publishing its format; community projects (lumo-tamer, an unofficial tracker) show an OpenAI-compatible interface.

Decisions (23 Oct 2026, the developer's)

• D1 Sign-ups wait for the operator's approval (over invitation links, which were recommended, and a sign-up window): anyone can sign up; nothing works until the operator approves them.
• D2 The assistant runs on the instance's key, granted per person (recommended, over each person's own key and both): the operator pays and switches it on for whom they choose.
• D3 Individual limits and statistics (the developer: "so I can see after the fact who potentially drove up the bill"): each person's use is limited and counted, and the operator sees it per person.
• D4 Several providers, all on the operator's keys (over personal keys alone, and over both, which was recommended): the operator sets up the assistants the instance offers and grants each person one or more.
• D5 Personal keys stay on the person's devices (recommended, over keeping them on the instance): the app calls the provider itself, and the instance sees neither key nor text.
• D6 Personal keys only where the operator allows them (over every instance, which was recommended): an instance setting, off unless switched on.

Proposal

1. Who signs up, per instance: SIGN_UP: open (as today), approval, or closed. On halkyon.li approval.
2. The operator is a role on the register. The first one: operatorSetupCode in the passwords; the sign-up that gives it becomes the operator, approved at once, and the code counts once. A rule by handle would let a stranger who signs up first take the operator's handle. The operator can name others.
3. Waiting accounts (D1): after signing up (SimpleLogin, handle, keys: ADR 0020), an account waits. Its sessions can only see that they wait, or delete the account; nothing syncs and nothing uploads. The operator sees who waits in the app (the handle and when they signed up, which is all the instance knows) and approves or refuses; a refusal deletes the account. Unapproved after 14 days, it is deleted. At most 10 waiting at once, and sign-ups per network limited as today's, so strangers can't flood the list. The operator's app shows a badge; there is no mail, as the instance keeps no addresses.
4. The assistants, set by the operator (D4): one or more, each with an ID, the name people see ("Claude Sonnet"), its protocol (anthropic, or openai for every OpenAI-compatible service), its base URL (the provider's own unless set) and model in deploy/.env (ASSISTANTS=claude,gpt, then ASSISTANT_CLAUDE_NAME, _PROTOCOL, _BASE_URL, _MODEL), and its key in the passwords (ASSISTANT_CLAUDE_KEY). There rather than in the app: the keys stay out of the database and its hourly backups, where the operator keeps the instance's other secrets. None set: no assistant on the instance, as today. Adding Lumo once its API is out, or switching a model, is a change of these and a restart; no app changes.
5. Two adapters of our own behind one interface, over plain HTTP, no SDKs, in a pure Dart package of their own, which the server uses for the operator's assistants and the app for personal keys (10). Features use only what both protocols share: a system prompt and messages, streamed text, and tool calls where a feature needs them. Tested against recorded replies of each. Not chosen: LiteLLM as a sidecar (a Python proxy for about 100 providers: one more container to run and update, for providers never used) and langchain_dart (a large framework for a small job).
6. Grants (D2, D4): nobody has an assistant until the operator grants it to them, one assistant at a time, each grant with its limits. The person agrees once to what goes where: the provider's name, that the instance sees the text on its way, and that the operator sees how much they used, never what.
7. Asking: the field offers the assistants granted beside the web search (the one last used first), the app shows what it will send, the person sends it to the instance (assistant.ask, the answer streamed), the instance checks the grant and what is left of the limit, forwards the request, and counts what the provider's reply reports (input and output tokens). Past the limit it refuses (limitReached), saying when it resets. A private studyset (ADR 0019) goes only as far as the person sends it, decrypted on the device for that request.
8. Limits and statistics (D3): per grant, per day and per month (a day's limit stops a runaway loop early). Counted per person, day and assistant: requests, input and output tokens, and the cost at the prices the operator enters per model (per million tokens; without prices, tokens only, and limits in tokens). Neither prompts nor answers are kept. The operator sees everyone's figures in the app, per person and month; each person sees their own.
9. The privacy page names the provider, says what is sent and that the instance sees it in passing, that the operator sees each person's figures, and that no content is kept.
10. Personal keys (D5, D6), where the operator switches them on (ASSISTANT_OWN_KEYS, off unless on; instance.info() says so, and the app offers them only then): a person adds a provider in Settings (protocol, base URL, model, their key), and may add several. The key goes into their settings, which are sealed with their personal key (ADR 0019), so it reaches their other devices and the instance can't read it. The app calls the provider directly with the same adapters (5); the instance sees neither key nor text and counts nothing, and the person's limits are their provider's. The instance can't stop a modified app from doing so, and needn't: nothing passes through it. Personal assistants are offered in the field beside those granted.

Consequences

• The operator pays the provider, and sees who used how much, never what for.
• The instance sees what is asked on its way to the provider; the provider's own terms then apply to it.
• A provider that is down stops the assistants on it; the others granted, personal keys and the web search go on.
• Strangers can sign up and wait, but can't use anything, and their accounts lapse.
• Public instances keep the rule of 10 Oct: their operators leave the assistants unset, and may allow personal keys.