feat(base,learn,todo): the operator's assistants, and the instance's privacy page (ADR 0021, step 2)
The server half of the assistants (points 4 to 8, D27 to D46): the operator sets language models in deploy/assistants.env, grants them to people with limits, and the module forwards their questions with the operator's key, counting what each used, never what was asked. - assistantsFrom: ASSISTANTS, ASSISTANT_ID_KEY, ASSISTANT_<ID>_NAME, _PROTOCOL (anthropic, openai, openai-responses), _BASE_URL, _MODEL, _KEY, _IMAGES, _PRICE_INPUT and _PRICE_OUTPUT (per million tokens), ASSISTANT_CURRENCY, ASSISTANT_OWN_KEYS; anything unusable stops the server as it starts, naming the variable. - The assistant endpoint (identified): offered, agree (D40), ask as a Serverpod stream (D35): the text as the model writes it, then the whole turn, sent once the request is counted; figures, the caller's own. Checked in order: at most 2 questions on their way per person (tooManyAtOnce), the grant, the agreement, limits in the pricing's unit now (limitOutdated), the month's and the day's limits (with when they reset, UTC), the question (malformed, imagesNotRead, tooLarge past 400,000 characters, an image over 3,932,160 bytes or more than 20 images); output capped at 16,384 tokens. Each request carries the person's opaque ID (HMAC-SHA256 under ASSISTANT_ID_KEY, D30). Once sent, the provider's reply is read to its end even if the app cancels, and counted on a session of its own; a provider's failure is logged by kind and status, told as providerKey, providerBusy (with its wait), providerRefused or providerFailed. - The assistantOperator endpoint (identified and operator, on the register): setUp, grants, grant (assistantNotSet, invalidLimit, notFound), revoke, figures of a UTC month. - Tables assistant_grant and assistant_figures (per person, assistant and UTC day), in the module's, Learn's and Todo's migrations. The daily pruning keeps this month and the two before (forgetOldAssistantFigures, D41); deletePerson deletes both. A member offers no assistant (D43). - instance.basics() gains assistantOwnKeys. - /privacy (D39, addPrivacyPageRoute, on the register's web server, before the profile pages' route): made from the instance's settings, English or German, no JavaScript: who may sign up, where the data and backups are, what the module keeps (the account, sessions, IP addresses, the server's log), the host's privacyPagePoints (Learn's: studysets synced, private ones encrypted on the devices with what stays readable, what is published, dictionary lookups, the learner count), each assistant with its model and host, what goes and what is counted, and whether personal keys are allowed. Never a key. Tests: 70 on the assistants (grants, limits, refusals in order, streaming, cancelling, counting, concurrency, the operator's denials, deletion, pruning), 19 on the settings, 8 on the privacy page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
038aca5439
commit
dbae802915
56 changed files with 56851 additions and 33 deletions
|
|
@ -4,7 +4,8 @@
|
|||
/// basics, people's public profiles and their pages, the instance's register
|
||||
/// (ADR 0016), signing in with a password and TOTP and getting back into an
|
||||
/// account (ADR 0017), signing in with SimpleLogin and its forwarding page
|
||||
/// (ADR 0020), the ways in an operator offers, and the module's models.
|
||||
/// (ADR 0020), the ways in an operator offers, the operator's assistants
|
||||
/// (Halkyon Learn's ADR 0021), and the module's models.
|
||||
library;
|
||||
|
||||
export 'src/admission/admission.dart'
|
||||
|
|
@ -15,6 +16,16 @@ export 'src/admission/admission.dart'
|
|||
unapprovedSignUpLifetime,
|
||||
waitingPassLifetime;
|
||||
export 'src/admission/sign_up_config.dart';
|
||||
export 'src/assistant/assistant_conversation.dart'
|
||||
show
|
||||
assistantImageMaxBytes,
|
||||
assistantQuestionMaxCharacters,
|
||||
assistantQuestionMaxImages;
|
||||
export 'src/assistant/assistant_forwarding.dart'
|
||||
show assistantUsageCounted, maxQuestionsAtOnce, useLlmClient;
|
||||
export 'src/assistant/assistant_grants.dart'
|
||||
show assistantFiguresMonths, forgetOldAssistantFigures;
|
||||
export 'src/assistant/assistants_config.dart';
|
||||
export 'src/auth/account.dart';
|
||||
export 'src/auth/auth_config.dart';
|
||||
export 'src/auth/code_mail.dart';
|
||||
|
|
@ -27,6 +38,7 @@ export 'src/common/short_code.dart';
|
|||
export 'src/generated/protocol.dart' hide Protocol;
|
||||
export 'src/halkyon_base.dart';
|
||||
export 'src/instance/data_location.dart';
|
||||
export 'src/instance/privacy_page.dart' show addPrivacyPageRoute, privacyPage;
|
||||
export 'src/instance/source_code.dart';
|
||||
export 'src/password/account_resets.dart'
|
||||
show forgetLapsedResets, resetDelay, resetLapsesAfter;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,15 @@
|
|||
### A piece of an assistant's answer, as it streams (Halkyon Learn's ADR
|
||||
### 0021, D35): text as the model writes it, then the whole turn.
|
||||
class: AssistantAnswer
|
||||
fields:
|
||||
### More of the text; null in the last piece.
|
||||
delta: String?
|
||||
### The model's whole turn, to be sent back with the next question; only
|
||||
### in the last piece.
|
||||
message: AssistantMessage?
|
||||
### Why it stopped; only in the last piece.
|
||||
stop: AssistantStop?
|
||||
### The input tokens counted; only in the last piece.
|
||||
inputTokens: int?
|
||||
### The output tokens counted; only in the last piece.
|
||||
outputTokens: int?
|
||||
|
|
@ -0,0 +1,248 @@
|
|||
import 'dart:convert';
|
||||
import 'dart:typed_data';
|
||||
|
||||
import 'package:halkyon_base_server/src/assistant/assistants_config.dart';
|
||||
import 'package:halkyon_base_server/src/generated/protocol.dart';
|
||||
import 'package:halkyon_llm/halkyon_llm.dart';
|
||||
|
||||
// A question as the app sends it, as halkyon_llm's request, and the model's
|
||||
// turn back as the app keeps it (Halkyon Learn's ADR 0021, point 7): every
|
||||
// part checked here, so a question the providers would refuse is refused
|
||||
// before anything goes.
|
||||
|
||||
/// The most characters a question holds, all its text together (the system
|
||||
/// prompt, the turns, the tools' results and calls, the tools' schemas):
|
||||
/// about 100,000 tokens, which bounds what one question costs, as the
|
||||
/// output's ceiling bounds the answer (D38).
|
||||
const assistantQuestionMaxCharacters = 400000;
|
||||
|
||||
/// The most bytes of one image: 5 MiB once in base64, the most Anthropic
|
||||
/// takes of one.
|
||||
const assistantImageMaxBytes = 3932160;
|
||||
|
||||
/// The most images in one question.
|
||||
const assistantQuestionMaxImages = 20;
|
||||
|
||||
/// The image types every protocol's providers take.
|
||||
const _imageTypes = {'image/jpeg', 'image/png', 'image/gif', 'image/webp'};
|
||||
|
||||
/// A tool's name as every protocol takes it.
|
||||
final _toolName = RegExp(r'^[a-zA-Z0-9_-]{1,64}$');
|
||||
|
||||
Never _refused(AssistantRefusal refusal) =>
|
||||
throw AssistantRefusedException(refusal: refusal);
|
||||
|
||||
/// The request [question] stands for, to [assistant], with [personId] as the
|
||||
/// provider's opaque ID of the person (D30). Refused
|
||||
/// ([AssistantRefusedException]) for a turn out of place or a part without
|
||||
/// what its kind needs ([AssistantRefusal.malformed]), an image to an
|
||||
/// assistant that reads none ([AssistantRefusal.imagesNotRead]) and a
|
||||
/// question past the limits above ([AssistantRefusal.tooLarge]).
|
||||
LlmRequest requestOf(
|
||||
AssistantQuestion question,
|
||||
AssistantSetting assistant, {
|
||||
required String personId,
|
||||
}) {
|
||||
var characters = question.system.length;
|
||||
var images = 0;
|
||||
final messages = <LlmMessage>[];
|
||||
for (final message in question.messages) {
|
||||
if (message.parts.isEmpty) {
|
||||
_refused(AssistantRefusal.malformed);
|
||||
}
|
||||
final parts = <LlmPart>[];
|
||||
for (final part in message.parts) {
|
||||
final converted = _partOf(part, message.role);
|
||||
switch (converted) {
|
||||
case LlmImage(:final bytes):
|
||||
if (!assistant.images) {
|
||||
_refused(AssistantRefusal.imagesNotRead);
|
||||
}
|
||||
if (bytes.length > assistantImageMaxBytes ||
|
||||
++images > assistantQuestionMaxImages) {
|
||||
_refused(AssistantRefusal.tooLarge);
|
||||
}
|
||||
case _:
|
||||
characters +=
|
||||
(part.text?.length ?? 0) +
|
||||
(part.json?.length ?? 0) +
|
||||
(part.extra?.length ?? 0);
|
||||
}
|
||||
parts.add(converted);
|
||||
}
|
||||
messages.add(switch (message.role) {
|
||||
AssistantRole.user => LlmMessage.user(parts),
|
||||
AssistantRole.assistant => LlmMessage.assistant(parts),
|
||||
});
|
||||
}
|
||||
if (messages.isEmpty || messages.last.role != LlmRole.user) {
|
||||
_refused(AssistantRefusal.malformed);
|
||||
}
|
||||
final tools = [
|
||||
for (final tool in question.tools)
|
||||
if (_toolName.hasMatch(tool.name))
|
||||
LlmTool(
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
inputSchema: _object(tool.inputSchema),
|
||||
)
|
||||
else
|
||||
_refused(AssistantRefusal.malformed),
|
||||
];
|
||||
for (final tool in question.tools) {
|
||||
characters += tool.description.length + tool.inputSchema.length;
|
||||
}
|
||||
if (characters > assistantQuestionMaxCharacters) {
|
||||
_refused(AssistantRefusal.tooLarge);
|
||||
}
|
||||
return LlmRequest(
|
||||
system: question.system,
|
||||
messages: messages,
|
||||
tools: tools,
|
||||
maxOutputTokens: question.maxOutputTokens.clamp(1, assistantOutputCeiling),
|
||||
personId: personId,
|
||||
);
|
||||
}
|
||||
|
||||
/// [part] of a turn of [role]'s, as halkyon_llm holds it: a person's turn
|
||||
/// holds text, images and tools' results; the model's, text, tool calls and
|
||||
/// its provider's blocks.
|
||||
LlmPart _partOf(AssistantPart part, AssistantRole role) {
|
||||
final byPerson = role == AssistantRole.user;
|
||||
switch (part) {
|
||||
case AssistantPart(kind: AssistantPartKind.text, :final text?):
|
||||
return LlmText(text, extra: _objectOrNone(part.extra));
|
||||
case AssistantPart(
|
||||
kind: AssistantPartKind.image,
|
||||
:final mediaType?,
|
||||
:final bytes?,
|
||||
)
|
||||
when byPerson && _imageTypes.contains(mediaType):
|
||||
return LlmImage(
|
||||
mediaType: mediaType,
|
||||
bytes: Uint8List.sublistView(bytes),
|
||||
);
|
||||
case AssistantPart(
|
||||
kind: AssistantPartKind.toolResult,
|
||||
:final callId?,
|
||||
:final text?,
|
||||
)
|
||||
when byPerson && callId.isNotEmpty:
|
||||
return LlmToolResult(
|
||||
callId: callId,
|
||||
content: text,
|
||||
isError: part.isError ?? false,
|
||||
);
|
||||
case AssistantPart(
|
||||
kind: AssistantPartKind.toolCall,
|
||||
:final callId?,
|
||||
:final name?,
|
||||
:final json?,
|
||||
)
|
||||
when !byPerson && callId.isNotEmpty && name.isNotEmpty:
|
||||
return LlmToolCall(
|
||||
id: callId,
|
||||
name: name,
|
||||
input: _object(json),
|
||||
extra: _objectOrNone(part.extra),
|
||||
);
|
||||
case AssistantPart(
|
||||
kind: AssistantPartKind.providerBlock,
|
||||
:final protocol?,
|
||||
:final json?,
|
||||
)
|
||||
when !byPerson:
|
||||
return LlmProviderBlock(
|
||||
protocol:
|
||||
LlmProtocol.named(protocol) ?? _refused(AssistantRefusal.malformed),
|
||||
block: _object(json),
|
||||
);
|
||||
case _:
|
||||
_refused(AssistantRefusal.malformed);
|
||||
}
|
||||
}
|
||||
|
||||
/// The JSON object [json] holds; refused as malformed for anything else.
|
||||
Map<String, Object?> _object(String json) {
|
||||
try {
|
||||
if (jsonDecode(json) case final Map<String, Object?> object) {
|
||||
return object;
|
||||
}
|
||||
} on FormatException {
|
||||
// Below.
|
||||
}
|
||||
_refused(AssistantRefusal.malformed);
|
||||
}
|
||||
|
||||
Map<String, Object?> _objectOrNone(String? json) =>
|
||||
json == null ? const {} : _object(json);
|
||||
|
||||
/// The model's turn [message], as the app keeps it and sends it back.
|
||||
AssistantMessage messageOf(LlmMessage message) => AssistantMessage(
|
||||
role: switch (message.role) {
|
||||
LlmRole.user => AssistantRole.user,
|
||||
LlmRole.assistant => AssistantRole.assistant,
|
||||
},
|
||||
parts: [
|
||||
for (final part in message.parts)
|
||||
switch (part) {
|
||||
LlmText(:final text, :final extra) => AssistantPart(
|
||||
kind: AssistantPartKind.text,
|
||||
text: text,
|
||||
extra: extra.isEmpty ? null : jsonEncode(extra),
|
||||
),
|
||||
LlmImage(:final mediaType, :final bytes) => AssistantPart(
|
||||
kind: AssistantPartKind.image,
|
||||
mediaType: mediaType,
|
||||
bytes: ByteData.sublistView(bytes),
|
||||
),
|
||||
LlmToolCall(:final id, :final name, :final input, :final extra) =>
|
||||
AssistantPart(
|
||||
kind: AssistantPartKind.toolCall,
|
||||
callId: id,
|
||||
name: name,
|
||||
json: jsonEncode(input),
|
||||
extra: extra.isEmpty ? null : jsonEncode(extra),
|
||||
),
|
||||
LlmToolResult(:final callId, :final content, :final isError) =>
|
||||
AssistantPart(
|
||||
kind: AssistantPartKind.toolResult,
|
||||
callId: callId,
|
||||
text: content,
|
||||
isError: isError,
|
||||
),
|
||||
LlmProviderBlock(:final protocol, :final block) => AssistantPart(
|
||||
kind: AssistantPartKind.providerBlock,
|
||||
protocol: protocol.key,
|
||||
json: jsonEncode(block),
|
||||
),
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
/// Why the model stopped, as the app reads it.
|
||||
AssistantStop stopOf(LlmStop stop) => switch (stop) {
|
||||
LlmStop.end => AssistantStop.end,
|
||||
LlmStop.maxTokens => AssistantStop.maxTokens,
|
||||
LlmStop.toolCalls => AssistantStop.toolCalls,
|
||||
LlmStop.refused => AssistantStop.refused,
|
||||
LlmStop.other => AssistantStop.other,
|
||||
};
|
||||
|
||||
/// What a provider's [failure] tells the person: the key's
|
||||
/// ([AssistantRefusal.providerKey]), busy for now
|
||||
/// ([AssistantRefusal.providerBusy], with its wait), the question refused
|
||||
/// ([AssistantRefusal.providerRefused]), or failed.
|
||||
AssistantRefusedException refusalOf(LlmFailure failure) =>
|
||||
AssistantRefusedException(
|
||||
refusal: switch (failure.kind) {
|
||||
LlmFailureKind.key => AssistantRefusal.providerKey,
|
||||
LlmFailureKind.rateLimited ||
|
||||
LlmFailureKind.unavailable => AssistantRefusal.providerBusy,
|
||||
LlmFailureKind.request => AssistantRefusal.providerRefused,
|
||||
LlmFailureKind.provider ||
|
||||
LlmFailureKind.network ||
|
||||
LlmFailureKind.malformed => AssistantRefusal.providerFailed,
|
||||
},
|
||||
retryAfterSeconds: failure.retryAfter?.inSeconds,
|
||||
);
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
import 'package:halkyon_base_server/src/assistant/assistant_forwarding.dart';
|
||||
import 'package:halkyon_base_server/src/assistant/assistant_grants.dart';
|
||||
import 'package:halkyon_base_server/src/auth/scopes.dart';
|
||||
import 'package:halkyon_base_server/src/generated/protocol.dart';
|
||||
import 'package:serverpod/serverpod.dart';
|
||||
import 'package:serverpod_auth_idp_server/core.dart';
|
||||
|
||||
/// The operator's assistants as a person uses them (Halkyon Learn's ADR
|
||||
/// 0021, points 6 to 9): those granted to them, their agreement, their
|
||||
/// questions with the answers streamed (D35), and their own figures.
|
||||
/// Refusals are `AssistantRefusedException`s.
|
||||
class AssistantEndpoint extends Endpoint {
|
||||
@override
|
||||
Set<Scope> get requiredScopes => {identified};
|
||||
|
||||
/// The assistants granted to the caller that are set, with what goes
|
||||
/// where and what is left of their limits, and whether they may add their
|
||||
/// own keys; none on a member (D43).
|
||||
Future<AssistantsOffered> offered(Session session) =>
|
||||
assistantsOffered(session, _caller(session));
|
||||
|
||||
/// The caller agrees to what goes where with [assistant], its provider as
|
||||
/// it is now (D40); refused without a grant of it
|
||||
/// (`AssistantRefusal.notGranted`).
|
||||
Future<void> agree(Session session, {required String assistant}) =>
|
||||
agreeToAssistant(session, _caller(session), assistant);
|
||||
|
||||
/// Forwards [question] to its assistant and streams the answer: the text
|
||||
/// as the model writes it, then the whole turn with what was counted.
|
||||
/// Refused before anything goes without a grant, an agreement or room
|
||||
/// within the limits, for a question that can't be sent, and with more
|
||||
/// than `maxQuestionsAtOnce` on their way; then for the provider's
|
||||
/// failure. Cancelled, the answer is still read to its end and counted.
|
||||
Stream<AssistantAnswer> ask(
|
||||
Session session, {
|
||||
required AssistantQuestion question,
|
||||
}) => askAssistant(session, _caller(session), question);
|
||||
|
||||
/// The caller's own figures, month by month, the latest first.
|
||||
Future<List<AssistantMonthFigures>> figures(Session session) =>
|
||||
ownAssistantFigures(session, _caller(session));
|
||||
}
|
||||
|
||||
UuidValue _caller(Session session) => session.authenticated!.authUserId;
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
### What one person used of one assistant on one UTC day (Halkyon Learn's
|
||||
### ADR 0021, D3, point 8, D41 and D42): never what they asked, nor the
|
||||
### answer. Kept 3 months, deleted with the person. Kept by the register
|
||||
### alone.
|
||||
class: AssistantFigures
|
||||
serverOnly: true
|
||||
table: assistant_figures
|
||||
fields:
|
||||
### The person: their ID (deleted with them, `deletePerson`).
|
||||
authUserId: UuidValue
|
||||
### The assistant's ID.
|
||||
assistant: String
|
||||
### The UTC day, at its midnight.
|
||||
day: DateTime
|
||||
### The requests forwarded.
|
||||
requests: int
|
||||
### The input tokens the provider counted.
|
||||
inputTokens: int
|
||||
### The output tokens the provider counted, thinking included.
|
||||
outputTokens: int
|
||||
### What it cost at the prices of the time, in millionths of the
|
||||
### currency; 0 where the assistant had no prices.
|
||||
costMicros: int
|
||||
indexes:
|
||||
### One row per person, assistant and day, which each request adds to.
|
||||
assistant_figures_day_idx:
|
||||
fields: authUserId, assistant, day
|
||||
unique: true
|
||||
### The days pruned after 3 months.
|
||||
assistant_figures_age_idx:
|
||||
fields: day
|
||||
|
|
@ -0,0 +1,218 @@
|
|||
import 'dart:async';
|
||||
|
||||
import 'package:clock/clock.dart';
|
||||
import 'package:halkyon_base_server/src/assistant/assistant_conversation.dart';
|
||||
import 'package:halkyon_base_server/src/assistant/assistant_grants.dart';
|
||||
import 'package:halkyon_base_server/src/assistant/assistants_config.dart';
|
||||
import 'package:halkyon_base_server/src/generated/protocol.dart';
|
||||
import 'package:halkyon_llm/halkyon_llm.dart';
|
||||
import 'package:meta/meta.dart';
|
||||
import 'package:serverpod/serverpod.dart';
|
||||
|
||||
// A question forwarded to the operator's assistant and its answer streamed
|
||||
// back (Halkyon Learn's ADR 0021, point 7, D35): checked first, then sent
|
||||
// with the operator's key and the person's opaque ID (D30), and counted once
|
||||
// the provider's reply ends (D38), on a session of its own.
|
||||
|
||||
/// The most questions one person has on their way at once: a modified app
|
||||
/// asking many at once would pass a limit with all of them, as each is
|
||||
/// checked against what was counted before it.
|
||||
const maxQuestionsAtOnce = 2;
|
||||
|
||||
/// [pod]'s client of the providers: the one tests put in place
|
||||
/// ([useLlmClient]), or one over HTTP for the process.
|
||||
LlmClient llmClientOf(Serverpod pod) => switch (_clientOf[pod]) {
|
||||
(:final client?) => client,
|
||||
_ => _http ??= LlmClient(),
|
||||
};
|
||||
|
||||
LlmClient? _http;
|
||||
|
||||
/// Puts [client] in place of the providers on [pod], null meaning HTTP's
|
||||
/// again: tests do.
|
||||
void useLlmClient(Serverpod pod, LlmClient? client) =>
|
||||
_clientOf[pod] = (client: client);
|
||||
|
||||
final _clientOf = Expando<({LlmClient? client})>('LLM client');
|
||||
|
||||
/// The questions on their way, per person, by the account's ID.
|
||||
final _asking = <String, int>{};
|
||||
|
||||
/// The forwardings under way, until counted.
|
||||
final _forwardings = <Future<void>>{};
|
||||
|
||||
/// Waits for the forwardings under way to be counted: tests do.
|
||||
@visibleForTesting
|
||||
Future<void> assistantUsageCounted() => Future.wait([..._forwardings]);
|
||||
|
||||
/// [person]'s [question], answered as the model writes: the text in
|
||||
/// pieces, then the whole turn with what was counted. Refusals
|
||||
/// ([AssistantRefusedException]) end the stream as its error, before
|
||||
/// anything is sent (`admittedAssistant`, `requestOf`) or for the
|
||||
/// provider's failure (`refusalOf`), which the server logs.
|
||||
///
|
||||
/// Once sent, the provider's reply is read to its end whatever the person
|
||||
/// does, and counted then: cancelled half-way, an answer would still be
|
||||
/// billed, and a provider that counts at the end (as OpenAI's chat
|
||||
/// completions do) would otherwise have counted nothing. The last piece
|
||||
/// comes once it is counted, so the next question of the app's loop is
|
||||
/// checked against it.
|
||||
Stream<AssistantAnswer> askAssistant(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
AssistantQuestion question,
|
||||
) {
|
||||
var listening = true;
|
||||
late final StreamController<AssistantAnswer> answers;
|
||||
answers = StreamController(
|
||||
onListen: () {
|
||||
final forwarding = _forward(
|
||||
session,
|
||||
person,
|
||||
question,
|
||||
answers,
|
||||
isListening: () => listening,
|
||||
);
|
||||
_forwardings.add(forwarding);
|
||||
unawaited(forwarding.whenComplete(() => _forwardings.remove(forwarding)));
|
||||
},
|
||||
onCancel: () => listening = false,
|
||||
);
|
||||
return answers.stream;
|
||||
}
|
||||
|
||||
Future<void> _forward(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
AssistantQuestion question,
|
||||
StreamController<AssistantAnswer> answers, {
|
||||
required bool Function() isListening,
|
||||
}) async {
|
||||
final key = person.uuid;
|
||||
if ((_asking[key] ?? 0) >= maxQuestionsAtOnce) {
|
||||
answers.addError(
|
||||
AssistantRefusedException(refusal: AssistantRefusal.tooManyAtOnce),
|
||||
);
|
||||
unawaited(answers.close());
|
||||
return;
|
||||
}
|
||||
_asking[key] = (_asking[key] ?? 0) + 1;
|
||||
try {
|
||||
final pod = session.serverpod;
|
||||
final AssistantSetting assistant;
|
||||
final LlmRequest request;
|
||||
try {
|
||||
assistant = await admittedAssistant(
|
||||
session,
|
||||
person,
|
||||
question.assistant,
|
||||
);
|
||||
request = requestOf(
|
||||
question,
|
||||
assistant,
|
||||
personId: assistantsOf(pod).personIdOf(person),
|
||||
);
|
||||
} on AssistantRefusedException catch (refusal) {
|
||||
answers.addError(refusal);
|
||||
return;
|
||||
}
|
||||
LlmFinished? finished;
|
||||
LlmFailure? failure;
|
||||
try {
|
||||
await for (final event in llmClientOf(
|
||||
pod,
|
||||
).send(assistant.endpoint, request)) {
|
||||
switch (event) {
|
||||
case LlmTextDelta(:final text):
|
||||
if (isListening()) {
|
||||
answers.add(AssistantAnswer(delta: text));
|
||||
}
|
||||
case final LlmFinished last:
|
||||
finished = last;
|
||||
}
|
||||
}
|
||||
} on LlmFailure catch (error) {
|
||||
failure = error;
|
||||
}
|
||||
await _count(
|
||||
pod,
|
||||
person,
|
||||
assistant,
|
||||
finished?.usage ?? failure?.usage ?? LlmUsage.none,
|
||||
failure: failure,
|
||||
);
|
||||
if (!isListening()) {
|
||||
return;
|
||||
}
|
||||
switch ((finished, failure)) {
|
||||
case (final last?, _):
|
||||
answers.add(
|
||||
AssistantAnswer(
|
||||
message: messageOf(last.message),
|
||||
stop: stopOf(last.stop),
|
||||
inputTokens: last.usage.inputTokens,
|
||||
outputTokens: last.usage.outputTokens,
|
||||
),
|
||||
);
|
||||
case (null, final failed?):
|
||||
answers.addError(refusalOf(failed));
|
||||
case (null, null):
|
||||
answers.addError(
|
||||
AssistantRefusedException(refusal: AssistantRefusal.providerFailed),
|
||||
);
|
||||
}
|
||||
} on Object catch (error, st) {
|
||||
// Anything else (the database out of reach) as Serverpod handles an
|
||||
// endpoint's error: logged, and the app told of an internal error.
|
||||
answers.addError(error, st);
|
||||
} finally {
|
||||
final left = (_asking[key] ?? 1) - 1;
|
||||
if (left > 0) {
|
||||
_asking[key] = left;
|
||||
} else {
|
||||
_asking.remove(key);
|
||||
}
|
||||
// Not waited for: once the person cancelled, it would never be done.
|
||||
unawaited(answers.close());
|
||||
}
|
||||
}
|
||||
|
||||
/// Counts one request on a session of its own, as the person's may have
|
||||
/// closed with their cancelling, and logs the provider's [failure] there:
|
||||
/// its kind, status and words, which name no person and quote no question
|
||||
/// (halkyon_llm cuts them, and keys from them). A failure to count is
|
||||
/// logged, never told the person, whose answer stands.
|
||||
Future<void> _count(
|
||||
Serverpod pod,
|
||||
UuidValue person,
|
||||
AssistantSetting assistant,
|
||||
LlmUsage usage, {
|
||||
required LlmFailure? failure,
|
||||
}) async {
|
||||
final session = await pod.createSession();
|
||||
try {
|
||||
if (failure != null) {
|
||||
session.log(
|
||||
'Assistant ${assistant.id} failed: ${failure.kind.name}'
|
||||
'${failure.status == null ? '' : ' (${failure.status})'}: '
|
||||
'${failure.detail}',
|
||||
level: LogLevel.warning,
|
||||
);
|
||||
}
|
||||
await countAssistantRequest(
|
||||
session,
|
||||
person: person,
|
||||
assistant: assistant,
|
||||
inputTokens: usage.inputTokens,
|
||||
outputTokens: usage.outputTokens,
|
||||
now: clock.now(),
|
||||
);
|
||||
} on Object catch (error) {
|
||||
session.log(
|
||||
'Counting a request to ${assistant.id} failed: ${error.runtimeType}',
|
||||
level: LogLevel.warning,
|
||||
);
|
||||
} finally {
|
||||
await session.close();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
### An assistant an operator granted a person (Halkyon Learn's ADR 0021, D2,
|
||||
### D4 and point 6), with its limits, and whether they agreed to what goes
|
||||
### where (D40). Kept by the register alone.
|
||||
class: AssistantGrant
|
||||
serverOnly: true
|
||||
table: assistant_grant
|
||||
fields:
|
||||
### The person: their ID (deleted with them, `deletePerson`).
|
||||
authUserId: UuidValue
|
||||
### The assistant's ID, as the operator set it (`ASSISTANTS`). A grant of
|
||||
### one no longer set stays, unused, until it is set again (D44).
|
||||
assistant: String
|
||||
### When it was granted, or its limits last changed.
|
||||
grantedAt: DateTime
|
||||
### What the limits count: the cost where the assistant had prices when
|
||||
### granted, else tokens. Should its prices come or go, the grant holds
|
||||
### no more until the operator sets its limits again.
|
||||
limitUnit: AssistantLimitUnit
|
||||
### The most a UTC day may use (D42), in [limitUnit]; null for no limit
|
||||
### per day.
|
||||
dailyLimit: int?
|
||||
### The most a UTC month may use, in [limitUnit]; null for no limit per
|
||||
### month.
|
||||
monthlyLimit: int?
|
||||
### When the person agreed to what goes where; null until they have.
|
||||
agreedAt: DateTime?
|
||||
### What they agreed to: the protocol and the provider's host
|
||||
### (`anthropic api.anthropic.com`). Moved to another provider, the
|
||||
### assistant asks again.
|
||||
agreedTo: String?
|
||||
indexes:
|
||||
### One grant per person and assistant.
|
||||
assistant_grant_person_idx:
|
||||
fields: authUserId, assistant
|
||||
unique: true
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
### A grant as the operator sees it (Halkyon Learn's ADR 0021, point 6).
|
||||
class: AssistantGrantOverview
|
||||
fields:
|
||||
### The person's account.
|
||||
account: UuidValue
|
||||
### Their handle; null for an account without one.
|
||||
handle: String?
|
||||
### The assistant's ID.
|
||||
assistant: String
|
||||
### Whether the instance still sets that assistant (D44).
|
||||
assistantSet: bool
|
||||
### When it was granted, or its limits last changed.
|
||||
grantedAt: DateTime
|
||||
### What its limits count.
|
||||
limitUnit: AssistantLimitUnit
|
||||
### The most a UTC day may use; null for none.
|
||||
dailyLimit: int?
|
||||
### The most a UTC month may use; null for none.
|
||||
monthlyLimit: int?
|
||||
### When the person agreed to the assistant's provider as it is now;
|
||||
### null until they have.
|
||||
agreedAt: DateTime?
|
||||
### Whether the limits count what the assistant counts no more, as its
|
||||
### prices came or went.
|
||||
outdated: bool
|
||||
|
|
@ -0,0 +1,571 @@
|
|||
import 'package:clock/clock.dart';
|
||||
import 'package:halkyon_base_server/src/assistant/assistants_config.dart';
|
||||
import 'package:halkyon_base_server/src/auth/membership.dart';
|
||||
import 'package:halkyon_base_server/src/generated/protocol.dart';
|
||||
import 'package:serverpod/serverpod.dart';
|
||||
import 'package:serverpod_auth_idp_server/core.dart';
|
||||
|
||||
// The operator's assistants granted to people, their limits, and what each
|
||||
// person used (Halkyon Learn's ADR 0021, points 6 and 8, D38 to D44): on the
|
||||
// register alone, as accounts are; a member offers none (D43). The figures
|
||||
// are counted per person, assistant and UTC day, never what was asked.
|
||||
|
||||
/// How many UTC months of figures are kept (D41): this one and the two
|
||||
/// before it, so last month's are there all this month.
|
||||
const assistantFiguresMonths = 3;
|
||||
|
||||
Never _refused(AssistantRefusal refusal, {DateTime? resetsAt}) =>
|
||||
throw AssistantRefusedException(refusal: refusal, resetsAt: resetsAt);
|
||||
|
||||
Never _refusedForOperator(OperatorRefusal reason) =>
|
||||
throw OperatorRefusedException(reason: reason);
|
||||
|
||||
/// What [assistant]'s limits count as it is set now: its cost where it has
|
||||
/// prices, else tokens.
|
||||
AssistantLimitUnit limitUnitOf(AssistantSetting assistant) =>
|
||||
assistant.prices == null
|
||||
? AssistantLimitUnit.tokens
|
||||
: AssistantLimitUnit.costMicros;
|
||||
|
||||
/// What a person agrees to of [assistant] (D40): its protocol and the
|
||||
/// provider's host (`anthropic api.anthropic.com`), not its model.
|
||||
String providerOf(AssistantSetting assistant) =>
|
||||
'${assistant.endpoint.protocol.key} ${assistant.endpoint.host}';
|
||||
|
||||
/// The UTC day [time] is in, at its midnight (D42).
|
||||
DateTime utcDayOf(DateTime time) {
|
||||
final utc = time.toUtc();
|
||||
return DateTime.utc(utc.year, utc.month, utc.day);
|
||||
}
|
||||
|
||||
/// The UTC month [time] is in, at midnight of its first day.
|
||||
DateTime utcMonthOf(DateTime time) {
|
||||
final utc = time.toUtc();
|
||||
return DateTime.utc(utc.year, utc.month);
|
||||
}
|
||||
|
||||
/// The assistant [id] as this service offers it: none on a member (D43),
|
||||
/// nor where the operator doesn't set it (D44).
|
||||
AssistantSetting? _offered(Serverpod pod, String id) =>
|
||||
isMember(pod) ? null : assistantsOf(pod).named(id);
|
||||
|
||||
/// [person]'s grant of [assistant]; null for none.
|
||||
Future<AssistantGrant?> _grantOf(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
String assistant, {
|
||||
Transaction? transaction,
|
||||
}) =>
|
||||
// SELECT … FROM "assistant_grant"
|
||||
// WHERE "authUserId" = $1 AND "assistant" = $2
|
||||
AssistantGrant.db.findFirstRow(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(person) & t.assistant.equals(assistant),
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
/// What [person] used of [assistant] on [now]'s UTC day and in its month,
|
||||
/// in [unit].
|
||||
Future<({int today, int thisMonth})> _usedOf(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
String assistant,
|
||||
AssistantLimitUnit unit,
|
||||
DateTime now,
|
||||
) async {
|
||||
final day = utcDayOf(now);
|
||||
// SELECT … FROM "assistant_figures" WHERE "authUserId" = $1
|
||||
// AND "assistant" = $2 AND "day" >= $3
|
||||
final days = await AssistantFigures.db.find(
|
||||
session,
|
||||
where: (t) =>
|
||||
t.authUserId.equals(person) &
|
||||
t.assistant.equals(assistant) &
|
||||
(t.day >= utcMonthOf(now)),
|
||||
);
|
||||
var today = 0;
|
||||
var thisMonth = 0;
|
||||
for (final figures in days) {
|
||||
final used = switch (unit) {
|
||||
AssistantLimitUnit.costMicros => figures.costMicros,
|
||||
AssistantLimitUnit.tokens => figures.inputTokens + figures.outputTokens,
|
||||
};
|
||||
thisMonth += used;
|
||||
if (!figures.day.isBefore(day)) {
|
||||
today += used;
|
||||
}
|
||||
}
|
||||
return (today: today, thisMonth: thisMonth);
|
||||
}
|
||||
|
||||
/// What [person] is offered of assistants (point 6): those granted to them
|
||||
/// that are set, in the operator's order, with what is left of each.
|
||||
Future<AssistantsOffered> assistantsOffered(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
) async {
|
||||
final pod = session.serverpod;
|
||||
final assistants = assistantsOf(pod);
|
||||
if (isMember(pod)) {
|
||||
return AssistantsOffered(
|
||||
assistants: const [],
|
||||
ownKeys: false,
|
||||
currency: assistants.currency,
|
||||
);
|
||||
}
|
||||
// SELECT … FROM "assistant_grant" WHERE "authUserId" = $1
|
||||
final grants = {
|
||||
for (final grant in await AssistantGrant.db.find(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(person),
|
||||
))
|
||||
grant.assistant: grant,
|
||||
};
|
||||
final now = clock.now();
|
||||
return AssistantsOffered(
|
||||
assistants: [
|
||||
for (final assistant in assistants.all)
|
||||
if (grants[assistant.id] case final grant?)
|
||||
await _offerOf(session, assistant, grant, now),
|
||||
],
|
||||
ownKeys: assistants.ownKeys,
|
||||
currency: assistants.currency,
|
||||
);
|
||||
}
|
||||
|
||||
Future<AssistantOffer> _offerOf(
|
||||
Session session,
|
||||
AssistantSetting assistant,
|
||||
AssistantGrant grant,
|
||||
DateTime now,
|
||||
) async {
|
||||
final used = await _usedOf(
|
||||
session,
|
||||
grant.authUserId,
|
||||
assistant.id,
|
||||
grant.limitUnit,
|
||||
now,
|
||||
);
|
||||
return AssistantOffer(
|
||||
id: assistant.id,
|
||||
name: assistant.name,
|
||||
protocol: assistant.endpoint.protocol.key,
|
||||
host: assistant.endpoint.host,
|
||||
model: assistant.endpoint.model,
|
||||
images: assistant.images,
|
||||
agreedAt: grant.agreedTo == providerOf(assistant) ? grant.agreedAt : null,
|
||||
limitUnit: grant.limitUnit,
|
||||
dailyLimit: grant.dailyLimit,
|
||||
monthlyLimit: grant.monthlyLimit,
|
||||
usedToday: used.today,
|
||||
usedThisMonth: used.thisMonth,
|
||||
outdated: grant.limitUnit != limitUnitOf(assistant),
|
||||
);
|
||||
}
|
||||
|
||||
/// [person] agrees to what goes where with [assistant] (D40), its provider
|
||||
/// as it is now. Refused ([AssistantRefusal.notGranted]) without a grant of
|
||||
/// it, or where it isn't set.
|
||||
Future<void> agreeToAssistant(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
String assistant,
|
||||
) async {
|
||||
final setting =
|
||||
_offered(session.serverpod, assistant) ??
|
||||
_refused(AssistantRefusal.notGranted);
|
||||
final grant =
|
||||
await _grantOf(session, person, assistant) ??
|
||||
_refused(AssistantRefusal.notGranted);
|
||||
// UPDATE "assistant_grant" SET "agreedAt" = $1, "agreedTo" = $2
|
||||
// WHERE "id" = $3
|
||||
await AssistantGrant.db.updateRow(
|
||||
session,
|
||||
grant.copyWith(agreedAt: clock.now(), agreedTo: providerOf(setting)),
|
||||
);
|
||||
}
|
||||
|
||||
/// The assistant [assistant] if [person] may ask it now: granted and set
|
||||
/// (else [AssistantRefusal.notGranted]), agreed to as its provider is now
|
||||
/// ([AssistantRefusal.notAgreed]), its limits counting what it counts
|
||||
/// ([AssistantRefusal.limitOutdated]), and neither this UTC month's limit
|
||||
/// nor this day's reached ([AssistantRefusal.limitReached], with when the
|
||||
/// first to lift resets). Checked against what was counted before (D38).
|
||||
Future<AssistantSetting> admittedAssistant(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
String assistant,
|
||||
) async {
|
||||
final setting =
|
||||
_offered(session.serverpod, assistant) ??
|
||||
_refused(AssistantRefusal.notGranted);
|
||||
final grant =
|
||||
await _grantOf(session, person, assistant) ??
|
||||
_refused(AssistantRefusal.notGranted);
|
||||
if (grant.agreedAt == null || grant.agreedTo != providerOf(setting)) {
|
||||
_refused(AssistantRefusal.notAgreed);
|
||||
}
|
||||
if (grant.limitUnit != limitUnitOf(setting)) {
|
||||
_refused(AssistantRefusal.limitOutdated);
|
||||
}
|
||||
if (grant.dailyLimit == null && grant.monthlyLimit == null) {
|
||||
return setting;
|
||||
}
|
||||
final now = clock.now();
|
||||
final used = await _usedOf(session, person, assistant, grant.limitUnit, now);
|
||||
// The month's first: a day's reset lifts nothing while the month's limit
|
||||
// holds.
|
||||
if (grant.monthlyLimit case final limit? when used.thisMonth >= limit) {
|
||||
final month = utcMonthOf(now);
|
||||
_refused(
|
||||
AssistantRefusal.limitReached,
|
||||
resetsAt: DateTime.utc(month.year, month.month + 1),
|
||||
);
|
||||
}
|
||||
if (grant.dailyLimit case final limit? when used.today >= limit) {
|
||||
_refused(
|
||||
AssistantRefusal.limitReached,
|
||||
resetsAt: utcDayOf(now).add(const Duration(days: 1)),
|
||||
);
|
||||
}
|
||||
return setting;
|
||||
}
|
||||
|
||||
/// Counts one request of [person] to [assistant] into its UTC day at [now]:
|
||||
/// the tokens [inputTokens] and [outputTokens], and their cost at its prices
|
||||
/// now (D38), kept as counted.
|
||||
Future<void> countAssistantRequest(
|
||||
Session session, {
|
||||
required UuidValue person,
|
||||
required AssistantSetting assistant,
|
||||
required int inputTokens,
|
||||
required int outputTokens,
|
||||
required DateTime now,
|
||||
}) => session.db.unsafeExecute(
|
||||
// One row per person, assistant and day, which each request adds to, in
|
||||
// one statement, so requests counted at once never lose one another.
|
||||
'INSERT INTO "assistant_figures" ("authUserId", "assistant", "day", '
|
||||
'"requests", "inputTokens", "outputTokens", "costMicros") '
|
||||
'VALUES (@person, @assistant, @day, 1, @input, @output, @cost) '
|
||||
'ON CONFLICT ("authUserId", "assistant", "day") DO UPDATE SET '
|
||||
'"requests" = "assistant_figures"."requests" + 1, '
|
||||
'"inputTokens" = "assistant_figures"."inputTokens" + EXCLUDED."inputTokens", '
|
||||
'"outputTokens" = "assistant_figures"."outputTokens" '
|
||||
'+ EXCLUDED."outputTokens", '
|
||||
'"costMicros" = "assistant_figures"."costMicros" + EXCLUDED."costMicros"',
|
||||
parameters: QueryParameters.named({
|
||||
'person': person.uuid,
|
||||
'assistant': assistant.id,
|
||||
'day': utcDayOf(now),
|
||||
'input': inputTokens,
|
||||
'output': outputTokens,
|
||||
'cost':
|
||||
assistant.prices?.costMicros(
|
||||
inputTokens: inputTokens,
|
||||
outputTokens: outputTokens,
|
||||
) ??
|
||||
0,
|
||||
}),
|
||||
);
|
||||
|
||||
/// The assistants the operator set, without their keys, to grant from.
|
||||
AssistantsSetUp assistantsSetUp(Serverpod pod) {
|
||||
final assistants = assistantsOf(pod);
|
||||
return AssistantsSetUp(
|
||||
assistants: [
|
||||
for (final assistant in assistants.all)
|
||||
AssistantInfo(
|
||||
id: assistant.id,
|
||||
name: assistant.name,
|
||||
protocol: assistant.endpoint.protocol.key,
|
||||
host: assistant.endpoint.host,
|
||||
model: assistant.endpoint.model,
|
||||
images: assistant.images,
|
||||
priceInput: assistant.prices?.input,
|
||||
priceOutput: assistant.prices?.output,
|
||||
),
|
||||
],
|
||||
currency: assistants.currency,
|
||||
ownKeys: assistants.ownKeys,
|
||||
);
|
||||
}
|
||||
|
||||
/// Grants [account] [assistant] with [dailyLimit] and [monthlyLimit] (null:
|
||||
/// none), counted in the unit of its pricing now; granted already, the
|
||||
/// limits change and its agreement stays. Refused for an assistant not set
|
||||
/// ([OperatorRefusal.assistantNotSet], D44), a limit below 0
|
||||
/// ([OperatorRefusal.invalidLimit]) and an account unknown
|
||||
/// ([OperatorRefusal.notFound]).
|
||||
Future<void> grantAssistant(
|
||||
Session session, {
|
||||
required UuidValue account,
|
||||
required String assistant,
|
||||
required int? dailyLimit,
|
||||
required int? monthlyLimit,
|
||||
}) async {
|
||||
final setting =
|
||||
_offered(session.serverpod, assistant) ??
|
||||
_refusedForOperator(OperatorRefusal.assistantNotSet);
|
||||
if ((dailyLimit ?? 0) < 0 || (monthlyLimit ?? 0) < 0) {
|
||||
_refusedForOperator(OperatorRefusal.invalidLimit);
|
||||
}
|
||||
await session.db.transaction((transaction) async {
|
||||
// SELECT … FROM "serverpod_auth_core_user" WHERE "id" = $1
|
||||
if (await AuthUser.db.findById(
|
||||
session,
|
||||
account,
|
||||
transaction: transaction,
|
||||
) ==
|
||||
null) {
|
||||
_refusedForOperator(OperatorRefusal.notFound);
|
||||
}
|
||||
final grant = AssistantGrant(
|
||||
authUserId: account,
|
||||
assistant: assistant,
|
||||
grantedAt: clock.now(),
|
||||
limitUnit: limitUnitOf(setting),
|
||||
dailyLimit: dailyLimit,
|
||||
monthlyLimit: monthlyLimit,
|
||||
);
|
||||
switch (await _grantOf(
|
||||
session,
|
||||
account,
|
||||
assistant,
|
||||
transaction: transaction,
|
||||
)) {
|
||||
case final granted?:
|
||||
// UPDATE "assistant_grant" SET … WHERE "id" = $1
|
||||
await AssistantGrant.db.updateRow(
|
||||
session,
|
||||
grant.copyWith(
|
||||
id: granted.id,
|
||||
agreedAt: granted.agreedAt,
|
||||
agreedTo: granted.agreedTo,
|
||||
),
|
||||
transaction: transaction,
|
||||
);
|
||||
case null:
|
||||
// INSERT INTO "assistant_grant" (…) VALUES (…)
|
||||
await AssistantGrant.db.insertRow(
|
||||
session,
|
||||
grant,
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/// Takes [assistant] back from [account]; their figures stay until their
|
||||
/// time. Refused ([OperatorRefusal.notFound]) where it isn't granted.
|
||||
Future<void> revokeAssistant(
|
||||
Session session, {
|
||||
required UuidValue account,
|
||||
required String assistant,
|
||||
}) async {
|
||||
// DELETE FROM "assistant_grant" WHERE "authUserId" = $1 AND "assistant" = $2
|
||||
final revoked = await AssistantGrant.db.deleteWhere(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(account) & t.assistant.equals(assistant),
|
||||
);
|
||||
if (revoked.isEmpty) {
|
||||
_refusedForOperator(OperatorRefusal.notFound);
|
||||
}
|
||||
}
|
||||
|
||||
/// Every grant, by handle, then in the operator's order of assistants
|
||||
/// (those set no more last).
|
||||
Future<List<AssistantGrantOverview>> assistantGrants(Session session) async {
|
||||
final assistants = assistantsOf(session.serverpod);
|
||||
// SELECT … FROM "assistant_grant"
|
||||
final grants = await AssistantGrant.db.find(session);
|
||||
final handles = await _handlesOf(session, {
|
||||
for (final grant in grants) grant.authUserId,
|
||||
});
|
||||
final overviews = [
|
||||
for (final grant in grants)
|
||||
AssistantGrantOverview(
|
||||
account: grant.authUserId,
|
||||
handle: handles[grant.authUserId],
|
||||
assistant: grant.assistant,
|
||||
assistantSet: assistants.named(grant.assistant) != null,
|
||||
grantedAt: grant.grantedAt,
|
||||
limitUnit: grant.limitUnit,
|
||||
dailyLimit: grant.dailyLimit,
|
||||
monthlyLimit: grant.monthlyLimit,
|
||||
agreedAt: switch (assistants.named(grant.assistant)) {
|
||||
final setting? when grant.agreedTo == providerOf(setting) =>
|
||||
grant.agreedAt,
|
||||
_ => null,
|
||||
},
|
||||
outdated: switch (assistants.named(grant.assistant)) {
|
||||
final setting? => grant.limitUnit != limitUnitOf(setting),
|
||||
null => false,
|
||||
},
|
||||
),
|
||||
];
|
||||
return overviews..sort(
|
||||
(a, b) => _byPersonAndAssistant(
|
||||
assistants,
|
||||
(a.handle, a.account, a.assistant),
|
||||
(b.handle, b.account, b.assistant),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/// Everyone's figures of [month]'s UTC month, per person and assistant, by
|
||||
/// handle, then in the operator's order of assistants.
|
||||
Future<List<AssistantMonthFigures>> everyonesAssistantFigures(
|
||||
Session session,
|
||||
DateTime month,
|
||||
) async {
|
||||
final first = utcMonthOf(month);
|
||||
// SELECT … FROM "assistant_figures" WHERE "day" >= $1 AND "day" < $2
|
||||
final days = await AssistantFigures.db.find(
|
||||
session,
|
||||
where: (t) =>
|
||||
(t.day >= first) & (t.day < DateTime.utc(first.year, first.month + 1)),
|
||||
);
|
||||
final handles = await _handlesOf(session, {
|
||||
for (final day in days) day.authUserId,
|
||||
});
|
||||
final months = _byMonth(days, handles: handles);
|
||||
final assistants = assistantsOf(session.serverpod);
|
||||
return months..sort(
|
||||
(a, b) => _byPersonAndAssistant(
|
||||
assistants,
|
||||
(a.handle, a.account, a.assistant),
|
||||
(b.handle, b.account, b.assistant),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/// [person]'s own figures, month by month, the latest first, without a
|
||||
/// handle.
|
||||
Future<List<AssistantMonthFigures>> ownAssistantFigures(
|
||||
Session session,
|
||||
UuidValue person,
|
||||
) async {
|
||||
// SELECT … FROM "assistant_figures" WHERE "authUserId" = $1
|
||||
final days = await AssistantFigures.db.find(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(person),
|
||||
);
|
||||
final assistants = assistantsOf(session.serverpod);
|
||||
return _byMonth(days, handles: const {})..sort((a, b) {
|
||||
final byMonth = b.month.compareTo(a.month);
|
||||
return byMonth != 0
|
||||
? byMonth
|
||||
: _assistantOrder(assistants, a.assistant, b.assistant);
|
||||
});
|
||||
}
|
||||
|
||||
/// [days] added up per person, assistant and UTC month.
|
||||
List<AssistantMonthFigures> _byMonth(
|
||||
List<AssistantFigures> days, {
|
||||
required Map<UuidValue, String> handles,
|
||||
}) {
|
||||
final months = <(UuidValue, String, DateTime), AssistantMonthFigures>{};
|
||||
for (final day in days) {
|
||||
final key = (day.authUserId, day.assistant, utcMonthOf(day.day));
|
||||
final sum = months[key];
|
||||
months[key] = AssistantMonthFigures(
|
||||
account: day.authUserId,
|
||||
handle: handles[day.authUserId],
|
||||
assistant: day.assistant,
|
||||
month: key.$3,
|
||||
requests: (sum?.requests ?? 0) + day.requests,
|
||||
inputTokens: (sum?.inputTokens ?? 0) + day.inputTokens,
|
||||
outputTokens: (sum?.outputTokens ?? 0) + day.outputTokens,
|
||||
costMicros: (sum?.costMicros ?? 0) + day.costMicros,
|
||||
);
|
||||
}
|
||||
return months.values.toList();
|
||||
}
|
||||
|
||||
/// [accounts]' handles, by account; those without one left out.
|
||||
Future<Map<UuidValue, String>> _handlesOf(
|
||||
Session session,
|
||||
Set<UuidValue> accounts,
|
||||
) async {
|
||||
if (accounts.isEmpty) {
|
||||
return const {};
|
||||
}
|
||||
// SELECT … FROM "public_profile" WHERE "authUserId" = ANY($1)
|
||||
return {
|
||||
for (final profile in await PublicProfile.db.find(
|
||||
session,
|
||||
where: (t) => t.authUserId.inSet(accounts),
|
||||
))
|
||||
profile.authUserId: profile.handle,
|
||||
};
|
||||
}
|
||||
|
||||
/// By handle, whatever its case (those without one last, by account), then
|
||||
/// [_assistantOrder].
|
||||
int _byPersonAndAssistant(
|
||||
Assistants assistants,
|
||||
(String?, UuidValue, String) a,
|
||||
(String?, UuidValue, String) b,
|
||||
) {
|
||||
final byHandle = switch ((a.$1, b.$1)) {
|
||||
(final x?, final y?) => x.toLowerCase().compareTo(y.toLowerCase()),
|
||||
(_?, null) => -1,
|
||||
(null, _?) => 1,
|
||||
(null, null) => 0,
|
||||
};
|
||||
if (byHandle != 0) {
|
||||
return byHandle;
|
||||
}
|
||||
final byAccount = a.$2.uuid.compareTo(b.$2.uuid);
|
||||
return byAccount != 0 ? byAccount : _assistantOrder(assistants, a.$3, b.$3);
|
||||
}
|
||||
|
||||
/// In the operator's order, those set no more after, by ID.
|
||||
int _assistantOrder(Assistants assistants, String a, String b) {
|
||||
int place(String id) {
|
||||
final index = assistants.all.indexWhere((assistant) => assistant.id == id);
|
||||
return index < 0 ? assistants.all.length : index;
|
||||
}
|
||||
|
||||
final byPlace = place(a).compareTo(place(b));
|
||||
return byPlace != 0 ? byPlace : a.compareTo(b);
|
||||
}
|
||||
|
||||
/// Forgets the figures of the days before the [assistantFiguresMonths] UTC
|
||||
/// months up to [now] (D41), and returns how many days went. Daily, with the
|
||||
/// pruning.
|
||||
Future<int> forgetOldAssistantFigures(
|
||||
Session session, {
|
||||
required DateTime now,
|
||||
}) async {
|
||||
final month = utcMonthOf(now);
|
||||
// DELETE FROM "assistant_figures" WHERE "day" < $1
|
||||
final forgotten = await AssistantFigures.db.deleteWhere(
|
||||
session,
|
||||
where: (t) =>
|
||||
t.day <
|
||||
DateTime.utc(month.year, month.month - (assistantFiguresMonths - 1)),
|
||||
);
|
||||
return forgotten.length;
|
||||
}
|
||||
|
||||
/// Deletes [person]'s grants and figures, with the account
|
||||
/// (`deletePerson`).
|
||||
Future<void> deleteAssistantDataOf(
|
||||
Session session,
|
||||
UuidValue person, {
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
// DELETE FROM "assistant_grant" WHERE "authUserId" = $1
|
||||
await AssistantGrant.db.deleteWhere(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(person),
|
||||
transaction: transaction,
|
||||
);
|
||||
// DELETE FROM "assistant_figures" WHERE "authUserId" = $1
|
||||
await AssistantFigures.db.deleteWhere(
|
||||
session,
|
||||
where: (t) => t.authUserId.equals(person),
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
### An assistant as the operator set it (Halkyon Learn's ADR 0021, point
|
||||
### 4), without its key: what their screen of grants shows.
|
||||
class: AssistantInfo
|
||||
fields:
|
||||
### Its ID.
|
||||
id: String
|
||||
### The name people see.
|
||||
name: String
|
||||
### Its protocol.
|
||||
protocol: String
|
||||
### The provider's host.
|
||||
host: String
|
||||
### The model.
|
||||
model: String
|
||||
### Whether it reads images.
|
||||
images: bool
|
||||
### Its price per million input tokens; null where it has none, so its
|
||||
### limits count tokens.
|
||||
priceInput: double?
|
||||
### Its price per million output tokens.
|
||||
priceOutput: double?
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
### What a grant's limits count (Halkyon Learn's ADR 0021, point 8).
|
||||
enum: AssistantLimitUnit
|
||||
serialized: byName
|
||||
values:
|
||||
### The cost, in millionths of the instance's currency, at the prices
|
||||
### the operator set.
|
||||
- costMicros
|
||||
### Input and output tokens together, where the operator set no prices.
|
||||
- tokens
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
### A turn of a conversation with an assistant.
|
||||
class: AssistantMessage
|
||||
fields:
|
||||
### Who said it.
|
||||
role: AssistantRole
|
||||
### What it holds, in order.
|
||||
parts: List<AssistantPart>
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
### What one person used of one assistant in one UTC month (Halkyon
|
||||
### Learn's ADR 0021, D3 and point 8): the operator sees everyone's, a
|
||||
### person their own; never what was asked.
|
||||
class: AssistantMonthFigures
|
||||
fields:
|
||||
### The person's account.
|
||||
account: UuidValue
|
||||
### Their handle; null for an account without one, or in a person's own.
|
||||
handle: String?
|
||||
### The assistant's ID.
|
||||
assistant: String
|
||||
### The month, at midnight UTC of its first day.
|
||||
month: DateTime
|
||||
### The requests forwarded.
|
||||
requests: int
|
||||
### The input tokens counted.
|
||||
inputTokens: int
|
||||
### The output tokens counted.
|
||||
outputTokens: int
|
||||
### What they cost, in millionths of the currency; 0 for an assistant
|
||||
### without prices.
|
||||
costMicros: int
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
### An assistant granted to the person who asks (Halkyon Learn's ADR 0021,
|
||||
### point 6 and D27): what it is, what goes where, and how much is left.
|
||||
class: AssistantOffer
|
||||
fields:
|
||||
### Its ID, which a question names.
|
||||
id: String
|
||||
### The name people see.
|
||||
name: String
|
||||
### Its protocol (`anthropic`, `openai`, `openai-responses`).
|
||||
protocol: String
|
||||
### The provider's host, where what is asked goes (`api.anthropic.com`).
|
||||
host: String
|
||||
### The model.
|
||||
model: String
|
||||
### Whether it reads images (D12).
|
||||
images: bool
|
||||
### When the person agreed; null until they have, or since it moved to
|
||||
### another provider: then they are asked first (D40).
|
||||
agreedAt: DateTime?
|
||||
### What its limits count.
|
||||
limitUnit: AssistantLimitUnit
|
||||
### The most a UTC day may use; null for none.
|
||||
dailyLimit: int?
|
||||
### The most a UTC month may use; null for none.
|
||||
monthlyLimit: int?
|
||||
### What this UTC day used so far, in [limitUnit].
|
||||
usedToday: int
|
||||
### What this UTC month used so far, in [limitUnit].
|
||||
usedThisMonth: int
|
||||
### Whether the grant holds no more until the operator sets its limits
|
||||
### again, as the assistant's prices came or went.
|
||||
outdated: bool
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
import 'package:halkyon_base_server/src/assistant/assistant_grants.dart';
|
||||
import 'package:halkyon_base_server/src/auth/scopes.dart';
|
||||
import 'package:halkyon_base_server/src/generated/protocol.dart';
|
||||
import 'package:serverpod/serverpod.dart';
|
||||
|
||||
/// The operator's assistants as the operator grants them (Halkyon Learn's
|
||||
/// ADR 0021, points 6 and 8): the assistants set, the grants with their
|
||||
/// limits, and everyone's figures, never what was asked. The operator's
|
||||
/// scope only the register's sessions carry. Refusals are
|
||||
/// `OperatorRefusedException`s.
|
||||
class AssistantOperatorEndpoint extends Endpoint {
|
||||
@override
|
||||
Set<Scope> get requiredScopes => {identified, instanceOperator};
|
||||
|
||||
/// The assistants set in `deploy/assistants.env`, without their keys.
|
||||
Future<AssistantsSetUp> setUp(Session session) =>
|
||||
Future.value(assistantsSetUp(session.serverpod));
|
||||
|
||||
/// Every grant, by handle.
|
||||
Future<List<AssistantGrantOverview>> grants(Session session) =>
|
||||
assistantGrants(session);
|
||||
|
||||
/// Grants [account] [assistant] with [dailyLimit] and [monthlyLimit] (null:
|
||||
/// none), in the unit of its pricing; granted already, the limits change
|
||||
/// and the agreement stays. Refused for an assistant not set
|
||||
/// (`OperatorRefusal.assistantNotSet`), a limit below 0
|
||||
/// (`OperatorRefusal.invalidLimit`) and an account unknown
|
||||
/// (`OperatorRefusal.notFound`).
|
||||
Future<void> grant(
|
||||
Session session, {
|
||||
required UuidValue account,
|
||||
required String assistant,
|
||||
required int? dailyLimit,
|
||||
required int? monthlyLimit,
|
||||
}) => grantAssistant(
|
||||
session,
|
||||
account: account,
|
||||
assistant: assistant,
|
||||
dailyLimit: dailyLimit,
|
||||
monthlyLimit: monthlyLimit,
|
||||
);
|
||||
|
||||
/// Takes [assistant] back from [account]; refused where it isn't granted
|
||||
/// (`OperatorRefusal.notFound`).
|
||||
Future<void> revoke(
|
||||
Session session, {
|
||||
required UuidValue account,
|
||||
required String assistant,
|
||||
}) => revokeAssistant(session, account: account, assistant: assistant);
|
||||
|
||||
/// Everyone's figures of the UTC month [month] is in, per person and
|
||||
/// assistant.
|
||||
Future<List<AssistantMonthFigures>> figures(
|
||||
Session session, {
|
||||
required DateTime month,
|
||||
}) => everyonesAssistantFigures(session, month);
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
### A part of a turn: one class for every kind, so a turn is a plain list.
|
||||
class: AssistantPart
|
||||
fields:
|
||||
### What it is, and so which fields it has.
|
||||
kind: AssistantPartKind
|
||||
### A text's text, or a tool's result.
|
||||
text: String?
|
||||
### An image's type (`image/png`).
|
||||
mediaType: String?
|
||||
### An image's bytes.
|
||||
bytes: ByteData?
|
||||
### A tool call's ID, or the call a result answers.
|
||||
callId: String?
|
||||
### A tool call's tool.
|
||||
name: String?
|
||||
### A tool call's input, or a provider's block, as JSON.
|
||||
json: String?
|
||||
### What the provider wants back with a text of the model's or a tool
|
||||
### call, as JSON; null for nothing.
|
||||
extra: String?
|
||||
### Whether a tool's result says it failed.
|
||||
isError: bool?
|
||||
### A provider's block's protocol.
|
||||
protocol: String?
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
### What a part of a turn is.
|
||||
enum: AssistantPartKind
|
||||
serialized: byName
|
||||
values:
|
||||
### Text ([AssistantPart.text]; in the model's turn, what the provider
|
||||
### wants back with it in [AssistantPart.extra]).
|
||||
- text
|
||||
### An image ([AssistantPart.mediaType], [AssistantPart.bytes]).
|
||||
- image
|
||||
### A call of a tool ([AssistantPart.callId], [AssistantPart.name], its
|
||||
### input as JSON in [AssistantPart.json], what the provider wants back
|
||||
### with it in [AssistantPart.extra]).
|
||||
- toolCall
|
||||
### A tool's result ([AssistantPart.callId], [AssistantPart.text],
|
||||
### [AssistantPart.isError]).
|
||||
- toolResult
|
||||
### A block only the provider reads ([AssistantPart.protocol], the block
|
||||
### as JSON in [AssistantPart.json]), sent back to it unchanged.
|
||||
- providerBlock
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
### What the app asks an operator's assistant (Halkyon Learn's ADR 0021,
|
||||
### point 7): the whole conversation each time, as the instance keeps none.
|
||||
class: AssistantQuestion
|
||||
fields:
|
||||
### The assistant's ID.
|
||||
assistant: String
|
||||
### What the model is told first; empty for nothing.
|
||||
system: String
|
||||
### The conversation, ending with the person's turn.
|
||||
messages: List<AssistantMessage>
|
||||
### The tools it may call.
|
||||
tools: List<AssistantTool>
|
||||
### The ceiling on the answer's tokens, the instance's own ceiling at
|
||||
### most.
|
||||
maxOutputTokens: int
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
### Why an assistant refused, or failed (Halkyon Learn's ADR 0021).
|
||||
enum: AssistantRefusal
|
||||
serialized: byName
|
||||
values:
|
||||
### The person has no grant of it, or the instance doesn't set it (any
|
||||
### more), or this service offers no assistant (D43).
|
||||
- notGranted
|
||||
### The person hasn't agreed to what goes where, or it moved to another
|
||||
### provider since (D40).
|
||||
- notAgreed
|
||||
### A day's or a month's limit is reached;
|
||||
### [AssistantRefusedException.resetsAt] says when it resets.
|
||||
- limitReached
|
||||
### The grant's limits count what the assistant counts no more, as its
|
||||
### prices came or went: the operator sets them again.
|
||||
- limitOutdated
|
||||
### The question holds an image, and the assistant reads none.
|
||||
- imagesNotRead
|
||||
### The question can't be sent: a turn out of place, or a tool call's
|
||||
### JSON unreadable.
|
||||
- malformed
|
||||
### The question is larger than the instance forwards: its text, or an
|
||||
### image.
|
||||
- tooLarge
|
||||
### The person has as many questions on their way as they may at once.
|
||||
- tooManyAtOnce
|
||||
### The provider refuses the instance's key or its account.
|
||||
- providerKey
|
||||
### The provider is busy or overloaded for now;
|
||||
### [AssistantRefusedException.retryAfterSeconds] may say for how long.
|
||||
- providerBusy
|
||||
### The provider refused the question: too long for the model, say.
|
||||
- providerRefused
|
||||
### The provider failed, couldn't be reached, or answered unreadably.
|
||||
- providerFailed
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
### An assistant refused a question, or failed on it.
|
||||
exception: AssistantRefusedException
|
||||
fields:
|
||||
### Why.
|
||||
refusal: AssistantRefusal
|
||||
### When the limit reached resets (UTC midnight, or the first of next
|
||||
### month); only for [AssistantRefusal.limitReached].
|
||||
resetsAt: DateTime?
|
||||
### How long the provider asks to wait, in seconds, where it says.
|
||||
retryAfterSeconds: int?
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue