feat(llm): a request carries how much the model thinks, in each provider's field
LlmRequest.effort (LlmEffort: low, medium, high, xhigh, max, Anthropic's names) goes as Anthropic's output_config.effort, as reasoning.effort to the Responses API and OpenRouter, and as reasoning_effort to OpenAI's chat completions, Gemini and any other host. endpoint.efforts says which levels a host takes (Gemini low to high, Mistral none); send refuses another before anything goes, the fake client too. LlmEffort.outputCeiling gives the room for output each level wants, as thinking counts towards the ceiling (Halkyon Learn's ADR 0021, D63). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
33c9a7df1c
commit
0590a4b43d
1 changed files with 291 additions and 10 deletions
|
|
@ -73,6 +73,15 @@ the local network, which may be shared.
|
|||
phase in `LlmText.extra`. Send the `LlmFinished.message` back unchanged.
|
||||
- **The person's opaque ID** (`LlmRequest.personId`) goes where the provider takes one:
|
||||
Anthropic's `metadata.user_id`, OpenAI's `safety_identifier`, OpenRouter's `user`.
|
||||
- **The effort** (`LlmRequest.effort`, `LlmEffort`: `low`, `medium`, `high`, `xhigh`,
|
||||
`max`, Anthropic's names; null leaves it to the model) goes as Anthropic's
|
||||
`output_config.effort`, the Responses API's and OpenRouter's `reasoning.effort`, and
|
||||
`reasoning_effort` to OpenAI's chat completions, Gemini and any other host.
|
||||
`endpoint.efforts` says which levels a host takes: Gemini's low to high, Mistral's
|
||||
none, every one elsewhere; `send` throws an `ArgumentError` for another. Which levels a
|
||||
model has is the model's, and its provider refuses one it lacks. The model's thinking
|
||||
counts towards `maxOutputTokens`, so a higher effort wants a higher ceiling:
|
||||
`LlmEffort.outputCeiling` (8,192 at low to 65,536 at max; 16,384 without an effort).
|
||||
|
||||
## Testing with it
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue