llm-client

The LLM client of Lumingo. Two wire protocols (openai_chat, anthropic_messages) behind the LlmClient trait, a capability probe, the structured-output ladder with local validation against contracts/, retries and limits, and keys held in memory only.

Use

let profile = EnvProfileLoader::with_default_paths().load()?.ok_or(NoProfile)?; // read-only `env`
let client = ProviderClient::connect(&profile, ClientOptions::default(), stored_capabilities)?;
let capabilities = client.probe(&cancel).await?;                          // store it as JSON
let stream = client.stream_text(request, cancel.clone()).await?;          // tutor text
let analysis = client.structured(structured_request, cancel).await?;      // validated JSON

Rules this crate keeps

• Every request goes through HttpClientFactory: only the active provider's host (plus setup hosts while a guard is held), HTTPS except for loopback, redirects only to allowed hosts, no system proxy.
• Timeouts: connect 5 s, first token 8 s, total 30 s for a tutor turn. Structured and probe calls have no first-token limit and 60 s in total.
• Retries: one for transport errors and 5xx with jitter; none for other 4xx; 429 follows Retry-After, up to two retries, and is not waited for when it would outlast the call.
• A key is an ApiKey: no Display, no Serialize, a redacting Debug. Errors carry fixed categories or sanitised provider text. The UI type is ProfileInfo (has_key, key_last4).
• providers.toml holds keys in plain text. It is written with mode 0600 on Unix. On Windows it inherits the permissions of the per-user data directory it is written into; no ACL call is made.

Payload log (the inspector)

PayloadLog (inspector.rs) is a bounded ring of the last provider requests and responses, for GET /api/inspector. Give one to ProviderClient::connect_logged or to an adapter with with_payload_log; a client without one records nothing.

• Capacity: 50 entries by default (PAYLOAD_LOG_CAPACITY). When full, the oldest entry is dropped and counted in PayloadSnapshot::dropped; nothing waits. Entry numbers are not reused, so a gap shows what was dropped.
• Bodies: at most 16 KiB each (PAYLOAD_MAX_BODY_BYTES), cut on a character boundary, with a *_truncated flag. A streamed reply is kept as its raw event-stream text.
• One entry per HTTP request, so a retry shows as its own entry.
• Never recorded: headers, the query string, the key. Bodies are scrubbed (redact::scrub) when captured, before they reach the ring: the exact key and key-shaped tokens become [redacted].
• Memory only; it is gone when the program stops. It holds learner text, so it is not part of any export.

Tests

cargo test -p llm-client replays hand-written fixtures (tests/fixtures/) through a scripted server on 127.0.0.1. No test contacts a provider.

Not verified

• The wire formats follow the providers' documentation as read on 2026-10-05. The owner's live smoke tests of 2026-10-07 and 2026-10-08 ran on the pre-merge line and found two gateway quirks, both handled here: a gateway that answers anthropic_messages with an OpenAI-shaped body (read as a fallback), and one that silently ignores the native JSON schema (the probe's check canary fails level 1 and the walk continues). Live check 2026-10-10: the owner ran tutor-cli probe from the merged tree and it passed — anthropic_messages, structured level Some(2), auth and stream true, exit 0 — so both fixes are validated live against the real gateway (the OpenAI-shaped body is read at step 1; the canary walks the ladder at step 3).
• The HTTPS path (native-tls) is compiled for Linux and checked for x86_64-pc-windows-msvc, but no test performs a TLS handshake, because the tests use loopback HTTP. Schannel on Windows is untested.
• The connect timeout is set on the HTTP client but no test reaches it.
• Rate-limit header names are the ones OpenAI and Anthropic document; other servers leave rate_limit empty.