LLM Profiles:
a name, a record,
a running model.

01One path, four owners

1. 1 · PRODUCTChoose a nameRole defaults and trusted scope overrides.models.json
   productProfileSelection
2. 2 · SERVERResolve the recordLook up and validate the saved LLM profile.ServerStateService
   profileAgentFactory
3. 3 · CONVERSATIONSave the selectionKeep an effective snapshot and any pending switch.llm_profile_ref
   llm_profile_snapshot
4. 4 · SDKSpeak the APIResolve private authentication and serialize native requests.createClientFromProfile
   provider clients

Bridges submit work for registered lanes. They do not each resolve models, handle provider quirks or implement a separate switching mechanism. A scope is supplied by trusted lane registration; incoming message tags cannot claim another channel’s configuration.

Source: product composition · selection resolver · profile schema · client factory.

02Configure roles with saved profile names

The product reads ~/.smolpaws/models.json. SMOLPAWS_HOME_DIR changes the default directory; SMOLPAWS_MODELS_CONFIG selects a different file. The names below are illustrative: each must already exist in the catalog of the server that uses it.

{
  "version": 1,
  "roles": {
    "agent": "general-agent",
    "condenser": "small-summarizer",
    "oracle": "consultant"
  },
  "scopes": {
    "whatsapp:main": { "agent": "main-agent" },
    "whatsapp:openhands": { "agent": "general-agent" },
    "whatsapp:hunting": { "agent": "general-agent" }
  }
}

Selection order: exact platform:scopeId override → shared roles entry. With no applicable mapping, creation uses its ordinary explicit/server-default settings and an existing conversation keeps its choice. A specialist role never implicitly borrows agent. Scheduled work retains its originating scope, including isolated runs; an explicitly configured isolated helper's profile takes precedence over these role selections.

Save the file atomically: write complete JSON to a temporary file, then rename it over the destination. The server rereads the applicable selection when conversation work is requested. It does not watch the file or interrupt an ongoing call merely because the file changed. An absent default file preserves ordinary behavior; an explicitly configured missing or malformed file fails instead of silently choosing a different model.

Configuration contract and examples · parser and selection tests.

03The catalog, snapshots and credentials live separately

The SDK defines an LLM profile; the host persists it. In the agent-server, ServerStateService owns state.json, containing llmProfiles, agentProfiles, settings and secret metadata. It loads once per service instance, serializes its mutations and replaces JSON atomically. This is not a multi-process profile database or a file-watching configuration service.

The generic conversation-root precedence is OPENHANDS_CONVERSATIONS_PATH → PERSISTENCE_DIR → relative workspace/conversations. The product bootstrap normally supplies PERSISTENCE_DIR=$SMOLPAWS_HOME_DIR/conversations. Canary overrides are deployment choices, not SDK defaults. A models.json shared by two hosts still requires the selected names to exist in each applicable host’s own catalog.

Only a missing state file seeds the initial default profile: OpenAI gpt-5-nano, Responses mode, and server conversation limit 500. Corrupt state fails rather than silently resetting. A profile record alone does not supply a credential.

What goes in an LLM profile

{
  "profileId": "main-agent",
  "providerId": "openai",
  "model": "gpt-5-mini",
  "openAiApiMode": "responses",
  "reasoningEffort": "low",
  "cachingPrompt": true,
  "useProfileKeyOverride": false
}

This is a schema-valid illustrative record. The strict SDK schema supplies defaults where declared and rejects unknown properties. Optional provider settings such as anthropicCacheTtl stay absent when omitted. The schema has no raw apiKey field and no fallback-model chain.

Key lookup is explicit

A SecretRef is a service/account pair; the default service is openhands. When useProfileKeyOverride is true and llm-profile:<profileId>:api-key exists, use it. Otherwise look up llm-provider:<providerId>. Missing both fails client construction. Transport classification does not rename the provider ID for credential lookup.

PATCH /api/settings can store llm_api_key through the active profile’s secret reference. General settings secrets and conversation secrets have separate account namespaces and redacted API representations. The session API key (OPENHANDS_SESSION_API_KEY, then SESSION_API_KEY) authenticates server API requests; it is not an LLM credential.

Subscription authentication is an explicit different path. It uses SDK-owned device/browser login, private OAuth persistence and refresh; it neither reads ~/.codex/auth.json nor silently falls back to an API key. The server exposes status, supported models, device start/poll and logout under /api/llm/subscription/openai/. Even a status request may refresh an expired account; it is not guaranteed to be an offline read.

Server state and mutations · server path defaults · conversation metadata · LLM profile schema · secret-store contract · profile routes · generated OpenAPI · OAuth persistence.

04Create, switch and restore the same conversation

1. Create from a saved reference.The request’s agent settings win over server defaults. The selected profile record is validated and snapshotted. REST creation still requires its initial requested/default reference to exist, even when product configuration will select another profile before execution.
2. Observe a requested configuration change.A run request causes the product selector to reread its applicable name. A busy conversation applies that request after the current complete model/tool step settles, before another model call.
3. Let the agent choose through switch_llm.The tool accepts profile_name and a reason. The server validates the target, prepares its client and durably queues it. The tool returns acceptance; it does not wait for the run that contains itself.
4. Activate at the safe boundary.After all results from the tool batch are recorded, the SDK can replace the agent’s LLM binding. Conversation identity, messages, tools, context, step budget and accumulated usage remain. A switch alongside finish still persists for the next turn.
5. Restore from durable state.The effective reference/snapshot and pending choice survive restart. The server does not reconstruct the choice by blindly applying today’s default to old history.

Changing the applicable configured name again takes precedence over an earlier tool choice. A missing target, malformed selection or authentication failure is surfaced rather than silently falling back. An unknown profile or failed client construction returns a tool error without replacing the active profile. After a persistence or cleanup error, saved metadata is authoritative because a write may already have committed. An activation-boundary failure stops that run; the next turn rebuilds the persisted binding.

Durable profile runtime · requested work and boundary wiring · upstream switch evidence and deliberate differences.

05One thin interface; native provider implementations

The shared interface is LLMClient.complete(messages, tools?). Dispatch uses the profile’s provider and endpoint, then its API mode. It does not choose credentials by guessing from the model name. Known Anthropic and Gemini providers use their native clients; other compatible providers select Chat Completions or Responses. An unknown provider ID may be classified by its base URL, but credential lookup still uses the original provider ID.

A gateway serving Claude still uses the gateway’s wire protocol. Model capabilities can affect caching or reasoning preparation; they do not silently change the chosen transport or credential owner. The TypeScript runtime does not depend on Python LiteLLM, even when its configured endpoint is a LiteLLM proxy.

Dispatch factory · capability helpers · Anthropic cache preparation · tool ordering · provider implementation guide.

06Record the provider’s usage; preserve uncertainty

Each model response returned to an Agent.step gets one durable llm_usage record, including available usage on invalid or incomplete responses. A response containing several tool calls is still one model call. A user message requiring many model steps produces many records. Statistics are derived from those records, deduplicated by record ID and restored without charging the same call twice.

• Keep prompt, output, total, cache-read, cache-write, cache-miss and reasoning counters distinct. Cache reads are part of input; reasoning is generally part of output. Do not add either again to a provider’s total.
• DeepSeek cache misses are ordinary uncached input, not evidence of a separately charged cache write. Missing fields stay unknown.
• Keep the profile, requested model and actually served model per call. A switch does not relabel earlier calls or reset accumulation.
• Prefer a provider-reported cost with its currency. A calculated estimate must retain its source, dated rates and assumptions. The current built-in pricing calculator is deliberately limited to recognized direct DeepSeek Flash requests.
• Expose measured subtotals alongside missing-usage, missing-cost and unmeasured-history coverage. Never turn absent billing information into zero dollars.

Proxy billing and conversation accounting are different views. A proxy key’s weekly spend includes every consumer of that key. To attribute it to one conversation, match its response IDs against proxy receipts and compare token counts. Missing local Fable cost was resolved this way for an operator report; it was not fabricated from a model alias or backfilled into historical events.

Opaque continuation also has provenance. Usage records can carry history_origin; the initial binding is anchored before a switch. Signed thinking or encrypted reasoning is projected out of incompatible outgoing requests while the saved events remain intact. Visible text, images and valid tool exchanges remain available across profiles.

Metrics contract · records and projections · bounded cost estimates · provider-history compatibility.

07Tests: contract, integration and real providers

Three claims require different evidence: deterministic behavior is correct; it matches the pinned Python contract; a real provider accepted and completed the request. Keep those claims separate.

# In smolpaws/openhands-agent: deterministic checks, no live provider request
npm test
npm run test:drift
npm run typecheck
npm run typecheck:drift
npm run lint
npm run build
npm run typecheck:live

# In smolpaws/smolpaws
npm run ci --prefix packages/openhands-agent-server
npm run relay-server:test
npm run relay-server:typecheck
npm run typecheck

The server CI also checks provenance/review evidence, generated OpenAPI parity, a local endpoint smoke, coverage, lint, build and packed-consumer use. The SDK projection gate compares against the pinned Python oracle. Ordinary CI typechecks live scripts; that does not execute them against a provider.

The GitHub environments are repository-scoped

Names and protection settings verified September 16, 2026. The table lists secret names, never values. An environment called LLM in one repository is independent of the same name in another.

The bounded SDK live checks

These are request/token/time bounds, not a fixed-dollar guarantee. The hardened checks use synthetic prompts, in-memory secrets and sanitized accounting summaries. They do not operate on production conversations. Each GitHub job has a six-minute timeout.

# Intentionally invokes real providers using the SDK repository's LLM environment.
# Choose this separately from ordinary PR/CI checks.
gh workflow run llm.yml --repo smolpaws/openhands-agent --ref main

Dated evidence: Run 35102961257 passed both DeepSeek and Haiku live steps on September 16 at SDK cf17c3e. It predates the SDK revision documented here; it is not a claim that today’s entire head has run through that workflow. The subsequent profile-switch work has separate local live evidence: DeepSeek → Haiku → actual switch_llm → DeepSeek, then restart with the tool-selected profile preserved.

SDK CI · SDK Live LLM · live-check runbook · SDK Examples · server Examples workflow · server profile scenario · subscription smoke.

08Rules for changing this part of the system

1. Give each decision one owner. Product policy chooses role/scope names. The server owns its catalog and durable conversation selection. The SDK owns schema, credential abstractions, native APIs and safe execution boundaries. Bridges should not acquire provider-specific branches.
2. Select profiles at the product boundary. REST/product callers pass saved references. Low-level provider clients remain advanced/testing surfaces. Do not copy provider settings into every channel configuration.
3. Keep authentication private. Persist credential references, never raw keys or OAuth material, in profile records, settings, snapshots, events or notebooks. Do not use provider failures as a reason to spill raw request/response objects into logs.
4. Honor provider behavior directly. TypeScript owns compatibility that Python may obtain through LiteLLM. Put a quirk in its provider serializer/parser or a small shared capability helper; add a fixture at that boundary. A provider fix can be required even when Python has no corresponding change.
5. Switch only at complete step boundaries. Finish tool observations before replacing the model. Persist intent and activation; preserve history and metrics. Test both a continuing run and a switch in a final tool batch, including restart and failure paths.
6. Preserve Python’s observable contract. Read the pinned source and port its relevant tests first. Classify a bounded OLD_PIN..NEW_PIN interval; do not treat “latest upstream” as an unbounded implementation target.
7. Document deliberate differences and missing work honestly. Register named DEV-*/EXT-*/EXC-* policy in the canonical manifests/contracts. Use tracked DEFERRED for required behavior that is absent. Correct current evidence without rewriting frozen historical reviews.
8. Change the SDK at its source. Merge smolpaws/openhands-agent, then reproducibly re-vendor it through scripts/vendor-openhands-agent.sh. Do not hand-edit the built copy under the server’s vendor/ directory.
9. Prove each claim at the right layer. Provider wire fixtures prove normalization; pinned differential tests prove parity; real-server tests prove persistence; credential-gated live tests prove provider viability. A successful network call alone is not parity evidence.
10. Finish the maintenance chain. Run relevant tests and typechecks, review feedback, merge upstream, synchronize the corresponding enyst/ fork and clean local branches to exact commit IDs, and update Beads. Deploy only after conversations and queued work settle, preserving durable state.

SDK compatibility contract · canonical pin and policy registry · server rules · maintenance notebook.

09Current limits to keep visible

Actual launch path · rename/delete behavior · runtime projections · unimplemented routes · accounting limits · option serialization.

Source map and related pages

The repository contracts are authoritative. This page explains their relationship at the revisions in the header; environment settings are a dated observation, not a permanent guarantee.