design phase 9 w10 llm providers - sinrimin/harness-nexus GitHub Wiki
Status: shipped (2026-09-11). Extends the W3 provider-config push β read the W3 section of
design-phase-9-harness-runtime.mdΒ§4.3 first.
W3's provider-config form makes the machine owner hand-assemble the whole spec (label + api flavor + base URL + model + credential) on every machine. In practice the same gateway (with its key and endpoint) is reused across machines, and the model id is copy-pasted blind β the platform never knows which models the endpoint actually serves.
The open-source cc-switch tool
established the UX this wave ports: a provider is a reusable
name + api kind + endpoint + key, and a one-click θ·ε樑ε button
discovers the model list from the endpoint's models API (/v1/models) instead
of pasting ids by hand.
-
LlmProviderentity (global | personalscope, like credentials): a named, reusable LLM route β{ name, api kind, baseUrl?, credentialName }. The API key still lives ONLY in the existing encryptedCredentialstore; the provider references it by name. -
Three api kinds β
openai-chat,openai-responses,anthropicβ and a per-Agent support matrix so the machine page only offers providers the target can actually speak. -
Model-list discovery:
POST /api/llm-providers/query-modelsfetches the provider's model catalog server-side (the plaintext never leaves the server; the browser cannot do this β CORS + key exposure). - Machine page provider-first flow: pick a provider β pick (or fetch) the model β apply. Manual entry stays as a fallback arm.
-
Multi-model (the "ι‘ΊδΎΏηη" question, ground truth in Β§6):
RuntimeConfigSpec.modelsβ an optional list of extra model ids beyond the default. Only dsh has a native home for it (its per-providermodels:list); the dsh writer now writes them.
core/domain/llm-provider.ts
{ id, name, api: 'openai-chat' | 'openai-responses' | 'anthropic',
baseUrl: string | null, // null = the kind's official endpoint
credentialName: string, // handle of a Credential carrying the API key
scope: 'global' | 'personal', ownerId: string | null,
createdAt, updatedAt }
SQLite table llm_providers (no FK β owners outlive rows the same way
credentials do). Name uniqueness is per (name, scope, owner), enforced
read-then-write in the route (409 PROVIDER_NAME_TAKEN), mirroring the
resources key rule. Repository port + UnitOfWork.llmProviders, both
drivers implemented.
Why reference a credential instead of storing the key? The apply path's invariant is "the plaintext leaves the server only inside the daemon's machine-PAT bundle, only for a distributable credential". Reusing the credential store keeps one encryption path, one distributability gate, and rotation for free. The provider is the route, the credential stays the secret.
The W3 RuntimeConfigSpec.api enum stays 'anthropic-messages' | 'openai'
(writers keyed on it are untouched). Providers use a FINER kind because
codex and dsh differ inside "openai":
| kind | speaks | maps to spec api |
|---|---|---|
anthropic |
Anthropic Messages | anthropic-messages |
openai-chat |
OpenAI Chat Completions | openai |
openai-responses |
OpenAI Responses | openai |
Per-Agent support matrix (PROVIDER_API_SUPPORT, shared):
-
claude-code:anthropiconly. -
codex:openai-responsesonly (codex-rs removedwire_api = "chat"β gateways must be Responses-compatible; W3 research Β§8.2). -
deepseek:anthropic+openai-chat(pi-aiKnownApiis exactlyanthropic-messages | openai-completions; W3 research Β§8.1).
The existing runtimeSpecUnsupportedReason gate still applies after mapping β
a provider can never smuggle an unsupported flavor onto a target.
-
providerId?: stringβ provenance only. Validated at PUT time (provider exists + visible to the caller, else404 PROVIDER_NOT_FOUND), echoed by the GET view so the machine form can pre-select the provider row. The spec stays a snapshot: deleting the provider later does not invalidate the stored row (the form falls back to manual prefill). -
models?: string[]β extra switchable model ids beyond the defaultmodel(β€16, deduped server-side). See Β§6.
Stored specs are a JSON column β no migration for either field.
Scope model identical to credentials: global readable by any authenticated
user, admin-only to mutate; personal owner-only; 404 (not 403) hides
foreign rows. Not-found and foreign rows are indistinguishable.
| Route | Behavior |
|---|---|
POST /api/llm-providers |
create; scope:'global' requires admin; referenced credential must be visible to the caller (404 CREDENTIAL_NOT_FOUND); name taken β 409 PROVIDER_NAME_TAKEN
|
GET /api/llm-providers |
caller's personal + all globals |
PATCH /api/llm-providers/:id |
name/api/baseUrl/credentialName mutable; scope immutable (409 PROVIDER_SCOPE_IMMUTABLE); re-validates credential + name |
DELETE /api/llm-providers/:id |
owner-or-admin; stored runtime configs keep applying (snapshot semantics) |
POST /api/llm-providers/query-models |
body {providerId} or {api, baseUrl?, credentialName} (works before the provider is saved β the create dialog and the machine page's manual arm both use it). Resolves + decrypts the credential, fetches the model list, returns {models: {id, name?}[]}
|
Deliberate outbound HTTP surface #2 (the marketplace fetch was #1 β that
allowlist rule is scoped to marketplace browsing). Precedent: the MCP
registry already dials user-configured upstream URLs. Bounds: http(s) only,
GET only, PROVIDER_MODELS_TIMEOUT_MS (default 10 s) via AbortSignal.timeout,
2 MiB response cap, results reduced to ids + display names, β€1000 entries.
-
openai kinds:
GET {base}/models,Authorization: Bearer <key>. Default basehttps://api.openai.com/v1. Candidate URLs: if the configured base already ends with/v1β{base}/models; otherwise try{base}/v1/modelsfirst, then{base}/models(gateways exist in both shapes; a 404 advances to the next candidate). -
anthropic:
GET {base}/v1/models?limit=1000,x-api-key+anthropic-version: 2023-06-01. Default basehttps://api.anthropic.com. - Response shape
{data: [{id, display_name? | name?}]}(a bare array at the root is tolerated for gateway quirkiness); mapped toLlmModelInfo, sorted by id, deduped. - Failures: upstream non-2xx (after candidates) β
502 PROVIDER_MODELS_FAILED; timeout β504 PROVIDER_MODELS_TIMEOUT. The URL builder (providerModelsUrls) is pure and lives in shared (unit-tested, no I/O).
The fetch is injectable ({fetch} option) and the route reads the timeout
from a decoration β tests never touch the network (vi.stubGlobal('fetch', β¦)).
-
New page
/llm-providers(nav: δΎεΊε / Providers,PlugZapIcon), mirroring Credentials: table (name, api badge, baseUrl, credential, scope), FormDialog create/edit (name, api select of the three kinds, baseUrl, credential picker filtered to distributable credentials β every real consumer of a provider is an apply-config), and a per-row θ·ε樑ε dialog listing the discovered ids. -
MachineDetail
ProviderConfigFormis now provider-first:- Loads providers + filters by
PROVIDER_API_SUPPORT[target]. Selecting one auto-fillsproviderLabel(provider name), api (mapped), baseUrl, credentialName; those render as a read-only summary (the only editable baseUrl case: deepseek requires one and the provider has none β then an input appears). - Model field + θ·ε樑ε button β fetches via the selected provider (or explicit fields in manual mode), then offers the list as a Select; the input stays hand-editable.
- ιε 樑ε (multi-model): after a fetch, a collapsible checkbox list of the remaining ids; only the dsh writer consumes them (hint says so).
-
Manual arm (fallback + pre-existing rows whose provider is gone):
the full W3 field set, also with a fetch button (spec flavor maps
openaiβopenai-chatfor the query; the models endpoint is the same for both openai kinds). - Prefill: a stored
providerIdthat still resolves selects that provider; otherwise manual mode with the stored spec fields.
- Loads providers + filters by
What each harness can ACTUALLY do with multiple models (sources:
research-phase-9-harness-runtime.md Β§2/Β§3/Β§8, verified against
codex-rs + dsh 0.1.2-rc.1; research-phase-9-w9-composer-controls.md):
-
dsh β natively multi-model per provider. A route's
models: PiAiModelProfile[]list (onlyidrequired; context defaults at route level) IS the switchable set; dsh groups session model options by provider and pushesconfig_option_updateon topology changes. W3 wrote exactly one entry; W10's writer writesunique([model, ...models]). The default selection staysagent-default-model+ the acp override (model). -
codex β single root
modelstring. Multiplemodel_providers.*blocks are legal but one is active; the session model picker reads codex's built-in presets (models_manager), not config.toml.modelsis ignored. -
claude-code β one default via top-level
model. The documented multi-slot vocabulary (ANTHROPIC_MODEL,ANTHROPIC_DEFAULT_{OPUS,SONNET, HAIKU}_MODEL) exists, but the wrapper's session picker reads SDK model infos, not settings.json.modelsis ignored.
So: machine-level default (W3, unchanged) + pre-seeded switchable set (dsh
only, W10) + in-session switching (W9 configOptions, orthogonal) is the
complete multi-model story this platform can express today.
- Per-provider convenience presets (logos, "official" catalogs) β cc-switch ships a static provider catalog; providers here are user data, not bundled content.
- Writing
ANTHROPIC_DEFAULT_*alias slots for claude-code (the session picker wouldn't read them anyway). - The claude wrapper's
providers/list|set|disablerouting surface (research W9 Β§"deferred"). - Provider health checks / latency probes; pricing data; per-profile model routing.
-
zcode/hermestargets (not runtime-managed).
-
packages/shared/test/llm-provider.test.tsβ schemas, support matrix, kindβflavor mapping, URL candidates. -
packages/server/test/llm-providers.test.tsβ CRUD scope rules, name clash, credential visibility, query-models (stubbed fetch) incl. timeout + upstream-error mapping; PUT runtime-configproviderId/modelsvalidation. -
packages/cli/test/runtime-config.test.tsβ dsh writer emits the deduped multi-modelmodels:list. -
scripts/smoke.mjsβ[9 W10]provider CRUD + provider-mode PUT.
-
codex-acp needs system
libssl.so.3β it is a native Rust binary; anode:20-bookworm-slimmachine container has only Node's bundled OpenSSL, so every spawn died at the dynamic loader untilapt-get install libssl3. The claude wrapper (pure Node) is unaffected. Ops fix + recurrence note indocs/dev/test-rig.md(machine-specific, git-ignored). -
apply-config does not reach ALREADY-OPEN channels (rig-found): a codex
session opened under the OLD spec keeps its startup model β codex-acp
reads
config.tomlonce at spawn. Symptom: user fixes the provider, sends a message in the OLD session, the turn still fires the stale model id at the new gateway β 404 β the turn dies ("εδΊδΈͺζΆζ―ε°±ε΄©δΊ"). Correct flow: apply the config, then open a NEW session. Related W9 boundary: codex's in-session model selector lists its built-in presets (not the gateway's catalog), so on a third-party gateway only the machine default is valid β switching in-session to a preset reproduces the same 404. A future improvement could mark/restart affected channels on apply-config, or pin the applied model as the selector's current value. -
Volcengine Ark note: the coding endpoint
(
β¦/api/coding/v3) DOES speak the OpenAI Responses protocol (codex works against it), while the general/api/v3surface 404s the coding-plan models β provider baseUrl should point at the CODING endpoint for coding-plan keys, and non-coding-plan models are rejected there withUnsupportedModel.
Rig finding: a codex session streaming a chat turn opens with
Model metadata for \deepseek-v4-flash` not found. Defaulting to fallback
metadata; this can degrade performance and cause issues. **as an assistant message chunk** β codex-acp announces the condition in-band, so it landed mid-transcript as if the model had said it. The daemon now drops that exact notice (mapAcpUpdate`, anchored on the stable prefix) since it is an adapter
diagnostic, not output.
The underlying condition is real: codex's built-in model registry does not
know our gateway's custom model ids, so it falls back to a default context
window (observed: 258400). Verified against codex-acp 0.16 that codex's
ROOT-level model_context_window (plus model_max_output_tokens) DOES
override the window it uses β 262144 configured β the adapter reported
249036 (95% usable) β but it does NOT silence the notice. Writing those keys
properly would therefore need a context-window field on the provider spec
(W10's spec carries model IDS only, and the W3 writer's TOML root-key merge is
already the right slot for it). Not scheduled; recorded here so the next wave
does not re-derive it.