design provider model registry - sinrimin/harness-nexus GitHub Wiki
Design: provider model registry — model configs live on the LlmProvider row
Issue #51. Status: SHIPPED 2026-10-14 (migration v19; query-models demoted to the provider-dialog seeder; runtime-config form cut over one-shot; applyPiConfig writes real metadata). Prerequisite for the platform assistant (#52). Supersedes the "query models at every config point" UX of W10/W3.
1. Problem
Every consumer of a provider re-fetches /v1/models at configuration time
(the machine runtime-config form; the assistant would have done the same).
That endpoint returns bare ids — the metadata that actually matters
(context window, max output, vision) is not fetchable anyway. The cure: models
are configured ONCE on the provider row; consumers only pick from the
configured set.
2. Data model
LlmProvider gains models: ProviderModelConfig[]. SQLite: a JSON column on
llm_providers (the chat_prewarm precedent — whole-row read/write, no
cross-row queries, no join shape worth a table).
ProviderModelConfig {
id: string // the API's model id
type: 'chat' | 'embedding' // embedding RESERVED — no consumer yet
displayName?: string // → pi models.json `name`
contextWindow?: number // tokens → pi `contextWindow`
maxOutputTokens?: number // → pi `maxTokens`
vision?: boolean
// —— advanced (provider-level policy, applied per consumer capability) ——
temperature?: number
topP?: number
serverTools?: string[] // ONLY meaningful for api='openai-responses'
// (web_search / file_search / code_interpreter /
// image_generation / mcp …) — OPEN string list,
// the catalog evolves; an enum would rot
extraBody?: Record<string, unknown> // ONLY meaningful for openai-compatible
// apis (openai-chat | openai-responses) — merged
// verbatim into the request body
}
Validation (shared zod): api-irrelevant combos (serverTools on anthropic,
extraBody on anthropic) WARN, never hard-reject — gateway compat layers
routinely bend the rules.
3. API surface
POST/PATCH /api/llm-providersacceptmodels[](full replace, strict shape);GETreturns them. Uniqueness:idunique within a provider row.POST /api/llm-providers/query-modelsSURVIVES, demoted: a seeding helper used only inside the provider edit dialog ("fetch candidate ids", then the user curates params per row). It disappears from every consumer form.
4. Consumer changes (one-shot cutover, no dual-track)
- Runtime-config form (W3/W13): pick provider → dropdown lists its
configured
type='chat'models → choose default + extras. The query-models button is REMOVED from this form. Existing bare-id spec snapshots keep pushing (read-compat), but editing re-picks from the registry. RuntimeConfigSpecDatasnapshot upgrades frommodels: string[]to full model objects →applyPiConfigfinally writes REALcontextWindow/maxTokens/nameinto machines'models.json, so pi's usagecontextSizestops hiding (the W16 honest-degradation note resolves).- Assistant (#52): references
providerId+ default model; the sender's Model select lists the provider'stype='chat'models. No assistant-side model copy. - Embedding: only the
typediscriminator +type==='chat'filtering in every picker. No consumer surface (dimensions, pricing — add when embedding is actually used; a JSON column makes that painless).
5. Honesty rule — advanced params apply per consumer capability
The provider row is the single source of truth; whether a consumer can USE a parameter is its capability, and the UI must not pretend otherwise:
| Consumer | temperature/topP | serverTools/extraBody | contextWindow/maxOutput/name/vision |
|---|---|---|---|
Assistant (pi-ai direct, we own streamFn) |
applied at request build | applied where pi-ai allows; otherwise injected by our streamFn wrapper (S0 spike verifies the seam) | applied (Model metadata) |
| pi machines (W3 push → models.json) | not expressible | not expressible | applied |
| claude-code / opencode / dsh | not expressible | not expressible | target surface decides |
Stored-but-unused is documented, never faked in the UI.
6. Slices
- S1 shared+core:
ProviderModelConfigzod +LlmProvider.modelsdomain type; schema tests. - S2 server: migration (JSON column) + sqlite/memory repos + route accept/return + query-models demoted-to-helper (no route change).
- S3 cli: runtime-config spec carries model objects;
applyPiConfigwrites real metadata; writer tests. - S4 web+sdk: provider FormDialog rework (model sub-table: regular columns + advanced collapse + seed-from-server button), runtime-config form cutover, sdk types. i18n en/zh.
- S5 verify: touched packages typecheck+test; agent-browser pass over both
forms;
scripts/smoke.mjsline for models round-trip.