design provider model registry - sinrimin/harness-nexus GitHub Wiki

Design: provider model registry — model configs live on the LlmProvider row

Issue #51. Status: SHIPPED 2026-10-14 (migration v19; query-models demoted to the provider-dialog seeder; runtime-config form cut over one-shot; applyPiConfig writes real metadata). Prerequisite for the platform assistant (#52). Supersedes the "query models at every config point" UX of W10/W3.

1. Problem

Every consumer of a provider re-fetches /v1/models at configuration time (the machine runtime-config form; the assistant would have done the same). That endpoint returns bare ids — the metadata that actually matters (context window, max output, vision) is not fetchable anyway. The cure: models are configured ONCE on the provider row; consumers only pick from the configured set.

2. Data model

LlmProvider gains models: ProviderModelConfig[]. SQLite: a JSON column on llm_providers (the chat_prewarm precedent — whole-row read/write, no cross-row queries, no join shape worth a table).

ProviderModelConfig {
  id: string                  // the API's model id
  type: 'chat' | 'embedding'  // embedding RESERVED — no consumer yet
  displayName?: string        // → pi models.json `name`
  contextWindow?: number      // tokens → pi `contextWindow`
  maxOutputTokens?: number    // → pi `maxTokens`
  vision?: boolean
  // —— advanced (provider-level policy, applied per consumer capability) ——
  temperature?: number
  topP?: number
  serverTools?: string[]      // ONLY meaningful for api='openai-responses'
                              // (web_search / file_search / code_interpreter /
                              //  image_generation / mcp …) — OPEN string list,
                              //  the catalog evolves; an enum would rot
  extraBody?: Record<string, unknown> // ONLY meaningful for openai-compatible
                              // apis (openai-chat | openai-responses) — merged
                              // verbatim into the request body
}

Validation (shared zod): api-irrelevant combos (serverTools on anthropic, extraBody on anthropic) WARN, never hard-reject — gateway compat layers routinely bend the rules.

3. API surface

  • POST/PATCH /api/llm-providers accept models[] (full replace, strict shape); GET returns them. Uniqueness: id unique within a provider row.
  • POST /api/llm-providers/query-models SURVIVES, demoted: a seeding helper used only inside the provider edit dialog ("fetch candidate ids", then the user curates params per row). It disappears from every consumer form.

4. Consumer changes (one-shot cutover, no dual-track)

  • Runtime-config form (W3/W13): pick provider → dropdown lists its configured type='chat' models → choose default + extras. The query-models button is REMOVED from this form. Existing bare-id spec snapshots keep pushing (read-compat), but editing re-picks from the registry.
  • RuntimeConfigSpecData snapshot upgrades from models: string[] to full model objects → applyPiConfig finally writes REAL contextWindow/maxTokens/name into machines' models.json, so pi's usage contextSize stops hiding (the W16 honest-degradation note resolves).
  • Assistant (#52): references providerId + default model; the sender's Model select lists the provider's type='chat' models. No assistant-side model copy.
  • Embedding: only the type discriminator + type==='chat' filtering in every picker. No consumer surface (dimensions, pricing — add when embedding is actually used; a JSON column makes that painless).

5. Honesty rule — advanced params apply per consumer capability

The provider row is the single source of truth; whether a consumer can USE a parameter is its capability, and the UI must not pretend otherwise:

Consumer temperature/topP serverTools/extraBody contextWindow/maxOutput/name/vision
Assistant (pi-ai direct, we own streamFn) applied at request build applied where pi-ai allows; otherwise injected by our streamFn wrapper (S0 spike verifies the seam) applied (Model metadata)
pi machines (W3 push → models.json) not expressible not expressible applied
claude-code / opencode / dsh not expressible not expressible target surface decides

Stored-but-unused is documented, never faked in the UI.

6. Slices

  • S1 shared+core: ProviderModelConfig zod + LlmProvider.models domain type; schema tests.
  • S2 server: migration (JSON column) + sqlite/memory repos + route accept/return + query-models demoted-to-helper (no route change).
  • S3 cli: runtime-config spec carries model objects; applyPiConfig writes real metadata; writer tests.
  • S4 web+sdk: provider FormDialog rework (model sub-table: regular columns + advanced collapse + seed-from-server button), runtime-config form cutover, sdk types. i18n en/zh.
  • S5 verify: touched packages typecheck+test; agent-browser pass over both forms; scripts/smoke.mjs line for models round-trip.