Model Hotel exposes three API surfaces: the Proxy API (OpenAI-compatible, for client applications), the Admin API (for management and configuration), and a Health endpoint without authentication.

OpenAI-compatible endpoints that require a virtual key. The proxy covers chat completions plus the multimodal OpenAI API surface (embeddings, images, audio); everything else (fine-tuning, files, assistants, batches) is intentionally out of scope: Model Hotel is a proxy, not a full API gateway.

Authorization: Bearer <virtual-key>

Virtual keys use the sk- prefix (sk- plus 32 hex characters, e.g. sk-a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6). Keys are created via the Admin API and are shown only once at creation time.

The x-api-key header (what the Anthropic SDKs send) carries the virtual key just as well, on every /v1 route rather than only /v1/messages. Authorization: Bearer is preferred when both are present.

All endpoints share the same model routing (hotel/<model> failover or <provider>/<model> direct), virtual-key authentication, allowed_providers access control, rate limiting, circuit breaker, and request logging. See Multimodal Endpoints below.

# List available models
curl http://localhost:8081/v1/models \
  -H "Authorization: Bearer $PROXY_KEY"

# Chat completion (with hotel routing for automatic failover)
curl -X POST http://localhost:8081/v1/chat/completions \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hotel/glm-4.6", "messages": [{"role": "user", "content": "Hello!"}]}'

# Anthropic Messages API (point Claude Code or the anthropic SDK at the gateway;
# x-api-key is accepted alongside Authorization: Bearer)
curl -X POST http://localhost:8081/v1/messages \
  -H "x-api-key: $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hotel/claude-sonnet-4-6", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello!"}]}'

# OpenAI Responses API (point Codex CLI or the openai SDK's Responses client at the gateway)
curl -X POST http://localhost:8081/v1/responses \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hotel/gpt-5.6-sol", "input": "Hello!", "stream": true}'

# Embeddings (multimodal endpoints support the same provider/model and hotel/ routing)
curl -X POST http://localhost:8081/v1/embeddings \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "OpenAI/text-embedding-3-small", "input": "Hello!"}'

# Speech-to-text (multipart upload)
curl -X POST http://localhost:8081/v1/audio/transcriptions \
  -H "Authorization: Bearer $PROXY_KEY" \
  -F model="OpenAI/whisper-1" -F [email protected]

Returns the model list in OpenAI-compatible format.

The list is scoped to what the calling key may actually call: the key's own allowed_providers intersected with its owner account's provider cap, the same pair a chat request is routed through. A key restricted on neither side sees the whole catalogue, so this is a no-op unless an operator has deliberately restricted access. A hotel/ failover group stays listed while any entry in its priority order sits on a provider the caller may reach, and is described by the entry that would actually serve the request.

The listing is all-or-nothing: if either read behind it fails, the endpoint answers 500 rather than a 200 that is missing rows. A partial catalogue is a discovery answer a client routes on, and nothing in the OpenAI-compatible body can mark it as incomplete. A caller that hangs up mid-read gets 499 (client closed request), which the access log records; model listings are not metered, so no request-log row is written either way.

Each id is what a request names to reach that model: <provider>/<model> for a direct model (the provider name normalized, spaces as dashes) and hotel/<group> for a failover group. provider is the provider's name, or hotel for a group, whose other fields describe the entry that would serve it. Optional fields (context_length, max_output_tokens, name, description, modality and more) appear when the catalogue knows them.

Response:

{
  "object": "list",
  "data": [
    {
      "id": "OpenAI/gpt-4o",
      "object": "model",
      "created": 1234567890,
      "owned_by": "openai",
      "provider": "OpenAI"
    },
    {
      "id": "hotel/gpt-4o",
      "object": "model",
      "created": 1234567890,
      "owned_by": "openai",
      "provider": "hotel"
    }
  ]
}

Chat completion endpoint supporting both streaming and non-streaming modes.

Request:

curl -X POST http://localhost:8081/v1/chat/completions \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hotel/glm-4.6",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

Request Body Parameters:

Message normalization: if a message in messages carries tool_calls: [] (an empty array), the proxy removes the field before forwarding. Some clients serialize aborted or filtered tool-call turns this way, and strict providers reject the whole request with a 400 (Invalid 'messages[N].tool_calls': empty array), which permanently breaks any conversation carrying such a turn in its history. Non-empty tool_calls and all other message content pass through untouched.

OpenCode Go session header: OpenCode Go refuses any chat request that carries no x-opencode-session (400 MissingSessionID), because it pins routing and prompt caching to that id. Requests to a provider of type opencode-go therefore always leave the gateway with one, and no other provider type ever sees the header. A client that sends its own x-opencode-session has it forwarded as sent after the safety checks, so its conversation keeps one route and one cache; the value is only accepted when it can safely be a header (trimmed, at most 128 bytes, printable ASCII) and does not claim the gateway's own mh- namespace, and anything else is treated as absent. Absent, the gateway substitutes a stable opaque id derived from the virtual key (mh- plus 32 hex characters), so the same key always maps to the same session while neither the key nor its hash leaves the gateway. Traffic with no virtual key gets a fixed id per surface: admin chat, the retirement probe and the dashboard Test button each have their own. The value is not logged by the proxy.

OpenAI Responses API re-route: OpenAI's newest models refuse function tools combined with reasoning on /v1/chat/completions, and the pro tier (o1-pro, o3-pro, gpt-5.x-pro) is not served there at all. OpenCode Zen and OpenCode Go serve their GPT models the same way and refuse Chat Completions with a 400 "Model does not support this protocol". When a direct OpenAI or OpenCode candidate refuses for any of these reasons, the proxy retries the same request against that provider's /v1/responses, translated in both directions, and remembers the requirement per model so later requests go straight there.

This is invisible to the client: ordinary Chat Completions in and out, streaming included, with the model's reasoning summary surfaced as reasoning_content and token usage metered as usual. Every /v1/responses call sends store: false, so OpenAI retains no conversation state. See Failover & Hotel Routing.

Anthropic Messages route for OpenCode's Claude models: OpenCode Zen and OpenCode Go serve their Claude models (claude-*) over Anthropic Messages alone and refuse Chat Completions with the same 400. A chat request to one of them goes straight to the provider's /messages by name, no refusal needed, translated in both directions by the adapter that serves anthropic-messages providers and built with that type's parameter rules: reasoning_effort becomes extended thinking (with the same dialect self-heal and parameter learning), top_p is dropped, and the Go session header rides along. An Anthropic-in request (/v1/messages) to one of them is forwarded verbatim, as it is to Anthropic itself, and the dashboard's Test button probes /messages too.

Prompt caching on translated Messages routes: every chat request the gateway translates to Anthropic Messages (an anthropic-messages provider, a Claude model on OpenCode, a document request to Anthropic itself) leaves with Anthropic prompt-cache breakpoints placed where the cache hierarchy makes a reusable prefix: the last tool definition, the system prompt (sent as one text block, since only a block carries cache_control), and the last block of each of the final two turns. That is Anthropic's maximum of four and a Chat Completions client cannot add more. Without them Anthropic caches nothing and every turn of an agent loop re-reads its whole context at full price; with them the repeated prefix is read back at the cache-read price, and the counts come back to the client as usage.prompt_tokens_details.cached_tokens (plus Anthropic's own cache_read_input_tokens / cache_creation_input_tokens). Prompts under a model's minimum cacheable length are simply not cached. Cache writes are priced too: the tokens Anthropic reports as cache_creation_input_tokens carry its write premium (1.25x the input price) in the row's cost_usd, on top of the cache-hit price for the read ones; a 1-hour write an Anthropic-in client requests with cache_control.ttl (reported as cache_creation.ephemeral_1h_input_tokens) carries its 2x. A Messages endpoint that rejects the cache_control field itself as unknown (Anthropic never refuses the field; an operator-entered endpoint might) is learned per model from its 400 and served without breakpoints from then on, the same self-heal that learns the thinking dialect; Anthropic's own body-specific cache_control complaints do not count. The retirement probe follows the same learned facts and places breakpoints like served traffic; only the dashboard Test button sends none. Anthropic-in requests (/v1/messages) are forwarded verbatim with whatever cache_control the client placed.

Model Routing:

• hotel/<model> - Failover routing (tries all providers that offer the model, moving to the next one on an upstream 5xx, 401, 402, 403, 404 or 499, and on 429 while failover_on_rate_limit is on, the default)
• <provider>/<model> - Direct routing to a specific named provider (no failover). The provider name is carried in the form routing uses, with every space replaced by a hyphen: a provider named my provider is addressed as my-provider/<model>

Streaming Response Format:

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

The native Anthropic Messages API surface, so Claude Code and the anthropic SDKs can drive the gateway directly and fail over across every provider in a hotel/ group, not just Claude. Model routing is identical to the rest of the proxy: send hotel/<group> or <provider>/<model> in the model field. Authenticate with x-api-key (what Anthropic clients send) or Authorization: Bearer.

curl -X POST http://localhost:8081/v1/messages \
  -H "x-api-key: $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hotel/claude-sonnet-4-6",
    "max_tokens": 1024,
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Two serving modes are chosen automatically, per failover attempt:

• Native passthrough when the resolved candidate is an Anthropic-family provider: the original Messages body is forwarded to the provider's own /v1/messages and the response is streamed back verbatim, so extended-thinking blocks (thinking / signature_delta), cache_control, and fine-grained tool streaming survive end to end.
• Translation for every other provider: the request is converted to the OpenAI Chat Completions shape (system prompt, text, vision, tools, tool_choice, and multi-turn tool_use/tool_result), run through the same pipeline, and the OpenAI response, SSE stream, or error is converted back to the Anthropic wire format on the way out. A Gemini 3 candidate signs each tool call and refuses the next turn without the signature; the tool_use block has no member for it, so the translation carries it inside the block's id as an opaque suffix in the id alphabet (around a kilobyte), the client echoes the id back on the tool_use block and the tool_result as it does any id, and the translation recovers the signature on the way in. A later attempt served natively by an Anthropic provider gets the ids with the suffix stripped. Treat tool_use ids as opaque strings of any length.

Because the choice is per attempt, a single hotel/claude-* request served natively by Anthropic transparently fails over to a translated provider (e.g. another vendor offering the same model) if Anthropic is unavailable. The translated path drops thinking output (v1); use a provider that serves the model natively to preserve it. Streaming, tool use, multi-turn tool results, vision, document blocks (base64 documents become OpenAI file parts, text documents become text parts; a document behind a remote URL is dropped), and Anthropic-shaped errors are all supported. A file part carrying its document inline (a data: URI in file.file_data) and bound for Google AI Studio is served through the native generateContent route, since Google's OpenAI-compatibility layer rejects file parts; a file_id or a remote URL in file.file_data is not rerouted, because neither route can fetch it. Token usage is metered the same way as chat and request/response content is never logged.

The OpenAI Responses API surface, so Responses-only clients (Codex CLI, the openai SDKs' responses client) can drive the gateway directly and fail over across every provider in a hotel/ group. Model routing is identical to the rest of the proxy: send hotel/<group> or <provider>/<model> in the model field.

curl -X POST http://localhost:8081/v1/responses \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hotel/gpt-5.6-sol",
    "instructions": "Be terse.",
    "input": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

Codex CLI configuration (~/.codex/config.toml):

[model_providers.hotel]
name = "Model Hotel"
base_url = "http://localhost:8081/v1"
env_key = "MODEL_HOTEL_KEY"
wire_api = "responses"

Then codex -c model_provider=hotel -m hotel/<group> (or -m <provider>/<model>), with the virtual key in MODEL_HOTEL_KEY.

Two serving modes are chosen automatically, per failover attempt:

• Native passthrough when the resolved candidate is OpenAI itself (api.openai.com): the original Responses body is forwarded to OpenAI's own /v1/responses with only model rewritten, and the response or event stream comes back verbatim, so hosted tools (web_search and the rest), include: ["reasoning.encrypted_content"], prompt_cache_key, text.verbosity and every other knob survive end to end.
• Translation for every other provider (an OpenAI-compatible relay of the openai type included): the request is converted to Chat Completions (instructions and developer turns become system messages, input messages with input_text / input_image / inline input_file parts, function_call / function_call_output history, function tools, tool_choice, max_output_tokens, reasoning.effort, text.format), run through the same pipeline, and the answer is rendered back as a Response object or as the Responses event stream (response.created ... response.output_item.added / response.output_text.delta / response.output_item.done ... response.completed, every event carrying a sequence_number). A function call closes with its full arguments on response.output_item.done, reasoning summaries stream as response.reasoning_summary_text.delta, a refusal streams as a refusal content part, a length or content_filter stop becomes response.incomplete with the matching incomplete_details.reason, and a failure ends the stream with response.failed. The Response object echoes the request's instructions, temperature, top_p, parallel_tool_calls, tool_choice and metadata; tools is reported empty. A tool inside a namespace tool is offered to the provider as <namespace>__<name> and comes back as a function_call item with namespace and the bare name; every tool name, namespaced or not, must be distinct. Non-streaming and streaming errors keep the OpenAI error envelope, which the Responses API shares.

The gateway keeps no conversation state, so every turn re-sends the transcript (what Codex does with store: false). Refused with a 400 naming the field, on both modes: previous_response_id, conversation, store: true, background: true, item_reference and compaction input items. Hosted tools other than web_search (file_search, code_interpreter, image_generation, computer, mcp, shell, apply_patch, local_shell), custom tools, file ids and audio parts are served only by OpenAI's own endpoint: a request carrying one is accepted when every resolved candidate is OpenAI itself, and refused with a 400 naming the member when any candidate would be translated, since no chat provider can execute them. A web_search tool is dropped on translated routes, because Codex advertises one to every provider by default and a provider behind a hotel/ group simply has no search. Prior reasoning items in input (encrypted reasoning from OpenAI) are dropped on translated routes: the model reasons fresh from the transcript. Because the mode is chosen per attempt, a hotel/ request served natively by OpenAI fails over to a translated provider when OpenAI is unavailable. Requests never hedge on this endpoint. Token usage is metered the same way as chat (the native mode reads the Responses usage block, cached tokens included) and request/response content is never logged.

The multimodal endpoints are transparent pass-through: the request is forwarded to the resolved provider with only the model field rewritten to the upstream model ID, and the provider's response is returned verbatim (JSON, SSE stream, or binary audio). The proxy extracts token counts from the usage object for metering; request and response content is never inspected or logged (see Privacy).

Failover applies the same way as chat: with hotel/<model> routing, an upstream 5xx, 401, 402, 403, 404 or 499 (and a 429 while failover_on_rate_limit is on) moves to the next provider in the group. Failover happens only before any response byte has been forwarded; once a binary or SSE stream starts, the proxy is committed to that provider.

curl -X POST http://localhost:8081/v1/embeddings \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "OpenAI/text-embedding-3-small", "input": ["Hello!", "World"]}'

Standard OpenAI body (input, dimensions, encoding_format); response is the provider's embeddings list with usage.prompt_tokens metered.

curl -X POST http://localhost:8081/v1/rerank \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "Cohere/rerank-v3.5", "query": "what is a capybara", "documents": ["The capybara is a giant rodent.", "Paris is the capital of France."], "top_n": 2}'

Cohere-style rerank body (query, documents, top_n), the de-facto standard shape also served by Jina, Voyage, and local TEI servers; the response (results with index and relevance_score) is returned verbatim. Cohere providers are routed to the native https://api.cohere.com/v2/rerank endpoint automatically (rerank is not part of Cohere's OpenAI-compatibility surface); any other provider receives the request at <base URL>/rerank. Cohere rerank models are auto-discovered alongside chat models, and hotel/<name> routes through a failover group like any other model once two providers list the same rerank id (see Failover and Hotel Routing).

Metering follows the provider's billing unit. Cohere bills per search unit (one query against up to 100 documents; a document over 500 tokens counts as several) and reports it as meta.billed_units.search_units: the proxy reads that figure, stores it on the request log row (search_units) and prices the row at the model's search_price_per_thousand (USD per 1,000 searches, from the built-in Cohere catalog or set by hand on the model), so rerank spend reaches the dashboard spend views and dollar budgets. Providers reporting usage.total_tokens (Jina, Voyage) are metered and priced per token as before. The virtual key's token limits are charged an estimate from the query and documents in either case, since a rerank answer carries no token usage.

curl -X POST http://localhost:8081/v1/images/generations \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "OpenAI/gpt-image-1", "prompt": "a hotel for AI models", "size": "1024x1024"}'

All provider parameters (n, size, quality, response_format, style, output_format, partial_images, stream) pass through untouched. Streaming partial images (SSE) are forwarded verbatim.

multipart/form-data with image file(s), model, and the provider's other form fields. The proxy parses the form once to read model, then rebuilds it per failover candidate with the resolved model ID; file bytes are forwarded unmodified.

curl -X POST http://localhost:8081/v1/audio/speech \
  -H "Authorization: Bearer $PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "OpenAI/gpt-4o-mini-tts", "input": "Welcome to Model Hotel", "voice": "alloy"}' \
  --output speech.mp3

Gemini TTS models (gemini-2.5-flash-preview-tts and kin on Google AI Studio or Vertex AI express) have no speech route on Google's OpenAI-compatibility layer, so the proxy serves them through generateContent instead of passing the request through: the text becomes a request for an AUDIO response with a speechConfig naming the voice, and the PCM the model answers with is delivered as the audio the client asked for. What that means for the request:

• response_format may be wav (the default when absent, in place of OpenAI's mp3) or pcm (raw 16-bit mono at the model's 24 kHz). The compressed formats (mp3, opus, aac, flac) need an encoder the gateway does not carry and are refused with a 400 naming the two it takes; in a hotel/ group holding a model that does produce them, the request fails over to that model instead.
• voice takes OpenAI's names (alloy, echo, fable, onyx, nova, shimmer, ash, ballad, coral, sage, verse, cedar, marin), each mapped to a Gemini prebuilt voice, or any Gemini voice name (Kore, Puck, Zephyr, ...) as it is. Absent means Kore.
• speed and instructions have no counterpart and are ignored; Gemini takes delivery style from the text itself ("Say cheerfully: ...").
• stream_format is not honoured; the whole clip is returned at once.

Usage is metered from the token counts the answer reports.

Except for the Gemini TTS models described above, the response is the provider's raw binary audio (Content-Type passed through, e.g. audio/mpeg), streamed with no buffering, and stream_format: "sse" responses stream through as SSE.

curl -X POST http://localhost:8081/v1/audio/transcriptions \
  -H "Authorization: Bearer $PROXY_KEY" \
  -F model="OpenAI/whisper-1" -F [email protected]

multipart/form-data with file and model (plus optional language, response_format, temperature, etc.). Returns the provider's JSON ({"text": ...}) or alternate response_format output verbatim.

Upload size: request bodies are capped by the MAX_REQUEST_SIZE environment variable (default 50MB, which accommodates OpenAI's 25MB audio limit plus multipart overhead). See Configuration.

Per-key rate limiting applies based on virtual key configuration. Returns 429 Too Many Requests when exceeded.

{
  "error": {
    "message": "rate limit exceeded",
    "type": "rate_limit_error",
    "code": 429
  }
}

The message names the limiter that refused: rate limit exceeded for a per-key or per-IP request cap, user rate limit exceeded for the caller's account-wide request cap, token rate limit exceeded and user token rate limit exceeded for the tokens-per-minute equivalents, and key budget exceeded or user budget exceeded (with the spend and the budget in the text) for a dollar budget, whose Retry-After runs to the end of the budget period.

Requires a signed-in identity: the admin token, or a session. What a caller may reach depends on its role and grants.

Authorization: Bearer <admin-token>

The admin token is generated on first startup (or taken from ADMIN_TOKEN) and its hash saved to <DATA_DIR>/admin-token (/data/admin-token in the Docker Compose setup). The plaintext is shown only once in the startup logs. With TOTP enabled the bare admin token no longer authorizes /api/* and must be exchanged via /api/totp/login.

A session works in its place. Every login front-end mints one: password (/api/auth/login), passkey, TOTP, OIDC or GitHub SSO, or the admin-token exchange (/api/auth/admin-exchange). The dashboard carries it in an HttpOnly cookie, and a cookie-authenticated POST/PUT/PATCH/DELETE must also send a matching X-CSRF-Token header (403 otherwise); a session token sent as Authorization: Bearer needs no CSRF header.

Roles and grants. The admin token and admin accounts reach everything. A non-admin user account reaches only what its grants open:

Any signed-in identity, grants or not, may use /api/auth/me, /api/auth/password, /api/auth/sessions*, the per-user /api/auth/totp/* routes, /api/system, /api/version/latest and /api/events. Everything else is admin only: provider writes, discovery, quota (/usage, /balance, /account), model writes and tests, /api/logs/purge, app logs, audit, settings, alerts, failover groups, backups, users, config sync and fleet routes. A missing grant is a 403.

Response:

[
  {
    "id": "uuid",
    "name": "OpenAI",
    "base_url": "https://api.openai.com/v1",
    "provider_type": "openai",
    "masked_key": "sk-p...c5d6",
    "enabled": true,
    "autodiscovery_enabled": true,
    "scheduled_disable_on": null,
    "quota_reserve_percent": 0,
    "max_in_flight": null,
    "last_discovered_at": "2024-01-01T00:00:00Z",
    "last_used_at": "2024-01-01T00:00:00Z",
    "created_at": "2024-01-01T00:00:00Z",
    "updated_at": "2024-01-01T00:00:00Z",
    "model_count": 15,
    "total_tokens": 1234567,
    "tokens_since": "2024-01-01T00:00:00Z"
  }
]

The plaintext API key is never returned; masked_key is a display-only preview. tokens_since (the timestamp of the oldest request log behind total_tokens) is omitted when the provider has no logged traffic, and a last_cap object is added only when the provider has answered an exhausted 429 since the process started. Non-admin callers see only their own traffic in total_tokens.

Request Body:

{
  "name": "OpenAI",
  "base_url": "https://api.openai.com/v1",
  "provider_type": "openai",
  "api_key": "sk-..."
}

Names must be routable. A request names a model as <provider>/<model> and the proxy splits it at the first /, so a name containing / could never be reached: it is refused with 400 and the code provider_name_slash. Every hotel/... model goes to a failover group, so the exact name hotel is refused with provider_name_reserved (Hotel or my hotel are fine: the prefix match is case-sensitive and whole). Spaces are allowed; see below. The rule applies to new names only: a provider that already has such a name keeps it, a save that resends it unchanged is accepted, and fleet sync carries it to members unchanged.

Names are unique in the form routing uses. A <provider>/<model> id replaces every space in the provider name with a hyphen, so my provider and my-provider are one name: creating the second, or renaming a provider onto it, returns 409. Renaming a provider to its own other spelling is allowed, since the name it routes under does not change. An install that already holds such a pair, from before the rule existed, refuses to start until one of them is renamed, because either name may be the one clients use. The dashboard is down at that point, so the rename happens in the database (the startup message names the pair): docker compose exec db psql -U modelhotel -d modelhotel -c "UPDATE providers SET name = 'my provider 2' WHERE name = 'my-provider'", then start again. Restoring a backup taken before the rule is the usual way to reach this.

Self-hosted providers must name their type. Omitting provider_type derives it from the hostname only, so http://box:11434 becomes a generic OpenAI-compatible provider rather than an Ollama one: no native discovery, no keyless waiver. Scripts that used to rely on the old port detection (11434 / 5001 / 1234) need "provider_type": "ollama" | "koboldcpp" | "lmstudio" | "localai" | "sglang" | "tabbyapi" added. When a self-hosted type is named, the address is probed before the provider is saved and the request fails with provider_type_mismatch, provider_type_unconfirmed or provider_unreachable if the server does not answer as that type, so the server must be running.

An Anthropic Messages endpoint must name its type too, for the same reason: anthropic-messages describes a wire format, not a vendor, so no hostname implies it. Named, the provider's chat traffic is translated to Anthropic's /v1/messages instead of being sent to /v1/chat/completions, and models are discovered from <base_url>/v1/models in Anthropic's shape. Omitted, the same base URL becomes a generic OpenAI-compatible provider whose every request would go to an endpoint that is not there. api.anthropic.com still derives anthropic, which serves the OpenAI-compatible route by default. As with any host that is not a known vendor's, a restricted install has to list the endpoint in ALLOWED_PROVIDER_HOSTS before the provider can be created.

Response: 201 Created with provider object

Request Body: all fields optional for partial update. Accepts name, base_url, provider_type, api_key, enabled, autodiscovery_enabled, scheduled_disable_on, max_in_flight and quota_reserve_percent; POST accepts only the first four of those.

provider_type can be corrected here, which matters for a row the legacy hostname rules filed under the wrong type: re-adding the provider instead would cascade its models away. A new self-hosted type is probed exactly as on create, as is a changed base_url. scheduled_disable_on is an ISO date (YYYY-MM-DD) that must not be in the past, or null to clear it. max_in_flight caps the provider's concurrent upstream requests: null means no ceiling, and any number outside 1-10000 is a 400. quota_reserve_percent is the share of the provider's longest quota window (weekly, or monthly for OpenCode Go) kept back for use outside the gateway: 0 (the default) drains it fully, 10 to 90 in steps of ten pins the provider once that much of that window is left; null reads as 0, anything else is a 400. See Quota reserve.

Response: 204 No Content

Cascades to delete associated models and updates failover groups.

Triggers manual model discovery for a specific provider.

Response:

{
  "discovered": 15,
  "models": [...],
  "diff": {
    "added": [{"model_id": "gpt-4o-2024-11", "reason": "new_model"}],
    "reenabled": [{"model_id": "o3-mini", "reason": "reappeared"}],
    "disabled": [{"model_id": "gpt-4o-2024-05", "reason": "not_listed"}],
    "updated": [{"model_id": "gpt-4o", "changes": [{"field": "input_price", "old": 5.0, "new": 2.5}]}],
    "failover_deleted_groups": [
      {"display_model": "o1-preview", "reason": "only 1 enabled provider (need 2+ for failover)", "provider_count": 1, "provider_names": []}
    ],
    "failover_updated_groups": [
      {"display_model": "glm-4.6", "removed_model_ids": ["uuid-old"], "added_model_ids": ["uuid-new"]}
    ],
    "failover_disabled_groups": [
      {"display_model": "my-custom-group", "effective_count": 1, "reason": "..."}
    ]
  }
}

The diff summarizes what the scan changed (all sections omitted when empty). Reasons are machine-readable codes: new_model, reappeared, not_listed. updated lists existing models whose metadata changed, one changes entry per field (input_price, output_price, input_price_cache, search_price, context_length), with old / new omitted when that side was unset. Failover groups of newly disabled models are re-synced as part of the scan; the resulting group changes appear in the three failover_* sections, failover_disabled_groups naming custom groups switched off because fewer than two routable members remain (their membership is kept). The dashboard shows this diff as a post-scan summary modal.

Returns usage/quota information for supported providers. Answers come from the stored quota snapshot (fetched live once on a cold miss), with the snapshot time in the X-Quota-Fetched-At header (RFC3339). A provider with no quota data to report (a NeuralWatt free-tier key, an OpenCode Go key with no subscription) answers 204 No Content; a dead or revoked provider key answers 424 Failed Dependency; a provider whose type serves a different kind (or none) is a 400. The same applies to /balance and /account.

Supported providers:

• zai-coding (Z.AI)
• nanogpt (Nano-GPT)
• openrouter (OpenRouter - returns key balance)
• neuralwatt (NeuralWatt - returns quota; 404 from the upstream quota endpoint means a free-tier key and yields no data)
• kimi-code (Kimi Code - returns 5-hour/weekly quota, parallel-request limit, and membership tier)
• minimax (MiniMax - returns 5-hour/weekly Token Plan quota per model class)
• opencode-go (OpenCode Go - returns rolling/weekly/monthly subscription usage; a 403 EntitlementError from the upstream usage endpoint means a key with no active Go subscription and yields no data)

Response (Z.AI example, abbreviated):

{
  "code": 200,
  "msg": "Operation successful",
  "success": true,
  "data": {
    "level": "pro",
    "limits": [
      {
        "type": "TOKENS_LIMIT",
        "unit": 3,
        "number": 5,
        "usage": 800000000,
        "currentValue": 12000000,
        "remaining": 788000000,
        "percentage": 1.5,
        "nextResetTime": 1784491200000,
        "usageDetails": [{"modelCode": "glm-4.6", "usage": 12000000}]
      }
    ]
  }
}

The service decodes the Z.AI quota payload into its modelled fields (the ones shown) and re-serializes them; usageDetails is omitted when empty and nextResetTime is epoch milliseconds.

Response (Kimi Code example, abbreviated):

{
  "user": {"userId": "u_123", "region": "global", "membership": {"level": "standard"}},
  "limits": [
    {"window": {"duration": 300, "timeUnit": "TIME_UNIT_MINUTE"}, "detail": {"limit": "100", "remaining": "62", "resetTime": "2026-07-19T18:00:00Z"}},
    {"window": {"duration": 10080, "timeUnit": "TIME_UNIT_MINUTE"}, "detail": {"limit": "700", "remaining": "410", "resetTime": "2026-07-24T00:00:00Z"}}
  ],
  "parallel": {"limit": "4"},
  "subType": "coding"
}

The service decodes the Kimi Code /usages payload into its modelled fields and re-serializes them (fields it does not model are dropped). Besides those shown, the modelled fields are a top-level usage and totalQuota block, and every detail, usage and totalQuota block carries limit, used, remaining and resetTime, all strings, empty when Kimi omitted them; the dashboard derives 5-hour and weekly percentages from the limits array by matching each window's duration (300 minutes = 5h, 10080 minutes = weekly).

Response (OpenCode Go example):

{
  "usage": {
    "rolling": {"status": "ok", "percent": 12, "resetsAt": "2026-09-08T18:25:46.155Z"},
    "weekly": {"status": "ok", "percent": 0, "resetsAt": "2026-09-14T00:00:00.155Z"},
    "monthly": {"status": "ok", "percent": 3.5, "resetsAt": "2026-10-08T13:25:13.155Z"}
  }
}

The service re-serializes the modelled fields of the /usage payload; fields not listed here are dropped. percent is the share of the window already consumed (0 to 100), resetsAt is RFC3339, and status is documented only for "ok": any other value is OpenCode Go reporting the window as unusable. A key whose Go subscription has lapsed answers 403 EntitlementError, which is served as 204 No Content (no response body, no badge); the null exists only in the persisted snapshot payload behind it. Any other 403, and the 401 a revoked key answers, is served as 424 like every other dead credential.

Response (MiniMax example):

{
  "model_remains": [
    {
      "model_name": "general",
      "current_interval_total_count": 4500,
      "current_interval_usage_count": 0,
      "current_weekly_total_count": 45000,
      "current_weekly_usage_count": 0,
      "current_interval_status": 1,
      "current_interval_remaining_percent": 100,
      "current_weekly_status": 1,
      "current_weekly_remaining_percent": 100,
      "start_time": 1784473200000,
      "end_time": 1784491200000,
      "remains_time": 16420081,
      "weekly_start_time": 1784073600000,
      "weekly_end_time": 1784678400000,
      "weekly_remains_time": 30820081
    }
  ],
  "base_resp": {"status_code": 0, "status_msg": "success"}
}

The service decodes the MiniMax /token_plan/remains payload into its modelled fields, including base_resp, and re-serializes them (fields it does not model are dropped). Times are epoch milliseconds and *_remains_time a countdown in milliseconds to the window's reset; *_usage_count is what was used of *_total_count, while the *_remaining_percent fields are what remains, not what was consumed (some plans report all-zero counts, leaving the percentage as the only signal). MiniMax reports business errors (such as 2062 for "no active token plan subscription") inside an HTTP 200 response, so the dashboard checks base_resp.status_code rather than the HTTP status to decide whether quota data is available.

Returns balance information for supported providers.

Supported providers:

• deepseek

Response:

{
  "is_available": true,
  "balance_infos": [
    {"currency": "CNY", "total_balance": "100.50", "granted_balance": "0.00", "topped_up_balance": "100.50"}
  ]
}

DeepSeek's /user/balance payload, re-serialized from its modelled fields. The balances are strings, one entry per currency.

Returns account information for supported providers.

Supported providers:

• ollama-cloud

Response:

{
  "id": "...",
  "email": "[email protected]",
  "name": "...",
  "plan": "pro",
  "customer_id": {"string": "cus_...", "valid": true},
  "subscription_id": {"string": "sub_...", "valid": true},
  "subscription_period_start": {"time": "2026-09-01T00:00:00Z", "valid": true},
  "subscription_period_end": {"time": "2026-10-01T00:00:00Z", "valid": true},
  "suspended_at": {"time": "0001-01-01T00:00:00Z", "valid": false}
}

Ollama Cloud's /api/me payload, re-serialized from its modelled fields. Nullable values arrive as {string|time, valid} pairs, valid: false meaning null. No credit or usage figure is reported.

Triggers discovery for all enabled providers.

Response:

{
  "results": [
    {"provider_name": "OpenAI", "discovered": 15, "diff": {"added": [{"model_id": "gpt-4o-2024-11", "reason": "new_model"}]}},
    {"provider_name": "Anthropic", "discovered": 5, "diff": {}},
    {"provider_name": "Broken", "discovered": 0, "error": "connection refused"}
  ],
  "succeeded": 2,
  "failed": 1,
  "discovered": 20
}

Each successful result carries the same per-provider diff as the single-provider endpoint (omitted when the provider's scan failed).

Refreshes quota/balance information for all providers that support it.

Response:

{
  "results": [
    {"provider_name": "Z.AI", "provider_type": "zai-coding", "refreshed": true},
    {"provider_name": "DeepSeek", "provider_type": "deepseek", "refreshed": true}
  ],
  "refreshed": 2,
  "failed": 0,
  "skipped": 5
}

Cursor (keyset) pagination walks the list by passing the previous response's next_cursor back instead of an offset, so a page stays stable while rows are inserted ahead of it.

Query Parameters:

Response:

[
  {
    "id": "uuid",
    "model_id": "gpt-4o",
    "name": "GPT-4o",
    "display_name": "GPT-4o",
    "provider_id": "uuid",
    "provider_name": "OpenAI",
    "provider_enabled": true,
    "capabilities": "{\"streaming\":true,\"vision\":true,\"reasoning\":false,\"audio_input\":false}",
    "context_length": 128000,
    "max_output_tokens": 16384,
    "input_price_per_million": 5.0,
    "output_price_per_million": 15.0,
    "input_price_per_million_cache_hit": 2.5,
    "search_price_per_thousand": null,
    "price_sources": {"input": "provider", "cache_hit": "modelsdev", "output": "provider"},
    "owned_by": "openai",
    "description": "Most capable model",
    "params": "{\"temperature\":0.7}",
    "modality": "text",
    "input_modalities": "[\"text\",\"image\"]",
    "output_modalities": "[\"text\"]",
    "enabled": true,
    "disabled_manually": false,
    "price_customized": false,
    "limits_customized": false,
    "capabilities_customized": false,
    "capabilities_editable": false,
    "provider_type": "openai",
    "created_at": "2024-01-01T00:00:00Z",
    "last_seen_at": "2024-01-01T00:00:00Z"
  }
]

capabilities, params, input_modalities and output_modalities are JSON documents carried as strings ("{}" / "[]" when unknown), so a client parses them a second time. search_price_per_thousand is the rerank price per 1,000 search units, null for token-billed models. disabled_manually marks a model an operator switched off (as opposed to one discovery disabled); price_customized, limits_customized and capabilities_customized mark prices, context limits and capabilities pinned against discovery. provider_type is the stored type of the model's provider. capabilities_editable says whether the capabilities can be set by hand (a custom or self-hosted provider).

price_sources says where each stored price came from, keyed by price field: provider (the provider's own listing), catalog (Model Hotel's embedded override), modelsdev (enrichment) or manual (an operator edit). A key is absent while that price is unset or was stored before sources were recorded.

Request Body: (all fields optional, but at least one is required: an empty body is a 400)

{
  "display_name": "Custom Name",
  "context_length": 128000,
  "max_output_tokens": 16384,
  "input_price_per_million": 5.0,
  "input_price_per_million_cache_hit": 2.5,
  "output_price_per_million": 15.0,
  "search_price_per_thousand": 2.0,
  "price_customized": true,
  "limits_customized": true,
  "capabilities": {"streaming": true, "vision": true, "tool_calling": true},
  "capabilities_customized": true,
  "enabled": true
}

Validation:

• display_name: 1-128 characters (empty clears it back to the discovered name)
• context_length: 256-2000000
• max_output_tokens: 1-128000
• input_price_per_million: 0-1000
• input_price_per_million_cache_hit: 0-1000
• output_price_per_million: 0-1000
• search_price_per_thousand: 0-1000
• limits_customized: boolean; any edit of context_length or max_output_tokens pins both against discovery, and false clears the pin and both limits so the next scan refills them
• capabilities: object of capability flags (streaming, vision, video_input, audio_input, reasoning, tool_calling, parallel_tool_calls, structured_output, pdf_upload), a custom or self-hosted provider's models only (capabilities_editable in the response says which); replaces the stored flags whole (an omitted flag is false; the dashboard, which has no streaming toggle, sends streaming back as stored) and pins them against discovery
• capabilities_customized: boolean, a custom or self-hosted provider's models only; false clears the pin so the next scan writes the server's reading again. capabilities, or capabilities_customized: true, on another provider type's model is a 400 with code capabilities_not_editable; the unpin is taken on any type, so a pin left behind by a provider type change can still be cleared
• price_customized: boolean; marks the prices as operator-set so discovery enrichment leaves them alone. An edited price is recorded as manual in the model's price_sources; unpinning clears the prices and their sources so the next scan writes both afresh.

Response: 204 No Content

Tests a model by sending a minimal prompt and measuring response. A rerank model is probed on its own route instead (one document, top_n: 1), and the answer reports ranked_results in place of response; a 200 that ranks nothing is reported as a failure, the verdict the proxy reaches for the same body. Every probe writes one request log row under virtual_key_name: internal, with the family it was sent on in endpoint_type, and priced like live traffic (cost_usd; a rerank probe also records its search_units).

Response:

{
  "success": true,
  "streaming": true,
  "duration_ms": 234,
  "ttft_ms": 123,
  "response_header_ms": 98,
  "response": "Hi"
}

streaming says whether the probe streamed; ttft_ms and response_header_ms are omitted when not measured.

For a rerank model:

{
  "success": true,
  "streaming": false,
  "duration_ms": 296,
  "response": "",
  "ranked_results": 1
}

On error:

{
  "success": false,
  "streaming": false,
  "duration_ms": 5000,
  "error": "HTTP 401: Invalid API key"
}

Response:

{
  "groups": [
    {
      "id": "uuid",
      "display_model": "glm-4.6",
      "display_name": "GLM 4.6 Failover",
      "description": "Primary failover group",
      "group_enabled": true,
      "auto_created": false,
      "auto_disabled": false,
      "entries": [
        {
          "model_uuid": "uuid",
          "model_id": "z-ai/glm-4.6",
          "provider_id": "uuid",
          "provider_name": "OpenRouter",
          "display_name": "GLM 4.6",
          "enabled": true,
          "model_enabled": true,
          "provider_enabled": true,
          "disabled_manually": false,
          "context_length": 200000,
          "owned_by": "z-ai"
        }
      ],
      "total_tokens": 123456,
      "created_at": "2024-01-01T00:00:00Z",
      "updated_at": "2024-01-01T00:00:00Z"
    }
  ],
  "last_synced_at": "2024-01-01T00:00:00Z"
}

auto_disabled marks a group switched off because it fell below two routable members (by sync or by the dashboard's floor cascade), not by an operator. An entry's disabled_manually tells an operator-disabled model from one discovery disabled.

Request Body:

{
  "display_model": "glm-4.6",
  "display_name": "GPT-4o Failover",
  "description": "Optional description",
  "entry_ids": ["uuid-1", "uuid-2", "uuid-3"]
}

Response: 201 Created with failover group object

Request Body: (all fields optional)

{
  "display_name": "Updated Name",
  "display_model": "glm-4.6",
  "description": "Updated description",
  "group_enabled": true,
  "priority_order": ["uuid-2", "uuid-1", "uuid-3"],
  "entry_enabled": {
    "uuid-1": true,
    "uuid-2": false,
    "uuid-3": true
  }
}

Validation: an active group must keep at least one enabled entry. Turning a group on (the off to on transition only) additionally requires at least 2 routable members, meaning entries whose model and whose provider are both enabled; short of that the request is a 400.

Find which failover group contains a given model UUID.

Response:

{
  "id": "uuid",
  "display_model": "glm-4.6",
  "position": 1,
  "total_entries": 3
}

Returns 404 if the model is not found in any failover group.

Re-synchronizes all failover groups with current model database state.

Response:

{
  "deleted_groups": [],
  "updated_groups": [{"display_model": "glm-4.6", "removed_model_ids": ["uuid-old"], "added_model_ids": ["uuid-new"]}],
  "purged_entries": [{"group_display_model": "glm-4.6", "pruned_model_ids": ["uuid-gone"]}],
  "disabled_groups": [{"display_model": "my-custom-group", "effective_count": 1, "reason": "..."}],
  "sync_errors": []
}

deleted_groups is always present; the other four are omitted when empty.

Returns available models that can be added to failover groups.

Response:

[
  {
    "model_uuid": "uuid",
    "model_id": "gpt-4o",
    "provider_id": "uuid",
    "provider_name": "OpenAI",
    "display_name": "GPT-4o",
    "context_length": 128000,
    "owned_by": "openai"
  }
]

Clears one provider's circuit, returning it to rotation immediately instead of waiting out the cooldown. Use it when you have fixed the provider yourself (rotated a dead key, restarted a local runtime, topped up a spent plan) and do not want to wait for the breaker to find out.

Response:

{
  "provider_id": "uuid",
  "previous_state": "open",
  "reset": true
}

Resetting an untracked or already-closed provider is a successful no-op (previous_state: "closed", reset: false), not an error: the breaker only tracks providers it has routed, so "no circuit" and "closed circuit" are the same healthy state. The reset clears state, failure count and any quota pin together; if the provider is still broken the circuit reopens after circuit_breaker_threshold failures.

?model=<resolved upstream model id> scopes the reset to that one circuit and leaves the provider's other circuits, and the charges they have legitimately accrued, alone; the response then carries "model". Without it every circuit of the provider is cleared, as before. Either way the breaker logs one circuit-breaker: manual reset line per circuit cleared, with cause=manual reset, previous_state and the model, so a fleet-wide clean-up reads in the app log as N circuits with N causes.

This endpoint backs the circular-arrow button ("Reset circuit breaker") beside each open or half-open member on the Failover page.

Response (400): invalid provider UUID. Response (503): the circuit breaker is not available.

Clears every circuit behind one failover group's entries on this member, in one call instead of one per provider. Only the group's (provider, resolved model) pairs are touched: a provider's circuits for models outside the group keep their state.

Response:

{
  "group_id": "uuid",
  "display_model": "glm53",
  "entries": 3,
  "cleared": 2,
  "recovered": 1
}

Like the other resets it is outside the managed-write guard: a circuit is local runtime health, not synced config. Front Desk fans this out to every member (POST /api/fleet/circuit-breaker/reset, see High Availability).

Response (404): unknown group. Response (503): the circuit breaker is not available.

Clears every tracked circuit at once, for recovering from a fleet-wide upstream incident without resetting providers one by one.

Response:

{
  "cleared": 7,
  "recovered": 2
}

This endpoint is deliberately API only: there is no UI control for it, by decision rather than omission. Per-provider reset is the right operation almost every time and it names the provider being recovered, whereas a fleet-wide button placed beside it is too easy to reach for when one circuit is the problem, and it throws away the breaker's evidence about every other provider in the process. Bulk reset is left to scripts and runbooks, where its blast radius is written down.

Response (503): the circuit breaker is not available.

Response: 204 No Content

Response:

[
  {
    "id": "uuid",
    "name": "Production Key",
    "key_preview": "sk-...c5d6",
    "tokens_used": 123456,
    "last_used_at": "2024-01-01T00:00:00Z",
    "created_at": "2024-01-01T00:00:00Z",
    "rate_limit_rps": null,
    "rate_limit_burst": null,
    "rate_limit_tpm": null,
    "allowed_providers": null,
    "strip_reasoning": false,
    "owner_user_id": "uuid",
    "owner_username": "maya",
    "budget_usd": null,
    "budget_period": null
  }
]

owner_user_id is the account that owns the key (null for an unowned key) and owner_username its name, omitted when there is none. A non-admin caller sees and edits only its own keys, and a key it creates is always its own: owner_user_id in the body is honoured for admins only.

Request Body:

{
  "name": "Production Key",
  "rate_limit_rps": 10.0,
  "rate_limit_burst": 20,
  "rate_limit_tpm": 50000,
  "allowed_providers": ["provider-uuid-1", "provider-uuid-2"],
  "strip_reasoning": false
}

Response: 201 Created

{
  "id": "uuid",
  "name": "Production Key",
  "key": "sk-a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6",
  "key_preview": "sk-...c5d6",
  "tokens_used": 0,
  "last_used_at": null,
  "created_at": "2024-01-01T00:00:00Z",
  "rate_limit_rps": 10.0,
  "rate_limit_burst": 20,
  "rate_limit_tpm": 50000,
  "budget_usd": 25,
  "budget_period": "month",
  "budget_spent_usd": 0
}

budget_spent_usd is the key's priced spend in the current period on the answering member; it is absent when the key has no budget, and also while the period's spend could not be summed yet (the proxy then refuses the key's requests with 503 until the store answers). User accounts (/api/users) take and return the same budget_usd + budget_period pair.

⚠️ Important: The full key is shown only once at creation time and cannot be retrieved later.

Request Body: Same as POST, name is required, other fields optional

Response: 204 No Content

Caching: responses are cached in-process, keyed by the raw query string. Each response carries an X-Cache: HIT or X-Cache: MISS header.

Query Parameters:

Response:

{
  "entries": [
    {
      "id": "uuid",
      "provider_id": "uuid",
      "provider_name": "OpenAI",
      "model_id": "gpt-4o",
      "request_hash": "...",
      "status_code": 200,
      "latency_ms": 234.5,
      "duration_ms": 456.7,
      "ttft_ms": 123.4,
      "response_header_ms": 98.7,
      "proxy_overhead_ms": 1.2,
      "parse_ms": 0.5,
      "failover_lookup_ms": 0.1,
      "model_lookup_ms": 0.3,
      "provider_lookup_ms": 0.2,
      "key_decrypt_ms": 0.8,
      "dial_ms": 0.1,
      "settings_read_ms": 0.1,
      "cache_hits": {"failover": true, "model": true, "provider": true, "key": true, "settings": true},
      "tokens_per_second": 45.6,
      "tokens_prompt": 100,
      "tokens_completion": 200,
      "tokens_completion_reasoning": 0,
      "tokens_prompt_cache_hit": 0,
      "tokens_prompt_cache_miss": 0,
      "search_units": 0,
      "cost_usd": 0.0039,
      "streaming": true,
      "virtual_key_name": "Production Key",
      "virtual_key_deleted": false,
      "virtual_key_id": "uuid",
      "client_ip": "203.0.113.7",
      "error_message": "",
      "error_kind": "",
      "failover_attempt": 1,
      "state": "completed",
      "endpoint_type": "chat",
      "created_at": "2024-01-01T00:00:00Z",
      "resolved_model_id": "gpt-4o-2024-08-06",
      "attempts": [
        {"attempt": 0, "provider_id": "uuid", "provider": "Neuralwatt", "model": "glm-5.3", "status": 429, "error_kind": "provider_saturated", "detail": "concurrent_budget_exceeded", "phrase": "concurrent_budget_exceeded", "duration_ms": 412, "breaker": "noop"},
        {"attempt": 1, "provider_id": "uuid", "provider": "OpenAI", "model": "gpt-4o", "status": 200, "duration_ms": 8299, "ttft_ms": 123.4, "breaker": "success"}
      ]
    }
  ],
  "total": 1000,
  "page": 1,
  "per_page": 20
}

search_units is what a rerank provider billed (0 on every other row), client_ip the resolved client address (empty on older rows and address-less paths), resolved_model_id the upstream model id that actually served the request (for a hotel/ request model_id holds the group name). cost_usd is null when the request could not be priced.

attempts is the per-attempt trail: one element per failover attempt, in order, hedged probes (a hedged attempt abandoned in flight appears with no status and the exit as its error_kind: hedge_superseded when another candidate won, failover_timeout, client_disconnect) and in-flight busy skips included; sorted by attempt, skips first. attempt is the loop's index (the same numbering as failover_attempt); -1 marks a candidate the circuit breaker refused before any request was made (breaker: "skipped"). status is the upstream status the attempt reached (omitted when no response was seen), detail at most 160 characters of the sanitized, credential-masked upstream error (never request content), phrase the rate-limit phrase-table entry a 429 matched, and breaker what the attempt did to the circuit: charge, noop, success, alive, skipped or disabled. The field is omitted for rows without a trail (rows from before it existed, rows an older member wrote, requests that never reached a candidate). The terminal attempt's values also stay in the flat columns, so nothing that reads them changes. See Failover: the per-attempt trail.

Request Body:

{ "older_than": "1h" }

Accepted values: 1h, 1d, 1w, 1m, all

Response: 204 No Content

Ring buffer mode (default):

Response: Array of log entries

[
  {
    "timestamp": "2024-01-01T00:00:00.000000000Z",
    "level": "info",
    "source": "proxy",
    "message": "Request completed successfully"
  }
]

History mode (?history=true):

Response:

{
  "entries": [...],
  "total": 500,
  "page": 1,
  "per_page": 20,
  "level_counts": {
    "info": 450,
    "warning": 30,
    "error": 20
  },
  "source_counts": {
    "proxy": 300,
    "auth": 100,
    "discovery": 100
  }
}

Request Body (optional):

{ "older_than": "1d" }

Accepted values: 1h, 1d, 1w, 1m, all. No body (or no older_than) means all; a body that cannot be read, or another value, is a 400.

Response:

{ "deleted": 1234 }

deleted is the number of database rows removed. Response (503): the app log writer did not drain its queue in time, so the purge was refused rather than undone by the pending flush; retry it.

Admin-only. Both endpoints require an admin session. The trail itself is written by middleware that records every mutating (POST/PUT/PATCH/DELETE) request on the authenticated /api/* surface from any signed-in account, admin or user. Request and response bodies are never stored.

Query Parameters:

Response:

{
  "entries": [
    {
      "id": "uuid",
      "created_at": "2026-01-01T00:00:00Z",
      "actor": "maya",
      "actor_role": "user",
      "method": "PUT",
      "route": "/api/virtual-keys/{id}",
      "path": "/api/virtual-keys/0a1b2c3d-...",
      "entity_id": "0a1b2c3d-...",
      "entity_name": "maya-cli",
      "status_code": 200,
      "remote_addr": "10.0.0.17:41822"
    }
  ],
  "total": 132,
  "has_more": true,
  "next_cursor": "..."
}

entity_name is resolved at read time from the entity's current display name (models, providers, virtual keys, failover groups, users) and is omitted when the entity has been deleted or the route has no mapped entity; the UUID in entity_id then remains the only trace. Names are never stored, so a rename shows the current name.

Request Body:

{ "older_than": "1w" }

Accepted values: 1h, 1d, 1w, 1m, all

Response: 204 No Content

The purge is itself a mutating request and is recorded by the audit middleware, so a wiped trail always shows who wiped it.

Response:

[
  {
    "filename": "backup_20240101_120000_123456.dump",
    "size_bytes": 10485760,
    "created_at": "2024-01-01T12:00:00Z",
    "origin": "manual",
    "signed": true
  }
]

origin is manual (created from the API or dashboard), scheduled (the rotation scheduler) or frontdesk (requested with ?origin=frontdesk); only scheduled backups are rotated. signed says a signature sidecar exists, not that it verifies: the check runs on download.

Creates a PostgreSQL backup using pg_dump (custom format, zstd-compressed: level 12 here, so the request stays short, and level 19 for scheduled backups; readable by pg_restore 16 or later built with zstd, which the postgres:16-alpine image is).

Response: 201 Created

{
  "filename": "backup_20240101_120000_123456.dump",
  "size_bytes": 10485760,
  "created_at": "2024-01-01T12:00:00Z",
  "origin": "manual",
  "signed": true
}

Error Responses:

• 409 Conflict - Backup already in progress
• 412 Precondition Failed - pg_dump not found (install postgresql-client)
• 500 Internal Server Error - Backup failed

Downloads the backup file.

Response: File download with Content-Disposition: attachment

Error Responses:

• 403 Forbidden - Read-only demo mode
• 404 Not Found - Backup does not exist
• 409 Conflict - A signed backup failed its integrity check (contents changed after signing); nothing is served
• 500 Internal Server Error - The file or its signature could not be read

Replaces the database with an uploaded dump. multipart/form-data, at most 100 MB:

Before restoring, the dump is checked with pg_restore --list: it must be a Model Hotel backup (carry schema_migrations), hold no dangerous objects, and not come from a newer version. The restore runs in a single transaction; on success the process exits and the container restart re-runs any missing migrations.

Response:

{ "migration_count": 91, "known_count": 92 }

migration_count is the migrations recorded in the dump, known_count those this build knows.

Error Responses:

• 400 Bad Request - Unreadable form, missing dump, signature mismatch, invalid dump, dangerous objects, no schema_migrations, or a dump from a newer version
• 401 Unauthorized - Missing or wrong admin_token
• 409 Conflict - Another backup or restore is in progress
• 412 Precondition Failed - pg_restore not found
• 500 Internal Server Error - Saving, verifying or restoring the dump failed

Returns the backup's HMAC signature sidecar, the value the restore endpoint takes in its signature form field. The download serves the dump alone, so this is how an operator without shell access to the backup directory carries the signature to a restore. Not verified here: the restore checks it against the uploaded bytes.

Response:

{ "signature": "3f1a…64 hex characters…" }

Error Responses:

• 400 Bad Request - Invalid filename
• 404 Not Found - Backup does not exist, or has no signature (backups predating signing, dumps copied in from another instance, or a backup whose signing failed at creation; see the backup.unsigned event)
• 500 Internal Server Error - The sidecar exists but cannot be read, or is not a valid signature

Response: 204 No Content

Error Responses:

• 400 Bad Request - Invalid filename
• 404 Not Found - Backup does not exist
• 409 Conflict - Another backup operation (a dump, restore, prune or delete) is in progress
• 500 Internal Server Error - The file could not be removed

Preview which backups would be pruned under the son/father/grandfather rotation scheme (three retention tiers: daily "son" backups, weekly "father", monthly "grandfather", each with its own count). Non-destructive (dry run).

Response:

{
  "son": [
    {"filename": "backup_20260608_120000_0000.dump", "size_bytes": 1048576, "created_at": "2026-06-08T12:00:00Z"}
  ],
  "father": [],
  "grandfather": [],
  "prune": [
    {"filename": "backup_20260301_120000_0000.dump", "size_bytes": 2097152, "created_at": "2026-03-01T12:00:00Z"}
  ]
}

Execute the son/father/grandfather rotation, deleting backups that fall outside the retention tiers. Returns the same classification as prune-preview.

Response: Same structure as prune-preview, reflecting the state after pruning.

Every value is a string, whatever its logical type. The map also carries a few read-only keys that PUT rejects: app_version, app_commit, log_export_json, log_export_metrics, log_export_otel and discovery_claim_window_days. Secret values (alert_apprise_targets, oidc_client_secret, github_client_secret) come back as a fixed placeholder, never as stored ciphertext or plaintext.

Response:

{
  "rate_limit_enabled": "true",
  "rate_limit_ip_rps": "30",
  "rate_limit_ip_burst": "60",
  "discovery_interval": "6h",
  "discovery_on_startup": "true",
  "circuit_breaker_enabled": "true",
  "app_version": "1.2.3"
}

Request Body:

{
  "rate_limit_ip_rps": "50",
  "discovery_interval": "12h"
}

A key outside the allowlist below is a 400 (unknown setting: <key>), as is a value longer than 500 characters or outside its range. A managed fleet member additionally answers 403 for any synced key: the primary owns those and replaces them on the next sync.

Allowed Settings:

Response: 200 OK with full settings map

Query Parameters:

Response:

{
  "total_requests_last_24h": 12345,
  "total_requests_last_7d": 54321,
  "by_model": {
    "OpenAI/gpt-4o": 5000,
    "Anthropic/claude-3-opus": 2000,
    "hotel/fast-chat": 1000
  },
  "by_provider": {
    "OpenAI": 8000,
    "Anthropic": 4000
  },
  "by_virtual_key": {
    "Production Key": 10000,
    "Development": 2000
  },
  "avg_latency_ms": 234.5,
  "error_rate": 0.02,
  "avg_overhead_ms": 1.2,
  "total_tokens_prompt": 500000,
  "total_tokens_completion": 750000,
  "total_tokens_cache_hit": 120000,
  "total_cost_usd": 12.34,
  "requests_unpriced": 3,
  "avg_tokens_per_request": 101.2,
  "rate_limit_hits": 15,
  "avg_ttft_ms": 123.4,
  "requests_last_1h": 500,
  "by_provider_latency": [
    {"provider_name": "OpenAI", "total_ms": 234.5, "overhead_ms": 1.2, "provider_ms": 233.3, "request_count": 8000}
  ]
}

total_tokens_cache_hit is the prompt tokens served from the provider's cache. by_provider_latency is filled only with include_latency=true (the six providers with the highest average total latency) and is null otherwise.

total_cost_usd sums request_logs.cost_usd over the period. Requests the proxy could not price add nothing, and requests_unpriced counts the served (2xx) ones among them, so the total is a floor by that many requests. The by_* maps carry whichever metric was asked for, so under metric=cost their values are dollars.

Query Parameters: Same as /api/stats

Response:

{
  "points": [
    {
      "bucket": "2024-01-01T00:00:00Z",
      "count": 100,
      "tokens": 50000,
      "tokens_cache_hit": 12000,
      "tokens_cache_miss": 8000,
      "cost_usd": 0.42,
      "errors": 2,
      "latency_ms": 234.5,
      "overhead_ms": 1.2,
      "provider_latency_ms": 230.0,
      "rate_limit_hits": 1,
      "avg_ttft_ms": 123.4
    }
  ]
}

Returns hourly buckets for 1h and 24h periods, daily buckets for 7d. Empty buckets are filled with zeros.

Query Parameters:

Response:

{
  "items": [
    {
      "name": "OpenAI",
      "count": 8000,
      "tokens": 0,
      "cost_usd": 0,
      "share": 66.7
    },
    {
      "name": "Anthropic",
      "count": 4000,
      "tokens": 0,
      "cost_usd": 0,
      "share": 33.3
    }
  ]
}

Query Parameters:

Response:

{
  "app": {
    "heap_alloc_mb": 45.6,
    "sys_memory_mb": 128.0,
    "goroutines": 50,
    "gc_cycles": 123,
    "memory_current_bytes": 134217728,
    "memory_limit_bytes": 1073741824,
    "in_container": true,
    "uptime_seconds": 86400,
    "cpu_percent": 2.5,
    "requests_today": 12345,
    "net_rx_bytes_sec": 1024.0,
    "net_tx_bytes_sec": 2048.0,
    "disk_read_bytes_sec": 512.0,
    "disk_write_bytes_sec": 256.0,
    "procs": 4
  },
  "db": {
    "size_mb": 256.0,
    "connections": 10,
    "cache_hit_ratio": 99.5,
    "cache_window_blocks": 48213,
    "tx_per_sec": 15.3,
    "dead_tuples": 1000,
    "lock_waits": 0
  },
  "docker": {
    "available": true,
    "cpu_percent": 5.0,
    "memory_usage_bytes": 536870912,
    "memory_limit_bytes": 2147483648,
    "net_rx_bytes_sec": 1024.0,
    "net_tx_bytes_sec": 2048.0,
    "disk_read_bytes_sec": 512.0,
    "disk_write_bytes_sec": 256.0,
    "procs": 12,
    "container_count": 3
  },
  "fleet": {
    "state": "member",
    "is_primary": false,
    "primary_name": "mh1",
    "frontdesk_id": "...",
    "managed_seen_at": "2024-01-01T12:00:00Z",
    "config_synced_at": "2024-01-01T11:55:00Z"
  },
  "instance_id": "uuid"
}

docker aggregates every container of the Compose project, as rates per second; available is false when the Docker API cannot be read. db.cache_window_blocks is how many block accesses the window behind cache_hit_ratio held, omitted when the ratio rests on no fresh activity. fleet is this instance's HA membership (state is primary, member or warning), omitted for a standalone instance Front Desk has never contacted. instance_id is the instance's stable identity, which Front Desk uses to recognize one instance reached under two URLs.

Long-lived SSE stream for real-time dashboard updates. Any signed-in identity may connect (admin token or session, as on the rest of /api). Admins receive every event. A non-admin receives only request.* lifecycle events (not request.discovery.*), and only when it holds the logs grant and the event belongs to one of its own virtual keys; every other event type stays admin only. The credentials are re-checked on every heartbeat, so a changed grant applies mid-stream and a revoked session or disabled account closes the stream after two failed checks.

Response Headers:

Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
X-Accel-Buffering: no

Event Format:

: connected

data: {"type":"discovery.complete","severity":"success","message":"Discovery complete: 15 models across 2 providers","metadata":{"source":"Startup","models_pruned":0}}

: heartbeat

Event Types:

Front Desk publishes its own event set (member.*, fleet.*, config.*, health.*, settings.changed, and more) on its own stream; those never appear here.

Heartbeat comments (: heartbeat) are sent every 30 seconds.

Circuit breaker event metadata:

circuit_breaker.open and circuit_breaker.closed carry the same metadata block:

next_retry_at is the retry deadline, not the provider's quota reset time: it is the open moment plus the cooldown actually in force, and the longer of a pin and a backoff governs. A pin can also be lifted early, without any event, when a quota refresh shows the provider back in credit. See Quota-pinned cooldowns.

circuit_breaker.unstable is not a state transition and carries a smaller block: provider_id, provider, model, model_id, opens (always 3) and window (the string 24h). It fires at most once per model per window and never disables anything: a model the provider has stopped serving is retired by the model-gone path instead.

Quota schema drift metadata:

quota.schema_drift is alert-only: it never affects routing, failover, or the circuit breaker. It exists because the failure it guards against is silent: a normalizer written against the old shape keeps answering, wrongly, and nothing else would ever say so.

Available only when WEBAUTHN_RP_ID is configured; without it none of these routes is mounted and each answers 404, /api/webauthn/available included (see Security for the full authentication flow).

Time-based one-time passwords (RFC 6238) as an admin-login second factor, independent of passkeys. Opt-in at runtime from Settings; no environment variable required (see Security for the full authentication flow and enforcement model).

When TOTP is enabled, the raw admin token alone no longer authorizes /api/*: it is a first factor that must be exchanged via /api/totp/login for a session token.

These endpoints proxy through the same completion handler as /v1/chat/completions, but authenticate a dashboard session instead of a virtual key: any signed-in identity holding the chat grant may use them, not only an admin. Streaming and non-streaming both work as on /v1.

Four limiters apply in order: the per-IP limiter every /api route carries, the per-key RPS limiter (bucketed on the route name, chat/arena/completions, since there is no virtual key here, alongside the caller's per-user request cap), the caller's own per-user tokens-per-minute cap, and the caller's per-user dollar budget. A user with a TPM cap or a budget is metered here exactly as on /v1.

Request/Response: Same format as /v1/chat/completions

Routes that exist but have no section of their own. Everything under /api carries the same admin/session authentication unless the row says otherwise.

Returns 200 with the body OK while the database answers, and 503 with the body DEGRADED when it does not, so a load balancer stops routing to an instance whose Postgres is down. The probe result is cached briefly, so a burst of health checks costs one database round-trip. Content type is text/plain; no authentication is required.

{
  "error": {
    "message": "error description",
    "type": "invalid_request_error",
    "code": 400
  }
}

code repeats the HTTP status as a number. type is derived from that status and is one of exactly seven values; messages are lowercase.

Invalid Virtual Key:

{
  "error": {
    "message": "invalid virtual key",
    "type": "authentication_error",
    "code": 401
  }
}

Rate Limit Exceeded:

{
  "error": {
    "message": "rate limit exceeded",
    "type": "rate_limit_error",
    "code": 429
  }
}

Model Not Found:

{
  "error": {
    "message": "model not found: hotel/gpt-5",
    "type": "not_found_error",
    "code": 404
  }
}

No Provider Left:

{
  "error": {
    "message": "virtual key does not have access to any provider for this model",
    "type": "permission_error",
    "code": 403
  }
}

Virtual keys can have custom rate limits configured. If not set, global defaults apply.

Headers (when rate limited):

Retry-After: 60
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 0
X-RateLimit-Burst: 20

Retry-After is only set when a wait was computed. X-RateLimit-Scope names the stage that refused: ip for the per-IP limiter and ratelimit-tpm for a tokens-per-minute cap (per key or per user); a per-key or per-user request-rate refusal carries none. A client can so tell an address-wide refusal from a key-wide one.

Applied to /api/*, /api/chat/* and /v1/* (not to /health, /metrics or the dashboard's static files), configurable via settings:

Trusted proxies (via TRUSTED_PROXIES env var) use the X-Forwarded-For header for IP identification.

CORS is configurable via the CORS_ORIGINS environment variable (comma-separated list).

Response Headers (when origin matches):

Access-Control-Allow-Origin: <origin>
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, X-CSRF-Token, x-api-key
Access-Control-Allow-Credentials: true
Access-Control-Max-Age: 86400

A preflight that names its headers in Access-Control-Request-Headers gets that list echoed back as Access-Control-Allow-Headers instead, so browser SDKs can send their own (anthropic-version, x-stainless-*). Every request carrying an Origin gets Vary: Origin, Access-Control-Request-Headers.

Preflight: an OPTIONS request carrying an Origin header returns 204 No Content; the CORS headers above are added only when the origin is allowed.

All responses include:

X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Referrer-Policy: strict-origin-when-cross-origin
Content-Security-Policy: default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; connect-src 'self'; frame-ancestors 'none'; base-uri 'self'; form-action 'self'

With ALLOW_EMBED=true the two framing controls are dropped, X-Frame-Options entirely and frame-ancestors 'none' out of the CSP, so any origin can put the dashboard in an iframe (workspace browsers, Home Assistant). Nothing else about the policy changes.

HSTS (Strict-Transport-Security: max-age=63072000; includeSubDomains; preload) is set only when this process terminated the TLS connection itself. Behind a reverse proxy that terminates TLS and forwards plain HTTP, the gateway does not set it, because a cached HSTS pin would point browsers at an HTTPS listener that does not exist; set the header on the proxy instead.