Virtual Keys - hugalafutro/model-hotel GitHub Wiki
Virtual keys are client-facing API keys that provide authenticated access to the /v1/* proxy endpoints. They enable per-client rate limiting, token usage tracking, and audit logging without exposing provider API keys.

Virtual Keys page: name, key preview, RPS, burst, TPM, budget, created, tokens used, and last used, with a name filter above the table. Clicking a row opens the key detail modal, which is where editing and deletion live.

Key creation dialog - the plaintext key is shown only once

Key detail modal: the key name heads it with its consumed tokens beside, then owner, id, created, last used, key preview, strip reasoning, budget, burst, TPM, RPS, and the providers the key can reach. The list is the key's provider scope with blocked providers marked: an unrestricted key whose owner has no provider cap reads "All providers", a capped owner's key lists the cap, and a scoped key lists the providers it names, any the owner's cap denies struck through with a note saying why.

The same modal after Edit: name, owner, rate limits, budget and period, provider access (click a provider to toggle it; a provider outside the owner's cap is struck through and cannot be toggled), and strip reasoning. Save Changes stays disabled until something differs.
- Client sends the virtual key as
Authorization: Bearer <virtual-key>(OpenAI-style clients) or asx-api-key: <virtual-key>(Anthropic-SDK clients). Bearer wins when both headers are present - Proxy hashes the key with SHA-256 and looks it up in the
virtual_keystable, joining the owning user row in the same query (1 DB query) - If the hash matches and the owner's account is enabled, the proxy puts the key identity, the key's limits, and the owner's aggregate limits into the request context
- Proxy looks up the requested model and provider
- Proxy decrypts the provider's real API key using
MASTER_KEY - Request is forwarded to the provider with the real API key
- Proxy streams the response back, logging token usage against the virtual key
Virtual keys follow the format: sk-<32 hex characters>
-
Prefix:
sk-(secret key identifier) - Payload: 32 hexadecimal characters (16 bytes of cryptographic randomness)
- Total length: 35 characters
-
Example:
sk-a1b2c3d4e5f6789012345678abcdef01
Key generation reads 16 bytes from Go's crypto/rand, hex-encodes them, and prefixes sk-.
-
Source:
crypto/rand- cryptographically secure pseudo-random number generator - Entropy: 128 bits (16 bytes × 8 bits)
- Encoding: Lowercase hexadecimal
- Uniqueness: Probability of collision is negligible (~1 in 2¹²⁸)
The key_preview field stores a human-readable identifier for the key:
-
Format: First 3 characters (the
sk-prefix) +...+ last 4 characters -
Example:
sk-a1b2c3d4e5f6789012345678abcdef01→sk-...ef01 - Purpose: UI identification without exposing the full key
-
Storage: Stored in plaintext alongside the hash in the
virtual_keystable
The four-character tail matches the provider card's masking. Because only the hash is stored, a key issued before the tail widened keeps its shorter two-character preview; the key itself still works.
Virtual keys are never stored in plaintext: the key is hashed with SHA-256 and the 64-character lowercase hex digest goes into key_hash. Hashing is deterministic, so an incoming key can be hashed and compared, and one-way, so a stored digest cannot be turned back into a key.
-
Storage: Only 64-character hex hash stored in
key_hashcolumn - Lookup: Incoming requests hash the provided token and compare hashes
- Deletion: Removing a key from the database immediately invalidates it
- Rotation: Create a new key, update clients, delete the old key
- Recovery: Lost keys cannot be recovered - generate a new one
ProxyKeyMiddleware runs ahead of every /v1 handler:
-
Extract: read the key from
Authorization: Beareror, failing that,x-api-key. Neither present is a401withmissing authorization header. -
Hash and look up: SHA-256 the key and fetch the row by hash, joining the owning user. The lookup carries its own 10-second timeout so a wedged database refuses requests (
500 internal error) instead of parking them; a client that disconnects mid-lookup gets499instead. -
Refuse cleanly: an unknown hash is
401 invalid virtual key; a key whose owner account is disabled is401 virtual key disabled: owner account is disabled. Every401from this middleware setsConnection: close, so a client trickling a body with no valid key hears the refusal immediately rather than after the body deadline. -
Populate context: key name, id, hash, the per-key RPS/burst/TPM overrides,
allowed_providers,strip_reasoning, the key's dollar budget when it has one, and, for an owned key, the owner id plus that account's aggregate limits, provider cap and dollar budget. -
Touch: update
last_used_atin a fire-and-forget goroutine with a 5-second timeout, so the proxy path never waits on it.
Rejections are logged at warn level with the remote address and, where known, the key name. Request and response content is never logged.
After successful authentication, the following values are available in request context:
| Key | Type | Purpose |
|---|---|---|
virtual_key_name |
string |
Human-readable key name (e.g., "production-app") |
virtual_key_id |
string |
Database primary key (UUID string) |
virtual_key_hash |
string |
SHA-256 hash (64 hex chars) |
virtual_key_rate_limit_rps |
*float64 |
Per-key RPS override (nil = use global) |
virtual_key_rate_limit_burst |
*int |
Per-key burst override (nil = use global) |
virtual_key_rate_limit_tpm |
*int |
Per-key tokens-per-minute cap (nil = no cap / global default) |
virtual_key_allowed_providers |
*[]string |
Provider access restriction (nil = all providers) |
virtual_key_strip_reasoning |
bool |
Whether to strip reasoning fields from streaming output |
key_budget |
*budget.Subject |
The key's dollar budget (absent when the key has none) |
virtual_key_owner_id |
string |
Owning dashboard user's UUID (absent for an unowned key) |
user_rate_limit_rps |
*float64 |
Owner's aggregate RPS cap (nil = no cap) |
user_rate_limit_burst |
*int |
Owner's aggregate burst (nil = no cap) |
user_rate_limit_tpm |
*int |
Owner's aggregate tokens-per-minute cap (nil = no cap) |
user_allowed_providers |
*[]string |
Owner's account provider cap (nil = no cap) |
user_budget |
*budget.Subject |
Owner's account dollar budget (absent when the account has none) |
The six owner values are set only when the key has an owner, and user_budget only when that account has a budget.
CREATE TABLE IF NOT EXISTS virtual_keys (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
key_hash TEXT NOT NULL UNIQUE,
key_preview TEXT NOT NULL DEFAULT '',
tokens_used BIGINT NOT NULL DEFAULT 0,
last_used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT now(),
rate_limit_rps DOUBLE PRECISION DEFAULT NULL,
rate_limit_burst INTEGER DEFAULT NULL,
rate_limit_tpm INTEGER DEFAULT NULL,
allowed_providers TEXT[] DEFAULT NULL,
strip_reasoning BOOLEAN NOT NULL DEFAULT false,
owner_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
budget_usd DOUBLE PRECISION,
budget_period TEXT,
CONSTRAINT virtual_keys_rate_limit_bounds CHECK (
(rate_limit_rps IS NULL OR rate_limit_rps >= 0) AND
(rate_limit_burst IS NULL OR rate_limit_burst >= 1) AND
(rate_limit_tpm IS NULL OR rate_limit_tpm >= 1)
),
CONSTRAINT virtual_keys_budget_check CHECK (
(budget_usd IS NULL) = (budget_period IS NULL)
AND (budget_usd IS NULL OR (budget_usd > 0 AND budget_usd <= 10000000))
AND (budget_period IS NULL OR budget_period IN ('day', 'week', 'month'))
)
);
CREATE INDEX IF NOT EXISTS idx_virtual_keys_key_hash ON virtual_keys(key_hash);
CREATE INDEX IF NOT EXISTS idx_virtual_keys_owner
ON virtual_keys (owner_user_id) WHERE owner_user_id IS NOT NULL;| Column | Type | Constraints | Description |
|---|---|---|---|
id |
UUID |
PRIMARY KEY, DEFAULT gen_random_uuid()
|
Unique identifier |
name |
TEXT |
NOT NULL | Human-readable name (1-100 chars, printable) |
key_hash |
TEXT |
NOT NULL, UNIQUE | SHA-256 hash (64 hex characters) |
key_preview |
TEXT |
NOT NULL | First 3 + last 4 chars (e.g., sk-...ef01) |
tokens_used |
BIGINT |
NOT NULL, DEFAULT 0 | Cumulative token count (prompt + completion) |
last_used_at |
TIMESTAMPTZ |
NULLABLE | Last authentication timestamp |
created_at |
TIMESTAMPTZ |
DEFAULT now()
|
Creation timestamp |
rate_limit_rps |
DOUBLE PRECISION |
NULLABLE | Per-key RPS override (null = global default) |
rate_limit_burst |
INTEGER |
NULLABLE | Per-key burst override (null = global default) |
rate_limit_tpm |
INTEGER |
NULLABLE | Per-key tokens-per-minute cap (null = no cap / global default) |
allowed_providers |
TEXT[] |
NULLABLE | Provider IDs this key may use (null = all providers accessible) |
strip_reasoning |
BOOLEAN |
NOT NULL, DEFAULT false | Strip reasoning/reasoning_content fields from streaming output for this key |
owner_user_id |
UUID |
NULLABLE, FK users(id) ON DELETE SET NULL |
Owning dashboard user (null = unowned) |
budget_usd |
DOUBLE PRECISION |
NULLABLE, above 0 and at most 10,000,000 | Dollar spending cap per period (null = no budget) |
budget_period |
TEXT |
NULLABLE, day, week or month
|
Calendar period (UTC) the budget resets on; set exactly when budget_usd is |
Deleting a user orphans their keys (owner_user_id becomes null) instead of deleting them, so an account cleanup cannot silently kill production traffic.
| Migration | File | Changes |
|---|---|---|
004 |
internal/db/migrations/004_virtual_keys.sql |
Initial table creation |
005 |
internal/db/migrations/005_virtual_key_preview.sql |
Added key_preview column |
012 |
internal/db/migrations/012_add_virtual_key_id_to_request_logs.sql |
Added virtual_key_id to request_logs
|
029 |
internal/db/migrations/029_virtual_key_rate_limits.sql |
Added per-key rate limit columns |
037 |
internal/db/migrations/037_virtual_key_allowed_providers.sql |
Added allowed_providers (per-key provider access restriction) |
038 |
internal/db/migrations/038_virtual_key_strip_reasoning.sql |
Added strip_reasoning flag |
046 |
internal/db/migrations/046_virtual_key_rate_limit_tpm.sql |
Added rate_limit_tpm (per-key tokens-per-minute cap) |
051 |
internal/db/migrations/051_user_limits_vk_ownership.sql |
Added owner_user_id plus its index, and the matching per-user limit columns |
064 |
internal/db/migrations/064_rate_limit_bounds.sql |
Nulled out-of-bounds limits and added the CHECK that keeps them in range |
074 |
internal/db/migrations/074_request_log_vk_index.sql |
Indexed request_logs.virtual_key_id for the Logs page's virtual-key filter |
081 |
internal/db/migrations/081_rate_limit_ceilings.sql |
Clamped per-key (and per-user) limits above the API ceilings (RPS and burst 10000, TPM 100000000); no CHECK, since config sync accepts above-ceiling values |
087 |
internal/db/migrations/087_budgets.sql |
Added budget_usd + budget_period and the virtual_keys_budget_check pairing constraint (the same pair on users) |
Virtual key endpoints are dashboard API routes, not proxy routes. A caller authenticates either with the browser session cookie or with Authorization: Bearer $ADMIN_TOKEN, and needs the virtual_keys grant, which covers reads and writes alike. Admins see and edit every key; a non-admin grant holder sees and edits only the keys they own, and a key belonging to someone else answers 404 rather than 403 so the listing and the detail route tell the same story.
Two things narrow that further:
- With TOTP two-factor enabled, a bare admin token is refused: exchange it for a session token via
POST /api/totp/loginfirst. - On a managed fleet member the three write routes answer
403, because the primary owns virtual keys and replaces them on the next sync. Create, update, and delete them on the primary.
Endpoint: POST /api/virtual-keys
Request Body:
{
"name": "production-app",
"rate_limit_rps": 5.0,
"rate_limit_burst": 10,
"rate_limit_tpm": 50000,
"allowed_providers": ["provider-uuid-1"],
"strip_reasoning": false
}| Field | Type | Required | Constraints |
|---|---|---|---|
name |
string |
Yes | 1-100 chars, printable Unicode, not reserved |
rate_limit_rps |
number |
No | 0 to 10000 (null = use global default) |
rate_limit_burst |
integer |
No | 1 to 10000 (null = use global default) |
rate_limit_tpm |
integer |
No | Tokens-per-minute cap, 1 to 100000000 (null = no cap / global default) |
allowed_providers |
array of UUID strings |
No | Restrict the key to the listed provider IDs (null/omitted = all providers; empty array rejected) |
strip_reasoning |
boolean |
No | Strip reasoning/reasoning_content from streaming output (default false) |
owner_user_id |
UUID string |
No | Dashboard user to own the key. Admin callers only: a non-admin's key is always created as their own, whatever the body says. Null or empty = unowned |
budget_usd |
number |
No | Dollar spending cap per period, above 0 and at most 10,000,000. Sent together with budget_period or not at all (null = no budget) |
budget_period |
string |
No |
day, week or month (UTC calendar periods); required with budget_usd
|
Reserved Names (cannot be used, compared case-insensitively):
chatarenacompletionsadmin
These are reserved because they conflict with built-in URL paths.
Response (201 Created):
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "production-app",
"key": "sk-a1b2c3d4e5f6789012345678abcdef01",
"key_preview": "sk-...ef01",
"tokens_used": 0,
"last_used_at": null,
"created_at": "2025-01-15T10:30:00Z",
"rate_limit_rps": 5.0,
"rate_limit_burst": 10,
"rate_limit_tpm": 50000,
"allowed_providers": ["provider-uuid-1"],
"strip_reasoning": false,
"owner_user_id": "770e8400-e29b-41d4-a716-446655440002",
"owner_username": "alice"
}
⚠️ Critical: Thekeyfield is returned only once on creation. It is never returned in subsequent API calls. Store it securely immediately.
Endpoint: GET /api/virtual-keys
Response (200 OK):
[
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "production-app",
"key_preview": "sk-...ef01",
"tokens_used": 125000,
"last_used_at": "2025-01-15T14:22:00Z",
"created_at": "2025-01-15T10:30:00Z",
"rate_limit_rps": 5.0,
"rate_limit_burst": 10,
"rate_limit_tpm": 50000
},
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"name": "dev-testing",
"key_preview": "sk-...9fab",
"tokens_used": 4500,
"last_used_at": null,
"created_at": "2025-01-14T08:15:00Z",
"rate_limit_rps": null,
"rate_limit_burst": null,
"rate_limit_tpm": null
}
]Notes on the shape: key is omitted entirely outside the create response. rate_limit_* are null when the key falls back to the global defaults. The examples above are trimmed; every response also carries allowed_providers, strip_reasoning, owner_user_id, and, for an owned key, owner_username. A non-admin caller's list contains only their own keys.
Endpoint: GET /api/virtual-keys/{id}
Response (200 OK):
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "production-app",
"key_preview": "sk-...ef01",
"tokens_used": 125000,
"last_used_at": "2025-01-15T14:22:00Z",
"created_at": "2025-01-15T10:30:00Z",
"rate_limit_rps": 5.0,
"rate_limit_burst": 10,
"rate_limit_tpm": 50000
}Endpoint: PUT /api/virtual-keys/{id}
Request Body:
{
"name": "production-app-v2",
"rate_limit_rps": 10.0,
"rate_limit_burst": 20,
"rate_limit_tpm": 30000
}Response (200 OK):
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "production-app-v2",
"key_preview": "sk-...ef01",
"tokens_used": 125000,
"last_used_at": "2025-01-15T14:22:00Z",
"created_at": "2025-01-15T10:30:00Z",
"rate_limit_rps": 10.0,
"rate_limit_burst": 20,
"rate_limit_tpm": 30000
}PUT is a full rewrite of the rate-limit fields: rate_limit_rps, rate_limit_burst, and rate_limit_tpm are written on every update, so omitting one clears it back to the global default rather than preserving the current value. Send the values you want kept.
Three fields behave the other way and are preserved when omitted: allowed_providers, strip_reasoning, and owner_user_id keep their stored value, so a script that only renames a key cannot accidentally drop its restrictions. For owner_user_id an explicit null from an admin unassigns the owner; a non-admin's update always keeps the key on their own account.
Endpoint: DELETE /api/virtual-keys/{id}
Response: 204 No Content (empty body)
⚠️ Permanent Deletion: Keys are permanently deleted, not disabled. There is no per-key "revoke" or "disable" endpoint. Once deleted:
- The key hash is removed from the database immediately
- Any subsequent request using that key receives
401 Unauthorized- Historical logs retain the
virtual_key_namefor auditing- Recovery is impossible - create a new key if needed
An owned key has a reversible alternative: disabling the owner's account rejects every key that account owns with 401, and re-enabling it brings them all back. See Ownership and per-user limits.
Each virtual key has an independent token bucket rate limiter.
| Setting | Default | Description |
|---|---|---|
rate_limit_enabled |
true |
Runtime toggle (DB setting) |
rate_limit_rps |
10 |
Requests per second (global default) |
rate_limit_burst |
20 |
Maximum burst size (global default) |
rate_limit_tpm |
0 |
Tokens-per-minute cap (global default; 0 = no cap; API-only, no Settings-UI control) |
rate_limit_max_wait_ms |
200 |
How long a request may be held before it is rejected (shared with the per-IP limiter) |
rate_limit_ip_enabled |
true |
Runtime toggle for the per-IP limiter |
rate_limit_ip_rps |
30 |
Per-IP requests per second |
rate_limit_ip_burst |
60 |
Per-IP burst size |
The RATE_LIMIT_ENABLED environment variable is the boot-time kill switch (default true). It gates whether the limiters are mounted at all; rate_limit_enabled is the runtime toggle on top of that.
Virtual keys can override global rate limits via rate_limit_rps and rate_limit_burst columns:
-
null: Use global settings fromsettingstable -
0for RPS: Unlimited requests (no rate limiting for this key) -
0for burst: Invalid - rejected on creation/update (must be ≥ 1) - Above the global maximum (rps 10000, burst 10000, tpm 100000000): Invalid - rejected on creation/update through the dashboard and API. A fleet config sync deliberately does not re-check them and honours the primary's larger limit as stored (an older member must not reject a whole envelope over a raised ceiling); the ceilings are already effectively unlimited, so nothing meaningful is relaxed. Rows set above a ceiling before it existed were clamped to it by migration 081
In addition to the request-rate limiter above, each key can cap its tokens
per minute via rate_limit_tpm (a separate token-budget bucket, refilled at
tpm / 60 per second with a full minute's budget available at once):
-
null: No per-key override, so the globalrate_limit_tpmsetting applies (default0= no cap). That global default is API-only; there is no Settings-UI control for it (per-VK is the primary surface). -
≥ 1: Cap the key's combined prompt + completion tokens (reasoning is part of completion) per minute.0is rejected on create/update (usenullfor no cap).
Because a request's token cost is unknown until it finishes, enforcement is admit-on-past-consumption, debit-on-completion: admission only checks whether the budget is already drained, and the actual token total is subtracted afterward. Consequently a key can overshoot by roughly one in-flight request's worth of tokens, and a single response larger than the whole minute budget still completes: it just blocks the next request until the budget refills. This is a consumer-side control: the rejected request never reaches the upstream provider, which is never throttled. Like the RPS limiter, the budget is in-process and not shared across replicas (effective limit is ~N× with N instances behind a load balancer).
A request over its rate is not refused straight away. If the bucket will have
room within rate_limit_max_wait_ms (200 ms by default), the request is held
for that long and then served. Only a wait longer than the ceiling becomes a
429. Short spikes therefore show up as latency rather than errors, and a
client that disconnects mid-wait has its reservation handed back.
When a key exceeds its rate limit:
HTTP/1.1 429 Too Many Requests
Retry-After: 2
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 0
X-RateLimit-Burst: 20
Content-Type: application/json
{
"error": {
"message": "rate limit exceeded",
"type": "rate_limit_error",
"code": 429
}
}
Four messages come out of this family, and the wording says which limit was hit:
| Message | Limit |
|---|---|
rate limit exceeded |
The key's request rate |
user rate limit exceeded |
The owner's aggregate request rate |
token rate limit exceeded |
The key's TPM budget |
user token rate limit exceeded |
The owner's aggregate TPM budget |
Both request-rate stages are checked together, and the reported one is
whichever forced the longer wait. A dollar budget refuses with the same status
and its own wording, key budget exceeded: $25.00 of $25.00 spent this month
(or user budget exceeded), with Retry-After set to the end of the period;
see Dollar Budgets.
A separate per-IP limiter (30 rps, burst 60, same rate_limit_max_wait_ms
backpressure) runs before key authentication and fronts the dashboard routes
as well as /v1. A client with no key, or an invalid one, can therefore be
throttled with a 429 before it ever gets its 401. Unlike the auth failures
behind it, this rejection keeps the connection alive, because a throttled
dashboard client is legitimate and will retry on it.
Its message is also rate limit exceeded, and the X-RateLimit-Scope: ip
header it sets stays on the response even when a later stage is the one that
rejects, so neither tells you which limiter fired. The reliable signal is the
call itself: a request whose key is known-bad returning 429 instead of 401
was stopped by IP, as was any 429 on a dashboard route.
On a fleet member managed by Front Desk, each configured cap is divided by the
number of active members, so the local shares add up to the configured global
limit instead of multiplying it. RPS divides exactly; burst and TPM are floored
to 1 so a small cap on a large fleet cannot round down to "block everything" or
to "no cap". An unlimited RPS (0) is never divided. The divisor reverts to 1
if Front Desk stops announcing for 24 hours, so a member that leaves the fleet
goes back to enforcing the full cap rather than a frozen fraction of it.
Outside a fleet the buckets are per-process and not shared between replicas, so N instances behind a plain load balancer enforce roughly N times the cap.
- Stale buckets: Automatically removed after 10 minutes of inactivity
- Disable → Re-enable: All buckets reset when rate limiting is re-enabled at runtime
- Limit change: Editing a key's RPS, burst or TPM adjusts its live bucket (refill rate and ceiling) and keeps what the key has already spent, so rewriting a cap never refills a drained bucket; a raised cap relieves a throttled key at the new refill rate rather than at once
A virtual key can belong to a dashboard user. owner_user_id is that link, and
setting it changes four things.
The owner's account switch reaches the proxy. Disabling the account rejects
every request on every key it owns with 401 virtual key disabled: owner account is disabled. The keys themselves are untouched, so re-enabling the account
restores them. This is the only reversible way to cut a key's traffic.
The owner's limits apply on top of the key's. A user row carries its own
rate_limit_rps, rate_limit_burst, and rate_limit_tpm, and they are
enforced in aggregate across every key that user owns. Unlike the per-key
fields, null here means "no cap" rather than "fall back to the global
setting", so an account without limits adds nothing. A request must clear the
key's bucket and the owner's bucket to be served, and the reply names whichever
one refused it.
The owner's provider cap binds the key. A key may never name a provider
outside its owner's allowed_providers. Creating or updating a key with a
provider the owner cannot reach is a 400; leaving allowed_providers unset on
a capped owner stores the owner's cap explicitly rather than leaving the key
unrestricted. The rule binds admins too, so the dashboard can never advertise
access the proxy would deny. Raising the account's access first is always
available to an admin.
The dashboard scopes itself to the owner. A non-admin with the
virtual_keys grant lists, creates, edits, and deletes only their own keys, and
any attempt to name a different owner in the request body is ignored: their keys
are always created and kept as their own. Only an admin can assign a key to
someone else or unassign it. Deleting a user does not delete their keys; it
orphans them (owner_user_id becomes null), and an orphaned key is subject to
neither an account switch nor account limits.
Owned keys show the owner's username as a chip next to the key name on the Virtual Keys page. See Multi-User for accounts, roles, and grants.
A key can carry a spending cap in dollars, and so can a user account. The pair
budget_usd + budget_period (day, week or month, calendar periods in
UTC; a week runs Monday to Sunday) is set on key creation and update, on the
account through the Users page, and by config sync. Both fields travel
together: a budget without a period, or a period without a budget, is refused
with 400, and so is an amount at or below zero, or above 10,000,000.
Who sets it. A key's budget belongs to whoever may edit the key: an admin, or the
owner through the virtual_keys grant, who can raise, lower or clear it on their own
keys and mint keys without one. It is a self-service cap, not a limit the owner cannot
cross. The durable cap on a user is the account budget, which only an admin sets; it sums
every request stamped to the user, whichever key wrote it, so a cleared key budget still
spends against it.
What counts. Spend is the sum of cost_usd over the period's request logs,
the same figure the Dashboard's $ view and the request log's Cost column
show. Rows the gateway could not price (a model with no known prices) count as
nothing: a budget cannot police what has no price, and the spend tile's
unpriced count says how much that is. A key's spend is its own rows; a user's
spend is every row written while they owned the key that wrote it, plus their
dashboard chat, so handing a key to someone else or deleting it leaves what
was spent with whoever spent it. The sum is read from the request logs, so a
log retention shorter than the budget period (Settings, Data & Storage) drops
the purged spend from the budget as well; keep retention at least as long as
the longest period in use.
How it refuses. Once the period's spend reaches the budget, the next
request answers 429 with key budget exceeded (or user budget exceeded)
and a Retry-After set to the seconds left in the period. The request that
crosses the line is served, since its cost is known only once the provider
reports usage. Both budgets apply when a key is owned: the key's own, then the
account's. The account budget also covers the owner's dashboard chat, which has
no key. A budgeted subject whose period has not been summed yet because the
database did not answer is refused with 503 (Retry-After: 5) rather than
admitted blind; once a period has been summed, a later failed refresh serves
the last known figure.
What a request costs. The row is priced by what the provider billed: its reported usage, or, when it reported none (some streams end without a usage chunk), the same estimate from the delivered bytes that charges the token limits. The row's token columns keep the provider's figures either way. An attempt that answered 2xx and then did not serve is charged too, at its own model's price (or the serving model's, when its own has none): an empty non-streaming answer that failed over at the prompt it reported, a stream whose first-token probe failed or that lost a hedged race at an estimate from the request text. A stream the provider ended with its own error frame is not charged.
Editing. A key or account update that does not mention the budget keeps
it: an integration written before budgets existed cannot drop a spending
guard by resubmitting the fields it knows. Sending both fields as null
clears it.
Where it is counted. Per member, like the token limits: each Model Hotel sums its own request logs, caches the figure for a minute and adds each priced request to it as it lands, so a burst inside that minute still counts. Spend metered on another fleet member is not seen until that member's rows are read by it, so a fleet-wide cap is the sum of each member's.
What it tells you. Two alert events fire once per period per key or user:
budget.warning at 80% of the budget and budget.exceeded at the first
refusal (see Alerting). A subject already past a threshold when a member
first sums it after a restart is not reported again. The key's row and detail
modal show $3.25 of $25.00 this month; the user's budget and its spend show
in the Users form.
Two additional per-key controls:
-
allowed_providersrestricts which providers a key may route to. When set, requests resolving to a provider outside the list are rejected, andhotel/failover candidates from disallowed providers are skipped.GET /v1/modelsis filtered by the same list, so a restricted key lists only the models it could actually call; a key restricted on neither its own list nor its owner's account cap is unaffected and still sees the whole catalog.nullmeans all providers are accessible; an empty array is rejected on create/update (usenullto clear the restriction). -
strip_reasoningremovesreasoning/reasoning_contentfields from streaming output for that key - useful for clients that mishandle reasoning deltas from thinking models. Token counting is unaffected.
Both are configurable on key creation and update (API or dashboard).
A completed request adds its token total to the key's tokens_used and stamps
last_used_at in the same statement. The same total is what the TPM budget is
debited by.
- When: After proxy request completion
-
What:
prompt_tokens + completion_tokensfrom the provider response (reasoning_tokensis a breakdown of completion, never added on top). Each figure is clamped to a sane bound before it is charged, so a nonsense count from an upstream cannot poison the tally, and a total of zero or less is not written at all -
How: Synchronously at the end of the request, on a detached context with a 5-second timeout (so a client that has already gone does not cancel the write). A failed write publishes a
tokens.errorevent rather than failing the request - Accuracy: Best-effort tally - a failed write is not retried
- Keyless requests: Admin chat has no virtual key, so it updates no key row. Its tokens are still debited from the owner's TPM budget
last_used_at is also stamped at authentication time, independently of whether
the request goes on to succeed.
-
When: During
ProxyKeyMiddleware(async, non-blocking) - Timeout: 5 seconds (prevents blocking proxy path)
- Purpose: Identify active vs. dormant keys for cleanup
The dashboard prints these for you. The Virtual Keys page carries a ready-made snippet block (cURL, PowerShell, Python, JavaScript, plus client configs for Claude Code, OpenCode, OpenClaw, LibreChat, ZED, and Hermes), and the same block appears in the creation dialog with the new key already substituted in, so it can be copied while the plaintext key is still on screen.
# Set virtual key
export PROXY_KEY="sk-a1b2c3d4e5f6789012345678abcdef01"
# List available models
curl http://localhost:8081/v1/models \
-H "Authorization: Bearer $PROXY_KEY"
# Chat completion
curl -X POST http://localhost:8081/v1/chat/completions \
-H "Authorization: Bearer $PROXY_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "hotel/glm-4.6",
"messages": [{"role": "user", "content": "Hello!"}]
}'import openai
client = openai.OpenAI(
base_url="http://localhost:8081/v1",
api_key="sk-a1b2c3d4e5f6789012345678abcdef01"
)
response = client.chat.completions.create(
model="hotel/glm-4.6",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'http://localhost:8081/v1',
apiKey: 'sk-a1b2c3d4e5f6789012345678abcdef01'
});
const response = await client.chat.completions.create({
model: 'hotel/glm-4.6',
messages: [{ role: 'user', content: 'Hello!' }]
});
console.log(response.choices[0].message.content);- Create new virtual key via API/dashboard
- Update client applications with new key
- Monitor usage of old key (via
last_used_at) - Delete old key when no longer needed
| Property | Implementation |
|---|---|
| Storage | SHA-256 hash only - raw key never persisted |
| Key format |
sk- + 32 hex chars (128 bits entropy) |
| Key preview | First 3 + last 4 chars stored as key_preview (e.g., sk-...ef01) |
| Deletion | Permanent - DELETE removes key entirely |
| Reversible cutoff | Disabling an owner's account rejects every key it owns, without deleting them |
| Ownership scoping | Non-admin grant holders see and edit only their own keys |
| Per-key tracking | Token usage logged per virtual key |
| Rate limiting | Independent token bucket per key |
| Audit trail | Logs retain virtual_key_name after deletion |
Symptoms: Requests rejected with Invalid virtual key
Causes:
- Key was deleted from database
- Typo in key value
- Missing
Bearerprefix in Authorization header (or an emptyx-api-key) - Key was never created (check creation response)
If the message is virtual key disabled: owner account is disabled instead, the
key is fine: its owner's account has been disabled. Re-enable the account, or
move the key to another owner.
Resolution:
- Verify key exists:
GET /api/virtual-keys - Check
key_previewmatches your key's last 4 chars - Ensure header format:
Authorization: Bearer sk-...orx-api-key: sk-...
Symptoms: rate limit exceeded (request rate) or token rate limit exceeded (TPM), both with a Retry-After header
Resolution:
- Read the message first. A
userprefix means the owner's aggregate limit is the one that was hit, so raise it on the account rather than on the key. A plainrate limit exceededon a request whose key is not even valid came from the per-IP limiter ahead of authentication: raiserate_limit_ip_rpsfor that one - Check per-key limits:
GET /api/virtual-keys/{id} - Increase
rate_limit_rpsor set to0for unlimited - Increase
rate_limit_burstfor traffic spikes - For
token rate limit exceeded, raise or clear (null)rate_limit_tpm - Wait for
Retry-Afterseconds before retrying
On a Front Desk fleet, remember each member enforces its share of the cap: a
429 at what looks like a fraction of the configured limit is the fair-share
divisor doing its job.
Symptoms: Plaintext key not saved; only preview available
Resolution:
- Impossible to recover - this is by design
- Create a new key and update clients
- Delete the old key once migrated
- Security - How provider keys are encrypted and managed
- Configuration - Global and per-key rate limit configuration
-
Multi-User - Accounts, roles, and the
virtual_keysgrant that gates this page - Request Logging - How virtual keys appear in audit logs