Home - hugalafutro/model-hotel GitHub Wiki

Model Hotel

Multi-Provider AI Gateway

A single OpenAI-compatible endpoint that sits in front of all your LLM providers. Route requests across models using your own priority ordering, fail over automatically when a provider goes down, and see exactly where your tokens are going.

Dashboard

Quick Start

git clone https://github.com/hugalafutro/model-hotel.git
cd model-hotel
cp .env.example .env
nano .env              # set MASTER_KEY and POSTGRES_PASSWORD; change HOST_PORT if 8081 is taken
docker compose up --build -d

To run the published image instead of building, follow Deploy without Git in the README: two files, no clone.

On first start the server prints the admin token once, in a boxed ADMIN TOKEN (save now ...) panel. Read it with docker compose logs app, and save it: it is never shown again. To stop, update or remove the stack, see Stop, Update, Remove in the README.

See Development for local setup details.

Key Features

  • Unified API - One OpenAI-compatible surface for all providers: /v1/chat/completions plus multimodal endpoints (embeddings, image generation/edits/variations, text-to-speech, speech-to-text)
  • Hotel Routing - Prefix models with hotel/ to route through failover groups (works on every endpoint)
  • Transparent Failover - Automatic retry on 5xx, 401/403, 402, 404, 499, and timeouts, plus 429 while the failover_on_rate_limit setting is on (the default)
  • Circuit Breaker - Health is tracked per provider and model: a repeatedly failing model is skipped, and a provider whose failures span enough of its models is taken out of rotation too
  • Virtual Keys - Per-client API keys with rate limiting and usage tracking
  • Model Discovery - Auto-sync models across 22 provider families, with a post-scan summary of what changed (added / re-enabled / disabled models, failover group updates)
  • Request Logging - Full latency decomposition (TTFT, overhead, per-stage timing)
  • Privacy by Design - Prompts are never logged, read, or stored
  • Interactive Chat & Arena - Built-in UI for testing and comparing models

Documentation

Getting Started

  • Configuration - Environment variables, runtime settings, appearance
  • Development - Local setup, project structure, contributing

Using

  • Virtual Keys - Per-client API key management, rate limiting, usage tracking
  • Multi-User - Named dashboard accounts with per-user roles and scoped access
  • API Reference - Proxy and admin API endpoints with examples
  • Request Logging - Latency decomposition, log management, app logs

Operating

  • Model Discovery - Automatic model synchronization with per-provider metadata
  • Failover and Hotel Routing - Transparent failover, hotel routing, circuit breaker
  • Observability - Prometheus metrics, JSON logs, OTLP log export, and a provisioned Grafana dashboard
  • Alerting - Outbound notifications for operational events, via Telegram, email, Discord, Slack, Matrix, webhooks, and more
  • High Availability - Front Desk control plane + Traefik for drop-in multi-instance HA
  • Bellhop - Android companion app: pair a phone with Front Desk and monitor the fleet
  • CrowdSec - Parsers and scenarios that turn repeated auth failures into an edge ban

Reference

  • Security - AES-256-GCM encryption, Argon2id, SHA-256 hashing, URL validation
  • Privacy - Data handling, what is and isn't captured, local deployment

Architecture

Architecture

Core packages: proxy/ (routing, failover loop, streaming), provider/ (discovery), failover/ (groups, circuit breaker), auth/ (provider key encryption), virtualkey/ (client key auth), ratelimit/ (rate limiting), budget/ (dollar budgets), model/ (caching, CRUD). PostgreSQL backend (schema managed by versioned migrations).