Pluggable speech-to-text layer. All backends speak the OpenAI /v1/audio/transcriptions protocol.

When STT and TTS both select Speaches, one container serves both endpoints. If one selects the CPU variant and the other the GPU variant, the GPU variant wins and the bootstrapper prints a notice.

GPU limitation: the speaches compose service requests no NVIDIA runtime or device (issue #1373, open). speaches-container-gpu therefore gets no GPU: it runs on CPU or fails CUDA initialisation. Use speaches-container-cpu, or parakeet-container-gpu for GPU transcription.

Speaches is the portable container option. Parakeet has NVIDIA and Apple Silicon implementations. whisper.cpp is a native host build with Metal on Apple Silicon. Language coverage, accuracy, throughput and memory depend on the model, the audio, the options and the hardware. Atlas publishes no hardware-independent ranking; benchmark representative inputs on the deployment host.

The default Speaches container starts without a model:

./start.sh
curl http://localhost:63060/health

/health reports process liveness only. Speaches does not download models itself (verified against speaches @ v0.9.0-rc.3), and the compose file sets PRELOAD_MODELS: '[]'. Until a model is installed, /v1/audio/transcriptions returns HTTP 404 ("Model is not installed locally"); issue #799 tracks this.

Download the whisper-1 target once:

curl -X POST http://localhost:63060/v1/models/Systran/faster-whisper-large-v3

The model persists in the speaches-cache volume. As an alternative, add the ID to the JSON array in PRELOAD_MODELS in services/speaches/compose.yml. The direct endpoint is then http://localhost:63060/v1/audio/transcriptions.

NVIDIA Parakeet (run from the repository root, which holds .env):

./start.sh --stt-provider-source parakeet-container-gpu

After startup, read the configured token before the transcription request. Run from the repository root with the bootstrapper dependencies available through uv. The parser reads .env by default; use ATLAS_ENV_FILE for the active custom file. Missing or blank tokens stop the request:

PYTHONPATH=bootstrapper uv run --project bootstrapper python - <<'PYSTT'
import subprocess
import sys
from core.config_parser import ConfigParser

parser = ConfigParser(root_dir=".")
if not parser.env_file_path.is_file():
    sys.exit("The active Atlas env file does not exist.")
env = parser.parse_env_file()
token = env.get("PARAKEET_API_TOKEN", "")
if not token:
    sys.exit("Set a nonempty PARAKEET_API_TOKEN in the active Atlas env file.")
port = env.get("STT_PROVIDER_PORT") or "63055"
subprocess.run(
    ["curl", "-X", "POST", f"http://localhost:{port}/v1/audio/transcriptions",
     "-H", f"Authorization: Bearer {token}",
     "-F", "[email protected]", "-F", "model=whisper-1"], check=True,
)
PYSTT

macOS native acceleration:

# Option A: whisper.cpp (Metal). Build whisper-server and download a model
# first: see the whisper-cpp README. Run this in the whisper.cpp checkout.
./build/bin/whisper-server --host 0.0.0.0 --port 63042 \
  --model models/ggml-large-v3.bin \
  --inference-path /v1/audio/transcriptions \
  --convert &

# From the Atlas repository root:
./start.sh --stt-provider-source whisper-cpp-localhost

Before launch, prepare the active Atlas env file (.env by default; copy .env.example only if the target does not exist). Set a nonempty PARAKEET_API_TOKEN there; preserve an existing token. To generate a new value, run python3.12 -c 'import secrets; print(secrets.token_urlsafe(32))'.

Run the preflight from the Atlas repository root. Use the same ATLAS_ENV_FILE setting in both terminals. The preflight reads the configured value and stops if it is missing or blank. Restart the native provider after changing its token.

# Option B: Parakeet-MLX (MLX-native), from the repository root
python3.12 -m venv .venv-parakeet-mlx
. .venv-parakeet-mlx/bin/activate
python -m pip install -r services/parakeet/provider/mlx/requirements-locked.txt
PARAKEET_API_TOKEN=$(
  PYTHONPATH=bootstrapper python - <<'PYTOKEN'
import sys
from core.config_parser import ConfigParser

token = ConfigParser().parse_env_file().get("PARAKEET_API_TOKEN", "")
if not token:
    sys.exit("Set a nonempty PARAKEET_API_TOKEN in the active Atlas env file.")
print(token, end="")
PYTOKEN
) || exit 1
export PARAKEET_API_TOKEN
export PARAKEET_LOCALHOST_BIND_HOST=0.0.0.0
export PARAKEET_LOCALHOST_PORT=63042
cd services/parakeet/provider && python -m mlx.api_server &
./start.sh --stt-provider-source parakeet-localhost

See the whisper-cpp README for the whisper.cpp build, models and options, or the MLX README for Parakeet-MLX.

Parakeet and whisper.cpp load their model at startup. Speaches needs the model download in §3.

Every engine implements the same call shape:

POST http://<endpoint>/v1/audio/transcriptions
Content-Type: multipart/form-data

file=<binary audio>
model=whisper-1
language=en               (optional)
response_format=json      (optional: json, text, verbose_json)

Parakeet and whisper.cpp ignore model and use the loaded checkpoint. Speaches resolves model and returns HTTP 404 if it is not downloaded (see §3). Send whisper-1: Speaches maps it to Systran/faster-whisper-large-v3.

Both Atlas-managed Parakeet providers:

• accept exactly json, text or verbose_json (srt and vtt are not supported);
• stream request bodies to bounded temporary files, and delete them after success, rejection or failure;
• run inference off the API event loop.

Atlas-managed Parakeet also applies these rules:

• GET /health is public. All other routes need Authorization: Bearer <PARAKEET_API_TOKEN>.
• Capacity is reserved before multipart parsing. When the provider is full, it returns 429 before it reads the body.
• Model load and inference share one deadline (PARAKEET_INFERENCE_TIMEOUT_SECONDS). On timeout the provider returns 504 and exits with status 70.
• Docker restarts the container. Run native Parakeet under a service manager with restart-on-failure.
• The container publishes on loopback. Set HOST_BIND_IP=0.0.0.0: only for separately protected external access.
• Native mode needs a non-loopback PARAKEET_LOCALHOST_BIND_HOST for remote clients.

These guarantees do not apply to Speaches or whisper.cpp.

The bootstrapper writes OPEN_WEB_UI_STT_ENGINE (openai), OPEN_WEB_UI_STT_API_URL (${STT_ENDPOINT}/v1) and OPEN_WEB_UI_STT_API_KEY to .env. The Open WebUI compose file maps them to AUDIO_STT_ENGINE, AUDIO_STT_OPENAI_API_BASE_URL and AUDIO_STT_OPENAI_API_KEY. It hard-codes AUDIO_STT_MODEL=whisper-1.

When the microphone button works depends on the engine:

• Parakeet and whisper.cpp: once the service is healthy.
• Speaches: only after Systran/faster-whisper-large-v3 (the whisper-1 target) is downloaded (see §3).

OPEN_WEB_UI_STT_API_KEY is the Parakeet provider token for a Parakeet source. It is sk-unused for other STT engines, and empty when STT is disabled. The key stays in the Open WebUI server process; browser code does not see it.

For the managed Parakeet GPU source, healthy means the model is loaded. The API starts a deadline-bounded background load, and /health returns 503 until inference is available. Speaches /health reports process liveness only.

• Parakeet providers expect WAV, FLAC, MP3, M4A, OGG or OPUS. Other extensions log a warning and still go to the model.
• whisper.cpp accepts WAV, MP3 and FLAC. Other formats need --convert and ffmpeg on the PATH.
• Open WebUI (v0.6.32) converts AAC audio to MP3 before it sends it. It also converts any codec other than FLAC, M4A, MP3, MP4, MPEG, WAV or WebM. Browser microphone recordings (Opus or AAC) therefore arrive as MP3.

• Speaches
• Faster-Whisper
• Parakeet-TDT v3 model card
• Parakeet-MLX
• whisper.cpp
• OpenAI Whisper API spec

parakeet — Support tier: experimental — Capability contract declared; no cited cold-start, workflow, or upgrade qualification run yet (evidence at v0.1.0). speaches — Support tier: experimental — Capability contract declared; no cited cold-start, workflow, or upgrade qualification run yet (evidence at v0.1.0).

Speaches returns 404 "Model is not installed locally" — download the model (see §3). The healthcheck passes as soon as Uvicorn is up; it does not wait for a model.

Open WebUI mic button does nothing — check the env vars:

docker exec <project>-open-web-ui env | grep AUDIO_STT

If AUDIO_STT_ENGINE and AUDIO_STT_OPENAI_API_BASE_URL are empty, STT_PROVIDER_SOURCE is disabled. With Speaches, also check that the model is downloaded.

Parakeet GPU container OOMs — the default model's weights alone are about 2.5 GB; Atlas has not measured total VRAM use. Use speaches-container-cpu or a localhost engine. NeMo's Parakeet loader has no int8 compute-type control.

whisper.cpp not detected as localhost — make sure it's serving the /v1/audio/transcriptions path (use --inference-path).

No upstream calls.

Open the full-size diagram for a full-screen view.

• stt-provider ↔ minio — Why: transcripts vanish with the HTTP response — nothing persists source audio or transcript JSON. Pushing both to MinIO gives every service a stable URL and enables re-transcription on engine swap. Mechanism: new stt-transcripts bucket provisioned by minio-init; post-transcribe hook puts s3://stt-transcripts/<sha256>.wav plus sidecar .json via S3 SigV4 over http://minio:9000. Effort: small. Confidence: high.
• stt-provider ↔ weaviate — Why: indexing durable transcripts turns long-form audio (meetings, podcasts, voice notes) into a semantically searchable corpus alongside the docling pipeline. Mechanism: Transcript class with text, start_ms, end_ms, source_audio_uri, vectorized by the active text2vec-openai module via http://weaviate:8080/v1/objects. Effort: medium. Confidence: medium.
• stt-provider ↔ redis — Why: transcription is expensive and deterministic in (audio-sha256, model, language). A cache cuts repeat cost to ~zero for n8n loops, re-runs, demos. Mechanism: a Redis database index that no other consumer uses (Redis README), with key stt:{sha256}:{model}:{lang} → transcript JSON, TTL 30d. A backend wrapper or a Kong plugin in front of STT_ENDPOINT does the lookup. Effort: small. Confidence: medium.
• stt-provider ↔ doc-processor — Why: docling parses PDFs/Office docs but does not handle audio. Composing stt → docling gives a unified "any media → markdown" ingest. Mechanism: caller hits STT_ENDPOINT, then POSTs transcript text to ${DOCLING_ENDPOINT}/v1/document/convert as text/plain. No new service. Effort: small. Confidence: medium.
• stt-provider ↔ openclaw — Why: Telegram/WhatsApp/Discord deliver voice notes as audio. OpenClaw's default text path uses LiteLLM. Atlas does not wire a direct Hermes bridge. Mechanism: Proposed OpenClaw middleware would POST incoming audio to ${STT_ENDPOINT}/v1/audio/transcriptions (multipart), then forward the transcript to its configured LLM path. Effort: small. Confidence: medium.
• stt-provider ↔ supabase — Why: transcript metadata (user, session, source URI, model, language, duration) belongs in a relational store. It gives open-webui and backend a "my transcripts" view keyed by Supabase JWT sub. Mechanism: transcripts table via PostgREST at http://supabase-api:3000, RLS on auth.uid(); post-transcribe hook writes rows pointing at MinIO URIs. Effort: medium. Confidence: medium.

• WhisperX (details) — Headline: fourth STT engine adding speaker diarization and word-aligned timestamps behind the existing OpenAI shape. Wires into: backend, n8n, open-webui, hermes, openclaw, minio, weaviate.

• Streaming / Realtime SSE+WebSocket — Why pursue: Speaches ships SSE-streamed transcription and a WebSocket realtime API; we only expose the batch /v1/audio/transcriptions. Enables live captions in open-webui and live agent voice loops in Hermes. Effort: medium.
• Translation endpoint — Why pursue: Speaches serves /v1/audio/translations, but no Atlas consumer calls it. Cheap multilingual UX gain. Effort: small.
• Per-engine model hot-swap — Why pursue: Speaches loads/unloads models on demand; we hard-pin one model per engine. Lets users A/B distil-large-v3 vs large-v3 without restarting. Effort: small.
• Word/segment timestamps in API responses — Why pursue: Parakeet and Speaches both expose them; open-webui wiring requests plain json and discards them. Needed for click-to-seek UX and for Weaviate chunking by utterance. Effort: small.
• Diarization — Why pursue: Speaches 0.9.0-rc.3 serves /v1/audio/diarization, but Atlas preloads no diarization model and no consumer calls it. It is a prerequisite for meeting-grade transcripts (see also the WhisperX candidate). Effort: medium.