Provider Parakeet Overview - thekaveh/atlas GitHub Wiki

5.3.2. Parakeet Provider Overview

Pluggable speech-to-text layer. All backends expose an OpenAI-compatible /v1/audio/transcriptions endpoint so Open WebUI, n8n, and the backend API can use them interchangeably.

1. Available backends

`STT_PROVIDER_SOURCE`	Engine	License	Runs on
`speaches-container-cpu`	Speaches (Faster-Whisper inside)	MIT	Linux + macOS Docker, CPU
`speaches-container-gpu`	Speaches CUDA build	MIT	Not usable yet: no GPU device is attached (#1373)
`parakeet-container-gpu`	NVIDIA Parakeet-TDT (NeMo)	Model CC-BY-4.0	NVIDIA
`parakeet-localhost`	Parakeet-MLX or another operator-run Parakeet server	Model CC-BY-4.0	macOS MLX / Linux
`whisper-cpp-localhost`	whisper.cpp	MIT	macOS Metal / Linux
`disabled`	none	—	—

The default for fresh installs is speaches-container-cpu. It starts on every platform with no host install, but it has no model. The pinned Speaches release does not download a missing model on request, and PRELOAD_MODELS is empty (#799). Install the model as described in the STT Provider quick start.

For Mac users, both whisper-cpp-localhost (Metal) and parakeet-localhost (MLX) provide native acceleration. Benchmark the chosen model and representative audio on the target host; Atlas does not make a hardware-independent speed or quality ranking.

2. Directory layout

services/parakeet/provider/
├── mlx/                Apple Silicon MLX server for Parakeet (parakeet-localhost)
│   ├── api_server.py
│   ├── README.md
│   ├── requirements.txt
│   └── requirements-locked.txt
├── gpu/                NVIDIA CUDA container build for Parakeet (parakeet-container-gpu)
│   ├── Dockerfile
│   ├── requirements.txt
│   └── transcribe.py
├── whisper-cpp/        whisper.cpp host install notes (whisper-cpp-localhost)
│   └── README.md
└── shared/             Common server scaffolding
    ├── api_server.py
    └── utils.py

The Speaches path doesn't have a directory here because it's an off-the-shelf container — see services/speaches/compose.yml for the runtime config.

3. Quick start

Speaches (default — already enabled in .env.example):

./start.sh
curl http://localhost:63060/health

This proves the default Speaches container is up. Transcription remains unavailable until the whisper-1 alias target is installed or preloaded; see the STT Provider quick start (#799).

Parakeet on NVIDIA GPU:

./start.sh --stt-provider-source parakeet-container-gpu

The GPU API starts a deadline-bounded background load for the configured Parakeet model. Its health endpoint and transcription routes return 503 until the model is loaded. Health-aware callers can therefore wait for inference readiness, even though consumers may start independently.

Parakeet on macOS MLX:

Before launch, prepare the active Atlas env file (.env by default; copy .env.example only if the target does not exist). Set a nonempty PARAKEET_API_TOKEN there; preserve an existing token. To generate a new value, run python3.12 -c 'import secrets; print(secrets.token_urlsafe(32))'.

Run the preflight from the Atlas repository root. Use the same ATLAS_ENV_FILE setting in both terminals. The preflight reads the configured value and stops if it is missing or blank. Restart the native provider after changing its token.

# Terminal 1 — run from repo root with the lock's certified interpreter
python3.12 -m venv .venv-parakeet-mlx
. .venv-parakeet-mlx/bin/activate
python -m pip install -r services/parakeet/provider/mlx/requirements-locked.txt
PARAKEET_API_TOKEN=$(
  PYTHONPATH=bootstrapper python - <<'PYTOKEN'
import sys
from core.config_parser import ConfigParser

token = ConfigParser().parse_env_file().get("PARAKEET_API_TOKEN", "")
if not token:
    sys.exit("Set a nonempty PARAKEET_API_TOKEN in the active Atlas env file.")
print(token, end="")
PYTOKEN
) || exit 1
export PARAKEET_API_TOKEN
export PARAKEET_LOCALHOST_BIND_HOST=0.0.0.0
export PARAKEET_LOCALHOST_PORT=63042
cd services/parakeet/provider && python -m mlx.api_server

# Terminal 2
./start.sh --stt-provider-source parakeet-localhost

The MLX health endpoint starts one shared background model load and returns 503 with status=loading while that work is in progress. Concurrent health and transcription requests share the same load; model initialization never runs on the API event loop. Both Parakeet providers default advanced segment timestamps to disabled unless return_timestamps=true is supplied.

PARAKEET_MAX_UPLOAD_BYTES is parsed as a positive integer during provider startup; malformed, zero, and negative values fail fast before the API serves. The whole request body must arrive within PARAKEET_UPLOAD_TIMEOUT_SECONDS (1-3600 seconds; default 120). Otherwise the provider returns 408 and releases its admission slot.

whisper.cpp on macOS (Metal):

# Terminal 1: in a whisper.cpp checkout with whisper-server built and
# models/ggml-large-v3.bin downloaded (see whisper-cpp/README.md)
./build/bin/whisper-server --host 0.0.0.0 --port 63042 \
  --model models/ggml-large-v3.bin \
  --inference-path /v1/audio/transcriptions \
  --convert

# Terminal 2: from the Atlas repository root
./start.sh --stt-provider-source whisper-cpp-localhost

See whisper-cpp/README.md for the build (Homebrew does not ship whisper-server), model downloads and options.

Disable STT entirely:

./start.sh --stt-provider-source disabled

4. Performance evaluation

Throughput and word-error rate depend on the exact checkpoint, audio corpus, codec, duration, timestamps, hardware, and thermal state. Benchmark representative inputs on the deployment host and record real-time factor, word-error rate, and peak resident/GPU memory before capacity planning.

5. How Open WebUI is wired

The bootstrapper sets these env vars on the open-web-ui container based on the chosen source:

AUDIO_STT_ENGINE=openai
AUDIO_STT_OPENAI_API_BASE_URL=${STT_ENDPOINT}/v1
AUDIO_STT_OPENAI_API_KEY=${OPEN_WEB_UI_STT_API_KEY} (source-aware bearer token)
AUDIO_STT_MODEL=whisper-1 (the OpenAI-compatible model name all engines accept)

You can change the model name in the Open WebUI admin panel — Audio settings.

6. Full configuration reference

See services/stt-provider/README.md.