WhisperLiveKit
Self-hosted speech recognition and real-time translation server supporting streaming Whisper, Voxtral, and diarization.
Self-host low-latency streaming transcription, speaker diarization, and translation with OpenAI-compatible APIs.
WhisperLiveKit is an open-source speech recognition and streaming translation server engineered for low-latency live audio processing. It serves localization teams, software developers, and translators who need real-time transcription and multilingual translation hosted on their own hardware. It provides OpenAI and Deepgram compatible endpoints out of the box.
What it is
The software buffers incoming audio streams and processes them incrementally using policies like AlignAtt and LocalAgreement to prevent mid-word truncation. It handles multi-speaker audio via streaming Sortformer diarization and translates speech into over 200 languages in real time. It exposes WebSocket endpoints, REST routes, and includes a built-in web interface alongside CLI tools.
Who it's for
It suits developers and localization specialists building live captioning systems, subtitling workflows, or voice applications without cloud dependencies. It does not suit non-technical users who want a one-click desktop application, as setup requires terminal usage and model selection.
Key features
Simultaneous Multilingual Speech Translation
Translates live audio into 200 languages using integrated NLLW and AlignAtt4LLM decoder models.
Real-Time Speaker Diarization
Identifies and separates multiple concurrent speakers live using Streaming Sortformer or NeMo backends.
API Compatibility Layer
Exposes endpoints compatible with OpenAI Python SDKs and Deepgram WebSockets for straightforward drop-in integration.
Low-Latency Incremental Processing
Uses AlignAtt and LocalAgreement policies to eliminate mid-syllable word clipping during live audio streaming.
Up and running in 3 steps
Our honest verdict
What's great
- Provides dual OpenAI REST and Deepgram WebSocket API compatibility for existing application codebases.
- Supports multiple modular backends including Whisper, Voxtral, Qwen3-ASR, and SenseVoice.
- Translates live audio contextually across 200 target languages using dedicated translation modules.
- Runs entirely on local hardware with support for CUDA, Apple Silicon MLX, and CPU execution.
Where it falls short
- Managing optional extras requires multiple isolated Python environments due to packaging conflicts between backends like vLLM and MLX.
- Low-latency performance on larger models demands modern GPU hardware or recent Apple Silicon chips.
- Setting up production instances requires technical knowledge of terminal commands, WebSocket protocols, and custom SSL configuration.
Pricing & license
WhisperLiveKit is completely free and open source under the Apache-2.0 license. The developer provides no paid hosted tiers, so your only expense is the server infrastructure or hardware needed to execute the models.
Open remote AI-training roles
870 open remote AI-training roles, updated November 14, 2026; most pay $20–$170 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
These reviewed AI tools do the same job instantly — from our scored directory.
Use it, or have us set it up
FAQ
Does WhisperLiveKit generate subtitle files?
Yes, you can run wlk transcribe --format srt file.mp3 to generate subtitle files directly from the command line.
Can I run this software on Apple Silicon Macs?
Yes, it provides dedicated optional packages for MLX-accelerated Whisper and Voxtral models on macOS hardware.
