WhisperX — 70x Faster Speech Recognition

What it is

WhisperX is a speech recognition system that accelerates OpenAI's Whisper model to 70x realtime speed through batched inference. It adds word-level timestamps via forced alignment and speaker diarization to identify who said what. The project requires under 8GB of VRAM and is licensed under BSD-2-Clause.

Researchers, podcast producers, and developers building transcription pipelines will find WhisperX useful when standard Whisper is too slow or when per-word timing and speaker labels are required.

How it saves time or tokens

Standard Whisper processes audio sequentially, making long recordings slow to transcribe. WhisperX batches audio segments and processes them in parallel, cutting transcription time drastically. The word-level alignment and diarization run as post-processing steps, so you get richer output without re-running the model.

How to use

1. Install WhisperX with pip and ensure you have a CUDA-capable GPU with at least 8GB VRAM.
2. Run the CLI command with your audio file and desired output format.
3. Optionally enable speaker diarization by providing a HuggingFace token for the pyannote models.

Example

# Basic transcription with word-level timestamps
whisperx audio.mp3 --model large-v2 --output_dir ./output

# With speaker diarization
whisperx audio.mp3 --model large-v2 --diarize \
  --hf_token YOUR_HF_TOKEN --output_dir ./output

# Specify language and output format
whisperx audio.mp3 --model large-v2 --language en \
  --output_format srt --output_dir ./output

Related on TokRepo

• AI tools for voice — Other voice and speech processing tools for AI workflows.
• AI tools for automation — Automate transcription pipelines and audio processing tasks.

Common pitfalls

• Running without a CUDA GPU. WhisperX's speed advantage comes from GPU batching; CPU-only mode is significantly slower.
• Forgetting the HuggingFace token for diarization. The pyannote speaker diarization models require authentication through HuggingFace.
• Using the wrong model size for your VRAM. The large-v2 model needs close to 8GB; smaller GPUs should use the medium or small variants.