# Vocode — Open-Source Voice AI Agent Framework

> Vocode is an open-source framework for building voice-based AI agents and applications. It provides modular abstractions for speech-to-text, LLM conversation, and text-to-speech, enabling developers to create phone callers, voice assistants, and interactive voice response systems.

## Install

Save in your project root:

# Vocode — Open-Source Voice AI Agent Framework

## Quick Use
```bash
pip install vocode
# Set up API keys for your chosen providers
export OPENAI_API_KEY=your_key
export DEEPGRAM_API_KEY=your_key
export AZURE_SPEECH_KEY=your_key
# Run a streaming conversation agent
python quickstart.py
```

## Introduction
Vocode is an open-source Python library for building real-time voice AI agents. It abstracts the complexity of coordinating speech recognition, language model conversation, and speech synthesis into a unified pipeline, so developers can focus on conversation design rather than audio infrastructure plumbing.

## What Vocode Does
- Orchestrates real-time voice conversations by chaining STT, LLM, and TTS components
- Supports telephony integration for building AI phone agents via Twilio and Vonage
- Provides streaming conversation handlers for WebSocket and raw audio connections
- Manages interruption handling, endpointing, and turn-taking in voice dialogues
- Includes action framework for triggering external functions mid-conversation

## Architecture Overview
Vocode uses an event-driven architecture centered around a conversation pipeline. Audio input streams through a configurable STT transcriber, the transcript feeds into an LLM-powered agent that generates responses, and the response text passes to a TTS synthesizer for audio output. Each component (transcriber, agent, synthesizer) is a pluggable abstraction, and the orchestrator manages streaming, buffering, and interruption detection across them.

## Self-Hosting & Configuration
- Install via pip and configure API keys for your preferred STT, LLM, and TTS providers
- Choose transcribers from Deepgram, AssemblyAI, Azure, or Whisper
- Select synthesizers from Azure, ElevenLabs, Play.ht, Google, or Bark
- Set up telephony with Twilio or Vonage for phone-based voice agents
- Deploy as a FastAPI server with WebSocket endpoints for web-based voice interfaces

## Key Features
- Real-time streaming with low-latency audio processing for natural conversations
- Interruption handling allows users to cut in mid-response like human conversation
- Action framework lets voice agents call APIs, query databases, or transfer calls
- Support for multiple STT/TTS providers with a unified interface for easy switching
- Phone integration enables building AI receptionists, appointment schedulers, and survey bots

## Comparison with Similar Tools
- **LiveKit Agents** — WebRTC-focused infrastructure; Vocode focuses on the conversation orchestration layer
- **Pipecat** — real-time voice AI framework; Vocode emphasizes telephony and modular provider swapping
- **Retell AI** — commercial voice agent platform; Vocode is open source and self-hostable
- **VAPI** — managed voice API service; Vocode gives full control over the pipeline locally
- **Bland AI** — commercial phone AI; Vocode provides the same telephony capabilities as open source

## FAQ
**Q: What is the end-to-end latency?**
A: Typical latency is 500ms-1.5s depending on the STT, LLM, and TTS providers selected. Streaming providers like Deepgram and ElevenLabs offer the lowest latency.

**Q: Can I use local models instead of cloud APIs?**
A: Yes. Configure Whisper for local STT, any OpenAI-compatible local LLM server, and Bark or Coqui for local TTS.

**Q: Does it handle multiple concurrent calls?**
A: Yes. Each conversation runs in its own async context, and the telephony server can handle multiple simultaneous calls.

**Q: Is it production-ready?**
A: Vocode is used in production by several startups for phone-based AI agents. Monitor provider rate limits and costs for high-volume deployments.

## Sources
- https://github.com/vocodedev/vocode-core
- https://docs.vocode.dev/

---
Source: https://tokrepo.com/en/workflows/asset-4b7bafb5
Author: AI Open Source