# Open-LLM-VTuber — Build Your Own AI Companion with Live2D and Voice

> Open-LLM-VTuber is a self-hosted framework for creating interactive AI companions that combine large language models with Live2D avatars and real-time speech synthesis for natural voice conversations displayed through animated characters.

## Install

Save in your project root:

# Open-LLM-VTuber — Build Your Own AI Companion with Live2D and Voice

## Quick Use
```bash
git clone https://github.com/Open-LLM-VTuber/Open-LLM-VTuber.git
cd Open-LLM-VTuber
cp conf.yaml.example conf.yaml
# Edit conf.yaml with your LLM API key and TTS settings
pip install -r requirements.txt
python main.py
```

## Introduction
Open-LLM-VTuber lets developers and creators build AI characters that listen, think, and speak with an animated Live2D avatar. It connects any LLM backend to speech recognition and text-to-speech engines, producing a real-time conversational experience rendered through expressive 2D character models.

## What Open-LLM-VTuber Does
- Drives Live2D character animations synchronized with AI-generated speech
- Supports multiple LLM backends including OpenAI, Ollama, and local models
- Integrates speech-to-text engines for real-time voice input from microphone
- Connects to various TTS providers for natural-sounding AI voice output
- Provides a web-based interface for interacting with the AI companion in a browser

## Architecture Overview
The system runs as a Python server with a web frontend. Audio from the user is captured by the browser, transcribed via a configurable ASR engine (Whisper, Faster Whisper, or cloud APIs), and sent to the chosen LLM for response generation. The LLM output is streamed to a TTS engine, and the resulting audio is synchronized with Live2D lip-sync and expression data using a motion mapping layer. The frontend renders the Live2D model via PixiJS and plays audio in sync.

## Self-Hosting & Configuration
- Clone the repo and install Python dependencies from requirements.txt
- Configure your preferred LLM provider in conf.yaml (OpenAI, Ollama, or any OpenAI-compatible API)
- Set up a TTS engine such as Edge TTS, VITS, or ElevenLabs in the config
- Choose an ASR provider for voice input (Whisper, Faster Whisper, or cloud services)
- Place Live2D model files in the models directory and reference them in the config

## Key Features
- Modular architecture lets you swap LLM, TTS, and ASR components independently
- Real-time lip-sync and expression mapping for natural character animation
- Supports both local and cloud AI backends for flexibility in deployment
- Browser-based interface requires no client installation
- Extensible persona system for defining character personality and behavior

## Comparison with Similar Tools
- **SillyTavern** — focuses on text-based character chat; Open-LLM-VTuber adds voice and animated avatar interaction
- **AI Vtuber** — typically relies on specific platforms; Open-LLM-VTuber is fully self-hosted and provider-agnostic
- **GPT-SoVITS** — specializes in voice cloning; Open-LLM-VTuber integrates voice with visual avatars and conversation
- **Character.AI** — proprietary cloud service; Open-LLM-VTuber runs entirely on your own infrastructure

## FAQ
**Q: Which LLMs work with Open-LLM-VTuber?**
A: Any model accessible via OpenAI-compatible API, including Ollama for local models and cloud providers.

**Q: Can I use my own Live2D model?**
A: Yes. Place your model files in the models directory and update the configuration to reference them.

**Q: Does it work offline?**
A: Yes, if you use local models for LLM (Ollama), TTS (VITS), and ASR (Whisper).

**Q: What are the hardware requirements?**
A: A modern CPU handles the pipeline. GPU is recommended only if running local LLM or TTS models.

## Sources
- https://github.com/Open-LLM-VTuber/Open-LLM-VTuber

---
Source: https://tokrepo.com/en/workflows/asset-cc97db30
Author: AI Open Source