so-vits-svc — Open-Source Singing Voice Conversion with AI

Introduction

so-vits-svc is a singing voice conversion (SVC) framework built on top of SoftVC and VITS. It allows users to transform the timbre of a singing voice into another while preserving the original melody and rhythm. With minimal training data, it can produce remarkably natural voice conversions suitable for music production, content creation, and research.

What so-vits-svc Does

• Converts singing voice timbre while preserving pitch and rhythm
• Trains custom voice models from short audio clips (as few as 5 minutes)
• Supports real-time voice conversion for live performance scenarios
• Provides a modular pipeline with feature extraction, training, and inference stages
• Integrates with common audio formats and DAW workflows

Architecture Overview

so-vits-svc combines a content encoder (HuBERT or ContentVec) to extract linguistic features, an F0 estimator for pitch tracking, and a VITS-based decoder that synthesizes the target voice. The content encoder strips speaker identity from the source audio while the decoder reconstructs it with the target speaker's characteristics. This separation of content and timbre enables high-quality conversion with minimal artifacts.

Self-Hosting & Configuration

• Requires Python 3.8+ and a CUDA-compatible GPU for training and fast inference
• Install dependencies via pip from the provided requirements file
• Prepare training data by segmenting audio into short clips and running preprocessing scripts
• Configure model parameters in JSON config files covering sample rate, batch size, and epochs
• Use the provided Jupyter notebooks or CLI scripts for both training and inference

Key Features

• Produces natural-sounding voice conversions with minimal training data
• Supports multiple content encoder backends including HuBERT and ContentVec
• Includes F0 prediction for accurate pitch preservation across octaves
• Provides both CLI and notebook interfaces for flexible workflows
• Active community with extensive guides and pre-trained model sharing

Comparison with Similar Tools

• RVC — Faster training and inference, simpler setup, but may produce slightly less natural results for singing
• DDSP-SVC — Uses differentiable DSP for synthesis, lighter weight but narrower voice range
• GPT-SoVITS — Focuses on text-to-speech with voice cloning rather than singing voice conversion
• OpenVoice — Targets cross-lingual voice cloning for speech, not optimized for singing

FAQ

Q: How much training data do I need? A: A minimum of 5-10 minutes of clean, isolated vocal audio is recommended for decent results. More data generally improves quality.

Q: Can I run inference without a GPU? A: Yes, CPU inference is supported but significantly slower. A CUDA GPU is recommended for practical use.

Q: Does it preserve the original pitch and melody? A: Yes, the F0 extraction pipeline preserves the original pitch contour while converting only the voice timbre.

Q: Is real-time conversion possible? A: With a capable GPU, near-real-time conversion is achievable, though some latency is expected depending on model size.

Sources

• https://github.com/svc-develop-team/so-vits-svc
• https://github.com/svc-develop-team/so-vits-svc/wiki