MeloTTS — High-Quality Multilingual Text-to-Speech by MyShell

Introduction

MeloTTS is a multilingual text-to-speech library developed by MyShell.ai that delivers high-quality speech synthesis across multiple languages. It is designed for both research and production use, with a focus on natural-sounding output, fast inference that works on CPU, and straightforward integration via a simple Python API.

What MeloTTS Does

• Synthesizes natural-sounding speech from text in English, Chinese, Japanese, Korean, French, and Spanish
• Runs efficiently on both CPU and GPU for flexible deployment
• Provides multiple speaker voices per language with distinct characteristics
• Supports mixed-language text handling within a single utterance
• Offers a simple Python API and CLI for quick integration

Architecture Overview

MeloTTS builds on a VITS-based architecture with language-specific text frontends for phoneme conversion and prosody prediction. Each language module handles text normalization, grapheme-to-phoneme conversion, and duration prediction tailored to that language's characteristics. The shared acoustic model and HiFi-GAN vocoder generate waveforms from the predicted mel spectrograms. The compact model size enables real-time inference on modern CPUs.

Self-Hosting & Configuration

• Install via pip with minimal dependencies
• Models download automatically on first use for each language
• Configure language and speaker ID via the Python API or CLI arguments
• Supports batch processing for converting large text documents to audio
• Deploy behind a FastAPI or Flask server for production TTS service

Key Features

• Covers six major languages in a single lightweight package
• CPU-friendly inference running faster than real-time without GPU
• Multiple speaker voices per language for diverse applications
• Clean Python API with just a few lines needed for basic synthesis
• Open-source MIT license allowing unrestricted commercial and research use

Comparison with Similar Tools

• Piper — Extremely fast and supports 30+ languages, but MeloTTS offers higher naturalness for its supported languages
• Coqui TTS — Broader model zoo and customization, while MeloTTS prioritizes ease of use and out-of-the-box quality
• Bark — Generates expressive audio with effects, while MeloTTS focuses on clean multilingual speech synthesis
• Kokoro — Similar lightweight approach, but MeloTTS covers more languages natively

FAQ

Q: Which languages are supported? A: English, Chinese (Mandarin), Japanese, Korean, French, and Spanish, with community efforts adding more.

Q: Can it run without a GPU? A: Yes, MeloTTS is optimized for CPU inference and achieves faster-than-real-time speeds on modern processors.

Q: How do I switch between speakers? A: Pass a different speaker_id to the tts_to_file method. Each language provides multiple built-in speaker options.

Q: Is commercial use permitted? A: Yes, MeloTTS is released under the MIT license, allowing commercial use without restrictions.

Sources

• https://github.com/myshell-ai/MeloTTS
• https://melo.myshell.ai/