# Wav2Lip — Accurate Lip Sync for Any Video

> An ACM Multimedia 2020 model that generates accurate lip movements on any face in a video to match a given speech audio.

## Install

Save as a script file and run:

# Wav2Lip — Accurate Lip Sync for Any Video

## Quick Use
```bash
git clone https://github.com/Rudrabha/Wav2Lip.git
cd Wav2Lip
pip install -r requirements.txt
python inference.py --checkpoint_path wav2lip_gan.pth --face input.mp4 --audio speech.wav
```

## Introduction
Wav2Lip generates accurate lip movements on any face in a video to match arbitrary speech audio. It works on faces of any identity, in any language, and preserves the original video quality outside the lip region.

## What Wav2Lip Does
- Synchronizes lip movements to any speech audio
- Works on videos with any face identity without retraining
- Handles multiple faces in a single video frame
- Preserves original video quality outside the mouth region
- Supports both pre-recorded and real-time audio inputs

## Architecture Overview
Wav2Lip uses a face encoder and an audio encoder that are jointly trained with a lip-sync discriminator. The discriminator is pre-trained as a sync expert that scores audio-lip alignment, forcing the generator to produce visually accurate mouth shapes. A face decoder reconstructs the lower face region while leaving the rest untouched.

## Self-Hosting & Configuration
- Requires Python 3.6+ with PyTorch and OpenCV
- Download pre-trained model weights from the project releases
- GPU with at least 4 GB VRAM for video inference
- Adjust face detection padding for better cropping results
- Use the GAN variant (wav2lip_gan.pth) for sharper output

## Key Features
- Language-agnostic lip sync that works across any spoken language
- Identity-agnostic approach requiring no per-person training
- High visual quality with the GAN-enhanced model variant
- Robust face detection handling partial occlusions
- Simple command-line interface for batch processing

## Comparison with Similar Tools
- **SadTalker** — generates full head motion; Wav2Lip focuses on lip accuracy
- **MuseTalk** — latent-space method; Wav2Lip uses direct pixel generation
- **Hallo** — portrait animation from still images; Wav2Lip works on existing video
- **Video Retalking** — adds face enhancement; Wav2Lip prioritizes sync precision

## FAQ
**Q: Does it work with singing or non-speech audio?**
A: It is trained on speech; singing may produce less accurate results.

**Q: Can I use it for dubbing films into other languages?**
A: Yes, this is one of the primary use cases for the model.

**Q: What video formats are supported?**
A: Any format readable by OpenCV, including MP4, AVI, and MOV.

**Q: How does quality compare to commercial solutions?**
A: The GAN model produces competitive results for most applications.

## Sources
- https://github.com/Rudrabha/Wav2Lip
- http://bhaasha.iiit.ac.in/lipsync/

---
Source: https://tokrepo.com/en/workflows/asset-d5285c54
Author: Script Depot