LatentSync
ByteDance lip-sync model that re-syncs mouths in real video to new audio using latent diffusion.
Latent diffusion lip-sync from ByteDance, with an 8 GB and an 18 GB VRAM release.
LatentSync is an end-to-end lip-sync method from ByteDance based on audio-conditioned latent diffusion. It suits developers who need dubbing or translation of existing footage with accurate mouth movement.
What it is
The README says it models audio-visual correlations directly with Stable Diffusion, with no intermediate motion representation. It ships v1.5 (8 GB VRAM inference) and v1.6 (18 GB), a Gradio app, command-line inference and training configs.
Who it's for
Suits developers and post-production teams with an NVIDIA GPU who already have footage of a speaker. It does not create a presenter from a script, and the weights are under a different licence from the code.
Key features
Latent diffusion lip-sync
End-to-end method without intermediate motion representation.
Two VRAM tiers
Inference needs 8 GB for v1.5 or 18 GB for v1.6.
Gradio app
Run the demo UI or the command-line script.
Training code
Stage 1 and stage 2 training configs are published.
Temporal layer
v1.5 adds a temporal layer for consistency.
Hugging Face weights
Checkpoints download from the ByteDance Hugging Face repo.
Up and running in 3 steps
Our honest verdict
What's great
- Apache-2.0 code from a large lab with training configs included.
- Offers an 8 GB option (v1.5) for mid-range GPUs.
- Works on real footage, so identity and background are preserved.
- Gradio and CLI inference paths are both provided.
Where it falls short
- Needs an NVIDIA GPU and a CUDA environment from setup_env.sh; v1.6 asks for 18 GB VRAM.
- The model weights use OpenRAIL++ on Hugging Face, which restricts some uses; read it before commercial work.
- It only changes lips in existing video; it does not generate a talking presenter.
- Training needs 20 GB or more of VRAM per the README.
Pricing & license
Free to run: Apache-2.0 code and OpenRAIL++ weights. The README lists no hosted plan, so the cost is your own or rented GPU time.
Open remote AI-training roles
767 open remote AI-training roles, updated November 14, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
These reviewed AI tools do the same job instantly — from our scored directory.
Use it, or have us set it up
FAQ
How much VRAM does LatentSync need?
The README says inference needs a minimum of 8 GB with v1.5 and 18 GB with v1.6.
Can I use LatentSync commercially?
The code is Apache-2.0, but the Hugging Face weights are listed under OpenRAIL++, which allows use with restrictions. Read that licence first.
Does it generate a new video from a photo?
No. It re-syncs the mouth in footage you provide.
