AI tools, scored for your job
Get paid to train AI
🤝 You're in the Free & Open Source library — community-built tools you can use without paying. A different concept from our scored AI directory.
Home › Free & Open Source › Video › Text2Video-Zero
✦ Free & open sourceVideo

Text2Video-Zero

Generates and edits temporally consistent videos using text-to-image diffusion models without additional training.

Zero-shot text-to-video generation and editing

Transform existing text-to-image diffusion pipelines into conditional video generators and instruction editors.

Free forever · source-available · Licence not stated in the repository
✓ Needs IT✓ Browser-based✓ Real-time collaboration
5.4
ZEKAI
Zero-shot video generation tool
Free & open source · Video
Ease of setup4
Usefulness8
Maintenance2
Community7.5
★ 4,243
GITHUB
May 2023
UPDATED
Licence not stated in the repository
LICENSE
🛠 Don't want to self-host? We can set it up →
⚡ Quick answer

Text2Video-Zero is a zero-shot video generation framework that adapts Stable Diffusion models for video production. It allows video creators and AI developers to generate videos from text, pose, edge, and depth inputs without training specialized video models. You should use it if you want to experiment with open-source video generation and instruction-based video editing.

Runs on
Cross-platform
Hosting
Cloud or self-host
Languages
Python
License
Licence not stated in the repository

What it is

The tool generates temporally consistent video clips directly from text prompts or structured guidance like pose skeletons and Canny edges. It enables instruction-guided video editing via Video Instruct-Pix2Pix and integrates with custom DreamBooth models. Integrations with Hugging Face Diffusers and Token Merging allow execution on consumer GPUs with as little as 7 GB VRAM.

Who it's for

This tool suits AI researchers, developers, and technical video editors looking to adapt Stable Diffusion pipelines for video creation. It is ideal for teams comfortable running Python and PyTorch environments on dedicated GPU hardware. It is not suitable for non-technical editors who require a production-ready graphical interface out of the box.

Key features

Zero-Shot Text to Video

Generates temporally coherent video sequences from text prompts using pre-trained Stable Diffusion models.

Pose and Edge Control

Guides video generation using motion skeletons, Canny edge detection, or MiDaS depth maps.

Instruction Based Video Editing

Edits existing video clips using text commands via Video Instruct-Pix2Pix.

Custom Model Support

Loads arbitrary Stable Diffusion base models and custom DreamBooth weights from Hugging Face or Civitai.

How to start

Up and running in 3 steps

1
Clone the repository and install requirements in a Python 3.9 environment with CUDA 11.6 support.
2
Instantiate the Model class in PyTorch using float16 precision for GPU execution.
3
Run inference functions like process_text2video or process_pix2pix with your target prompts and video paths.
Real talk

Our honest verdict

What's great

  • Requires zero training on video datasets to generate coherent clip outputs.
  • Supports multiple guidance modes including pose, edge, depth, and DreamBooth customization.
  • Integrated into Hugging Face Diffusers for simplified pipeline access.
  • Optimizations like Token Merging allow execution on GPUs with under 7 GB VRAM.

Where it falls short

  • Requires local CUDA hardware or paid cloud GPU instances to perform inference.
  • Repository lacks an explicit open-source license statement in its metadata.
  • Default video outputs are short frame sequences that require fine-tuning hyperparameter steps.
  • Requires technical familiarity with Python, virtual environments, and PyTorch configurations.
Cost

Pricing & license

The code is freely accessible on GitHub, but no explicit software license is specified in the repository. The primary cost comes from the GPU compute hardware required to execute inference locally or on cloud servers.

Paid work

Open remote AI-training roles

880 open remote AI-training roles, updated November 15, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.

See all open roles →
Compare

Alternatives

🎁 Free & open-source alternatives

These reviewed AI tools do the same job instantly — from our scored directory.

Get more

Use it, or have us set it up

📄 Free PDF: Text2Video-Zero for Video
A plain-language guide to getting your first result. Sent to your inbox.
🔔 64 people want a full video tutorial — add your vote
🛠 Don't want to set it up yourself?
We'll install, host & configure Text2Video-Zero for you — and train your team.
Get it done for you ↗
Questions

FAQ

What hardware is required to run Text2Video-Zero locally?

You need a CUDA-capable GPU with at least 12 GB VRAM by default, or less than 7 GB VRAM when using Token Merging optimizations.

Can I use custom Stable Diffusion weights or fine-tuned models?

Yes, you can load any Stable Diffusion base model or custom DreamBooth weights converted to Diffusers format.

How do I edit existing video footage with text prompts?

You can use the Video Instruct-Pix2Pix method by providing an input video file along with text instructions.

Radar

Latest AI news

✉️ The weekly free toolkit
The best new free & open-source tools for Video — one short email a week, in your language.
Free · 1 email/week · unsubscribe anytime
Get paid to train AI →

See Zekai first in Google