PyTorch implementation for text-to-360-degree panorama image generation using Stable Diffusion.
CVPR 2024 highlight implementing PanFusion for immersive panorama creation and evaluation.
PanFusion is a PyTorch-based framework that tames Stable Diffusion for text-to-360-degree panorama image generation. It suits researchers and developers working on generative AI, 3D environments, and spatial imaging.
The tool provides complete data preprocessing, training, testing, and evaluation code for generating 360-degree panoramas from text prompts. It includes modules for stitching Matterport3D skybox images, captioning with BLIP, rendering layouts, and training evaluation models like FAED and HorizonNet.
It suits machine learning researchers and computer vision engineers with access to multi-GPU hardware like NVIDIA A100s. It does not suit casual users or those without deep learning infrastructure and Python experience.
Tames diffusion models for generating 360-degree equirectangular panorama images.
Finetunes a ControlNet model for layout-conditioned panorama synthesis.
Uses Weights & Biases for logging, visualizing, and identifying training issues.
Includes training code for evaluation models to measure panorama and layout quality.
Free under its MIT license, with no hosted cloud options mentioned in the README. The only cost is running the hardware required for training and inference.
877 open remote AI-training roles, updated November 16, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.
Training the text-to-image generation model requires 4x NVIDIA A100 GPUs, while evaluation models like FAED or HorizonNet can run on a single NVIDIA A100.
The framework primarily uses the Matterport3D skybox dataset and Matterport3DLayoutAnnotation dataset.
You can train and use the included FAED model or HorizonNet checkpoints to evaluate image and layout quality.
See Zekai first in Google