Stability AI Generative Models — Reference Implementation for SDXL and SVD

Introduction

This repository is Stability AI's official reference implementation for their generative model family, including Stable Diffusion XL (SDXL), Stable Video Diffusion (SVD), and related architectures. It provides a modular, research-friendly codebase built on PyTorch that separates model components (encoders, denoisers, samplers) into composable pieces.

What Stability Generative Models Does

• Implements Stable Diffusion XL for high-resolution text-to-image generation
• Provides Stable Video Diffusion for image-to-video and text-to-video synthesis
• Includes modular denoiser, conditioner, and sampler components for custom pipelines
• Supports various sampling methods including DDPM, DDIM, Euler, and DPM-Solver
• Enables fine-tuning and LoRA training on custom datasets

Architecture Overview

The codebase follows a modular design using SGM (Stability Generative Models) primitives. A DiffusionEngine orchestrates the forward and sampling processes by composing a Denoiser network, one or more Conditioners (text, image, timestep), a Discretization scheme, and a Sampler. This separation allows mixing components freely for experimentation without rewriting the training or inference loop.

Self-Hosting & Configuration

• Requires Python 3.10+ and PyTorch 2.0+ with CUDA support
• Download model weights from Hugging Face Hub or Stability AI's model registry
• Configure generation parameters via YAML config files in the configs directory
• Use the provided Streamlit demos for interactive image and video generation
• Set up training with the sgm training scripts and custom dataset loaders

Key Features

• Modular architecture allowing independent swapping of samplers, conditioners, and denoisers
• Support for both latent diffusion and pixel-space diffusion approaches
• Video generation via temporal attention layers in Stable Video Diffusion
• Configurable refinement pipeline for SDXL (base + refiner two-stage generation)
• Research-oriented with clean separation between model logic and training infrastructure

Comparison with Similar Tools

• diffusers (Hugging Face) — user-friendly pipeline API; Stability's repo is the canonical research implementation
• AUTOMATIC1111 WebUI — end-user interface; this repo is the underlying model code
• ComfyUI — visual node editor for diffusion workflows; this provides the raw model components
• InvokeAI — polished creative tool; Stability's repo targets researchers and developers
• kohya-ss — training scripts; this repo includes both inference and the original training approach

FAQ

Q: What hardware is needed to run SDXL? A: SDXL base requires a GPU with at least 8GB VRAM for inference. Training requires significantly more (40GB+ recommended).

Q: Can I use these models commercially? A: Check the specific model license. SDXL uses a permissive CreativeML Open RAIL++-M license allowing commercial use with some restrictions.

Q: How does this differ from Hugging Face diffusers? A: This is the original research implementation. Hugging Face diffusers ports these models into their standardized pipeline format for easier deployment.

Q: Does it support training from scratch? A: Yes. The training scripts and configs allow both fine-tuning from pretrained checkpoints and training new models from scratch.

Sources

• https://github.com/Stability-AI/generative-models
• https://stability.ai/research