Tutorial "Diffusion models: Acceleration and Distillation"

How do today’s fastest diffusion pipelines actually achieve near-real-time generation? In this talk, we’ll unpack the numerical heart of acceleration — from high-order ODE/SDE solvers and adaptive step schedules to feature/activation caching that cuts compute without sacrificing fidelity. We’ll then trace model-level speedups — knowledge distillation and consistency models — showing how they’re trained, why they work, and where they break. Finally, we’ll compare these methods head-to-head, profiling the properties that matter for acceleration — maximum speedup (step count and wall-clock), trainable-parameter budget, extra data/compute for distillation, robustness to guidance/conditioning and edits, quality at ultra-low step counts, memory/VRAM footprint, stability and mode coverage, and ease of integration — so you leave with a clear map of when each approach wins.

Autoregressive modeling currently dominates the field of language generation. Recently, however, diffusion models have emerged as a compelling alternative paradigm. This talk will examine the principal approaches to modeling language with diffusion-based methods and analyze the associated challenges and methodological subtleties involved in their training. We will trace the development of diffusion models for language, highlighting key milestones and research directions that have shaped their evolution. Finally, we will consider the prospective role of diffusion in the future of language modeling and discuss whether the long-standing dominance of autoregressive methods may ultimately come to an end.