Diffusion probabilistic models (DPMs) are emerging powerful generative models. Despite their high-quality generation performance, DPMs still suffer from their slow sampling as they generally need hundreds or thousands of sequential function evaluations (steps) of large neural networks to draw a sample. Sampling from DPMs can be viewed alternatively as solving the corresponding diffusion ordinary differential equations (ODEs). In this work, we propose an exact formulation of the solution of diffusion ODEs. The formulation analytically computes the linear part of the solution, rather than leaving all terms to black-box ODE solvers as adopted in previous works. By applying change-of-variable, the solution can be equivalently simplified to an exponentially weighted integral of the neural network. Based on our formulation, we propose DPM-Solver, a fast dedicated high-order solver for diffusion ODEs with the convergence order guarantee. DPM-Solver is suitable for both discrete-time and continuous-time DPMs without any further training. Experimental results show that DPM-Solver can generate high-quality samples in only 10 to 20 function evaluations on various datasets. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset, and a $4\sim 16\times$ speedup compared with previous state-of-the-art training-free samplers on various datasets.
核心贡献 · Key contributions
提出扩散 ODE 解的精确公式，通过解析计算线性部分。 Proposes exact formulation of diffusion ODE solutions by analytically computing the linear part.
通过变量替换将解简化为神经网络的指数加权积分。 Simplifies solution to exponentially weighted integral of neural network via change-of-variable.
开发 DPM-Solver，一种具有收敛保证的高阶扩散 ODE 求解器。 Develops DPM-Solver, a high-order solver for diffusion ODEs with convergence guarantee.
在 10-20 次函数评估中实现高质量采样，比先前方法加速 4-16 倍。 Achieves high-quality sampling in 10-20 function evaluations, 4-16x speedup over prior methods.
适用于离散时间和连续时间 DPM，无需额外训练。 Applies to both discrete-time and continuous-time DPMs without additional training.
提供一阶、二阶和三阶求解器的理论收敛阶分析。 Provides theoretical convergence order analysis for first, second, and third-order solvers.
局限 · Limitations
不适用于加速 DPM 的似然评估。 Not suitable for accelerating likelihood evaluations of DPMs.
与 GAN 相比，仍不足以用于实时应用。 Still not fast enough for real-time applications compared to GANs.
可能被滥用于生成有害虚假内容，加速效果可能放大影响。 Potential misuse for generating adverse fake content, amplified by speedup.
仅考虑最高三阶求解器，更高阶留待未来研究。 Only considers solvers up to third order; higher orders left for future work.
论文章节 · Sections（共 19）
摘要Abstract
1 引言1 Introduction
2 扩散概率模型2 Diffusion Probabilistic Models
2.1 前向过程与扩散 SDE2.1 Forward Process and Diffusion SDEs