We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices. This lets us identify several changes to both the sampling and training processes, as well as preconditioning of the score networks. Together, our improvements yield new state-of-the-art FID of 1.79 for CIFAR-10 in a class-conditional setting and 1.97 in an unconditional setting, with much faster sampling (35 network evaluations per image) than prior designs. To further demonstrate their modular nature, we show that our design changes dramatically improve both the efficiency and quality obtainable with pre-trained score networks from previous work, including improving the FID of a previously trained ImageNet-64 model from 2.07 to near-SOTA 1.55, and after re-training with our proposed improvements to a new SOTA of 1.36.
核心贡献 · Key contributions
提出了扩散生成模型的模块化设计空间，将采样、训练与 score 网络预处理选择解耦。 Presents a modular design space for diffusion-based generative models, decoupling sampling, training, and score-network preconditioning choices.
开发了改进的确定性采样器：采用 Heun 二阶积分器、优化噪声时间表和 sigma(t)=t，大幅减少网络评估次数。 Develops an improved deterministic sampler using a Heun 2nd-order integrator, optimized noise schedule, and sigma(t)=t, drastically reducing network evaluations.
提出定制的随机采样器，通过显式噪声搅动与启发式控制纠正早前采样误差，同时指出需经验调参。 Proposes a tailored stochastic sampler with explicit noise churn and heuristic controls, correcting earlier sampling errors while noting empirical tuning requirements.
为输入、输出、跳跃连接和噪声条件推导出有原理依据的预处理方案，并提出对数正态训练噪声分布与损失加权。 Derives principled preconditioning for inputs, outputs, skip connections, and noise conditioning, plus a log-normal training noise distribution and loss weighting.
取得 SOTA FID：CIFAR-10 条件 1.79、无条件 1.97，ImageNet-64 1.36；CIFAR-10 每图仅需 35 次网络评估。 Achieves state-of-the-art FID: 1.79 class-conditional and 1.97 unconditional CIFAR-10, and 1.36 ImageNet-64 with only 35 network evaluations for CIFAR-10.
局限 · Limitations
随机采样器依赖对每个模型和数据集逐个网格搜索的启发式参数，泛化受限。 The stochastic sampler relies on heuristic parameters that must be grid-searched per model and dataset, limiting generalization.
随机性并非普遍有益：改进的训练下 CIFAR-10 上反而有害，而 ImageNet-64 上仍有效，其与训练目标的交互仍未解决。 Stochasticity is not universally beneficial: with improved CIFAR-10 training it hurts performance, while it helps ImageNet-64, leaving its interaction with training unresolved.
许多提议的参数值可能需针对更高分辨率数据集与不同架构重新调整。 Many proposed parameter values likely need readjustment for higher-resolution datasets and different architectures.
随机采样中过度搅动会导致细节丢失与过饱和，部分归因于学得的 score 场非保守。 Excessive churn in stochastic sampling causes loss of detail and oversaturation, partly attributed to non-conservative learned score fields.
训练消耗约 250 MWh 电力；成果可能放大大规模部署中的虚假信息与偏见。 Training consumed about 250 MWh and advances may amplify misinformation and bias in large-scale deployment.
论文章节 · Sections（共 10）
摘要Abstract
1 引言1 Introduction
2 Expressing diffusion models in a common framework2 Expressing diffusion models in a common framework
3 Improvements to deterministic sampling3 Improvements to deterministic sampling