A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm.
核心贡献 · Key contributions
受非平衡统计物理启发，提出扩散概率模型，同时实现灵活性与可处理性。 Introduces diffusion probabilistic models inspired by non-equilibrium statistical physics, achieving both flexibility and tractability.
定义前向扩散过程逐步破坏数据结构，并学习反向过程恢复结构。 Defines a forward diffusion process that gradually destroys data structure and learns a reverse process to restore it.
支持精确采样、对数似然评估，以及与其他分布相乘计算后验。 Enables exact sampling, log-likelihood evaluation, and multiplication with other distributions for posterior computation.
在 MNIST、CIFAR-10 和枯叶图像等多个数据集上展示高对数似然。 Demonstrates high log-likelihood on diverse datasets including MNIST, CIFAR-10, and dead leaves images.
为反向扩散过程提供熵界，并为训练提供对数似然下界。 Provides entropy bounds for the reverse diffusion process and a lower bound on log-likelihood for training.
发布开源参考实现以促进可复现性。 Releases open-source reference implementation for reproducibility.
局限 · Limitations
需要大量扩散步骤（数千步）以获得高质量样本，增加计算成本。 Requires many diffusion steps (thousands) for high-quality samples, increasing computational cost.
性能依赖于扩散调度和噪声类型（高斯或二项式）的选择。 Performance depends on the choice of diffusion schedule and noise type (Gaussian or binomial).
该方法假设目标分布平滑；高度多峰或不连续分布可能具有挑战性。 The method assumes smooth target distributions; highly multimodal or discontinuous distributions may be challenging.
训练需要仔细调整超参数，如步长和网络架构。 Training requires careful tuning of hyperparameters like step size and network architecture.
对于有限扩散步数，对数似然下界可能较松，影响模型评估。 The lower bound on log-likelihood may be loose for finite diffusion steps, affecting model evaluation.