通过漂移的生成建模

Generative Modeling via Drifting

何恺明 Kaiming He · MIT · 2026-02-04 · arXiv:2602.04770 ↗ · 被引 67

打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→

摘要 · Abstract

生成建模可以表述为学习一个映射 f，使其前推分布与数据分布匹配。在推理时，前推行为可以迭代进行，例如在扩散和基于流的模型中。本文提出了一种称为漂移模型的新范式，它在训练过程中演化前推分布，并自然地允许单步推理。我们引入了一个漂移场，它控制样本移动，并在分布匹配时达到平衡。这导致了一个训练目标，允许神经网络优化器演化分布。在实验中，我们的单步生成器在 256×256 分辨率的 ImageNet 上取得了最先进的结果，潜在空间 FID 为 1.54，像素空间 FID 为 1.61。我们希望我们的工作为高质量单步生成开辟新的机会。

Generative modeling can be formulated as learning a mapping f such that its pushforward distribution matches the data distribution. The pushforward behavior can be carried out iteratively at inference time, for example in diffusion and flow-based models. In this paper, we propose a new paradigm called Drifting Models, which evolve the pushforward distribution during training and naturally admit one-step inference. We introduce a drifting field that governs the sample movement and achieves equilibrium when the distributions match. This leads to a training objective that allows the neural network optimizer to evolve the distribution. In experiments, our one-step generator achieves state-of-the-art results on ImageNet at 256 x 256 resolution, with an FID of 1.54 in latent space and 1.61 in pixel space. We hope that our work opens up new opportunities for high-quality one-step generation.

核心贡献 · Key contributions

• 提出漂移模型，一种新的生成范式，在训练期间演化前推分布，实现单步推理。
  Proposes Drifting Models, a new generative paradigm that evolves pushforward distribution during training, enabling one-step inference.
• 引入漂移场，控制样本移动，并在分布匹配时达到平衡。
  Introduces a drifting field that governs sample movement and achieves equilibrium when distributions match.
• 在 ImageNet 256x256 上，潜在空间 FID 达 1.54，像素空间 FID 达 1.61，均为单步生成最优。
  Achieves state-of-the-art one-step FID of 1.54 on ImageNet 256x256 in latent space and 1.61 in pixel space.
• 提供最小化漂移范数的训练目标，兼容 SGD 等标准优化器。
  Provides a training objective minimizing drift norm, compatible with standard optimizers like SGD.
• 扩展到特征空间和多尺度损失，提升高维数据生成质量。
  Extends to feature space and multi-scale losses, improving high-dimensional data generation.
• 在机器人控制任务中，单步漂移模型匹配 100 步扩散策略的性能。
  Demonstrates effectiveness on robotics control, matching 100-step diffusion policy with one step.

局限 · Limitations

• 零漂移意味着分布匹配的逆命题在理论上不保证成立。
  The converse implication (zero drift implies distribution match) is not theoretically guaranteed.
• 需要预训练特征编码器；无编码器时在 ImageNet 上失效。
  Requires a pre-trained feature encoder; method fails on ImageNet without one.
• 漂移场、核函数、架构等设计选择可能非最优，需进一步探索。
  Design choices (drifting field, kernel, architecture) may be suboptimal and need further exploration.
• 训练依赖正负样本批次采样，性能对批次大小敏感。
  Training relies on batch sampling of positives and negatives; performance sensitive to batch size.
• 对收敛到数据分布的条件缺乏充分理论分析。
  Limited theoretical analysis on conditions for convergence to data distribution.

论文章节 · Sections（共 15）

• 摘要 Abstract
• 1 引言 1 Introduction
• 2 相关工作 2 Related Work
• 3 生成中的漂移模型 3 Drifting Models for Generation
• 3.1 训练时的前推 3.1 Pushforward at Training Time
• 3.2 训练用的漂移场 3.2 Drifting Field for Training
• 3.3 设计漂移场 3.3 Designing the Drifting Field
• 3.4 特征空间中的漂移 3.4 Drifting in Feature Space
• 3.5 无分类器引导 3.5 Classifier-Free Guidance
• 4 图像生成的实现 4 Implementation for Image Generation
• 5.1 玩具实验 5.1 Toy Experiments
• 5.2 ImageNet 实验 5.2 ImageNet Experiments
• 5.3 机器人控制实验 5.3 Experiments on Robotic Control
• 6 讨论与结论 6 Discussion and Conclusion
• 致谢 Acknowledgements

阅读逐段中英对照全文 →