用自身的一个劣化版本来引导扩散模型

Guiding a Diffusion Model with a Bad Version of Itself

泰罗·卡拉斯 Tero Karras · NVIDIA · 2024-06-04 · arXiv:2406.02507 ↗ · 被引 280

打开互动全文版(逐段中英对照 + 图/公式 + 论文问答)→

摘要 · Abstract

图像生成扩散模型的主要关注点是图像质量、结果多样性以及结果与给定条件(如类别标签或文本提示)的对齐程度。流行的无分类器引导方法使用无条件模型来引导条件模型,从而同时提高提示对齐度和图像质量,但代价是多样性降低。这些效果似乎内在纠缠,难以控制。我们有一个令人惊讶的发现:通过使用模型自身的一个更小、训练更少的版本(而非无条件模型)来引导生成,可以在不牺牲多样性的情况下独立控制图像质量。这显著提升了 ImageNet 上的生成效果,使用公开可用的网络,在 64x64 和 512x512 分辨率上分别创下了 1.01 和 1.25 的 FID 记录。此外,该方法也适用于无条件扩散模型,大幅提升了其质量。

The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.

核心贡献 · Key contributions

局限 · Limitations

论文章节 · Sections(共 10)

阅读逐段中英对照全文 →