Diffusion models are the main driver of progress in image and video synthesis, but suffer from slow inference speed. Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from many-shot to single-step inference, albeit at the cost of expensive and difficult optimization due to its reliance on a fixed pretrained DINOv2 discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models. This approach simplifies training and enhances performance, enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the performance of state-of-the-art text-to-image generators using only four unguided sampling steps. Moreover, we systematically investigate its scaling behavior and demonstrate LADD's effectiveness in various applications such as image editing and inpainting.
核心贡献 · Key contributions
提出潜在对抗扩散蒸馏（LADD），一种利用预训练潜在扩散模型生成特征的新型蒸馏方法。 Introduces Latent Adversarial Diffusion Distillation (LADD), a novel distillation method using generative features from pretrained latent diffusion models.
将 LADD 应用于 Stable Diffusion 3（8B）得到 SD3-Turbo，在四步无引导采样中达到最先进性能。 Applies LADD to Stable Diffusion 3 (8B) to obtain SD3-Turbo, achieving state-of-the-art performance in four unguided sampling steps.
系统研究 LADD 的缩放行为，表明学生模型大小是最具影响力的因素。 Systematically investigates scaling behavior of LADD, showing student model size as the most impactful factor.
展示 LADD 在图像编辑和修复任务中的多功能性，实现单步推理。 Demonstrates LADD's versatility in image editing and inpainting tasks with single-step inference.
通过在潜在空间中统一教师和判别器简化训练，消除解码到像素空间的需求。 Simplifies training by unifying teacher and discriminator in latent space, eliminating decoding to pixel space.
实现多宽高比高分辨率图像合成，训练稳定。 Achieves multi-aspect ratio high-resolution image synthesis with stable adversarial training.
局限 · Limitations
SD3-Turbo 相比教师 SD3 提示对齐降低，导致对象重复和空间提示问题。 SD3-Turbo shows reduced prompt alignment compared to teacher SD3, causing object duplication and spatial prompting issues.
图像编辑缺乏对图像和文本引导强度的控制，限制了灵活性。 Image editing lacks control over image and text guidance strengths, limiting flexibility.
模型在编辑中表现出刚性，过于贴近输入，难以进行大改动。 Model exhibits rigidity in editing, adhering too closely to input and making large changes challenging.
教师模型和数据质量的缩放收益在超过一定阈值后趋于平稳。 Scaling benefits plateau for teacher model and data quality beyond certain thresholds.
对抗训练仍需仔细调整噪声分布参数。 Adversarial training may still require careful tuning of noise distribution parameters.