We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. We design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. We also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL shows drastically improved performance compared the previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators. In the spirit of promoting open research and fostering transparency in large model training and evaluation, we provide access to code and model weights at https://github.com/Stability-AI/generative-models
核心贡献 · Key contributions
提出 SDXL，采用 3 倍大的 UNet 骨干网络和双文本编码器，提升文本到图像合成性能。 Introduces SDXL with a 3x larger UNet backbone and dual text encoders for improved text-to-image synthesis.
提出基于原始图像尺寸和裁剪坐标的新型条件机制，减少裁剪伪影并改善物体居中。 Proposes novel conditioning on original image size and crop coordinates to reduce cropping artifacts and improve object centering.
实现多宽高比训练与尺寸条件，支持生成多种宽高比图像且不损失质量。 Implements multi-aspect ratio training with size conditioning, enabling generation at various aspect ratios without quality loss.
开发独立的精炼模型，利用 SDEdit 提升基础 SDXL 样本的视觉保真度。 Develops a separate refinement model using SDEdit to enhance visual fidelity of base SDXL samples.
在开源的同时达到与黑盒最先进图像生成器竞争的性能。 Achieves competitive performance with black-box state-of-the-art image generators while being open-source.
公开代码和模型权重，促进大型模型训练中的开放研究和透明度。 Provides code and model weights to promote open research and transparency in large model training.
局限 · Limitations
两阶段流水线需加载两个大模型，增加内存占用并降低采样速度。 Two-stage pipeline requires loading two large models, increasing memory usage and reducing sampling speed.
由于训练数据中高方差，合成人手等复杂结构时存在困难。 Struggles with synthesizing intricate structures like human hands due to high variance in training data.
可能出现概念渗漏，即不同视觉元素意外合并或交换属性。 May exhibit concept bleeding, where distinct visual elements unintentionally merge or swap attributes.
长文本或清晰文本的渲染仍有挑战，常产生随机字符或不一致。 Text rendering for long or legible text remains challenging, often producing random characters or inconsistencies.
训练数据偏差可能导致生成图像中的社会和种族偏见，需谨慎缓解。 Training data biases can lead to social and racial biases in generated images, requiring careful mitigation.
论文章节 · Sections（共 19）
摘要Abstract
1 引言1 Introduction
2 改进稳定扩散2 Improving Stable Diffusion
2.1 架构与规模2.1 Architecture & Scale
2.2 微条件化2.2 Micro-Conditioning
2.3 多宽高比训练2.3 Multi-Aspect Training
2.4 改进的自编码器2.4 Improved Autoencoder
2.5 整合所有改进2.5 Putting Everything Together
3 未来工作3 Future Work
附录 A 致谢Appendix A Acknowledgements
附录 B 局限性Appendix B Limitations
附录 C 扩散模型Appendix C Diffusion Models
附录 D 与现有最优方法的比较Appendix D Comparison to the State of the Art
E.1 总体投票E.1 Overall Votes
E.2 PartiPrompts (P2)上的类别与挑战比较E.2 Category & challenge comparisons on PartiPrompts (P2)
附录 F 关于生成式文本-图像基础模型的 FID 评估Appendix F On FID Assessment of Generative Text-Image Foundation Models
附录 G 单阶段与两阶段 SDXL 管线的额外比较Appendix G Additional Comparison between Single- and Two-Stage SDXL pipeline
附录 H SD 1.5、SD 2.1 与 SDXL 的比较Appendix H Comparison between SD 1.5 vs. SD 2.1 vs. SDXL
附录 I 多宽高比训练超参数Appendix I Multi-Aspect Training Hyperparameters