Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through a large-scale study, we demonstrate the superior performance of this approach compared to established diffusion formulations for high-resolution text-to-image synthesis. Additionally, we present a novel transformer-based architecture for text-to-image generation that uses separate weights for the two modalities and enables a bidirectional flow of information between image and text tokens, improving text comprehension, typography, and human preference ratings. We demonstrate that this architecture follows predictable scaling trends and correlates lower validation loss to improved text-to-image synthesis as measured by various metrics and human evaluations. Our largest models outperform state-of-the-art models, and we will make our experimental data, code, and model weights publicly available.
核心贡献 · Key contributions
提出了新的噪声采样技术，用于修正流模型，偏向感知相关尺度。 Introduces new noise sampling techniques for rectified flow models, biasing towards perceptually relevant scales.
提出了一种新颖的基于 Transformer 的架构（MM-DiT），为图像和文本令牌使用独立权重，实现双向信息流。 Presents a novel transformer-based architecture (MM-DiT) with separate weights for image and text tokens, enabling bidirectional information flow.
展示了可预测的缩放趋势，以及较低的验证损失与改进的文本到图像合成之间的强相关性。 Demonstrates predictable scaling trends and strong correlation between lower validation loss and improved text-to-image synthesis.
进行了大规模系统研究，比较了修正流公式与现有扩散模型。 Conducts a large-scale systematic study comparing rectified flow formulations with established diffusion models.
在文本到图像合成中实现了最先进的性能，超越了开源和专有模型。 Achieves state-of-the-art performance in text-to-image synthesis, outperforming both open and proprietary models.
公开了实验数据、代码和模型权重，以促进可重复性。 Provides public release of experimental data, code, and model weights for reproducibility.
局限 · Limitations
研究仅限于高分辨率文本到图像合成；视频结果是初步的。 Study limited to high-resolution text-to-image synthesis; video results are preliminary.
缩放实验最多进行到 8B 参数；未观察到饱和，但可能在更大规模时出现。 Scaling experiments conducted up to 8B parameters; saturation not observed but may occur at larger scales.
依赖于预训练的自编码器和文本编码器；重建质量限制了最终图像质量。 Dependence on pretrained autoencoder and text encoders; reconstruction quality bounds final image quality.
来自 CogVLM 的合成标题可能引入偏差或限制概念覆盖。 Synthetic captions from CogVLM may introduce biases or limit concept coverage.
训练稳定性需要 QK 归一化；并非适用于所有设置的通用方法。 Training stability requires QK-normalization; not a universal recipe for all setups.
论文章节 · Sections（共 11）
摘要Abstract
1 引言1 Introduction
2 无模拟训练流模型2 Simulation-Free Training of Flows
3 流轨迹3 Flow Trajectories
3.1 针对 RF 模型的定制信噪比采样器3.1 Tailored SNR Samplers for RF models
4 文生图架构4 Text-to-Image Architecture
5.1 改进整流流5.1 Improving Rectified Flows
5.2 改进模态特定表示5.2 Improving Modality Specific Representations