We present JanusFlow, a powerful framework that unifies image understanding and generation in a single model. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flow, a state-of-the-art method in generative modeling. Our key finding demonstrates that rectified flow can be straightforwardly trained within the large language model framework, eliminating the need for complex architectural modifications. To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training. Extensive experiments show that JanusFlow achieves comparable or superior performance to specialized models in their respective domains, while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models.
核心贡献 · Key contributions
提出 JanusFlow，一个统一框架，将自回归大语言模型与整流流结合，用于图像理解与生成。 Introduces JanusFlow, a unified framework integrating autoregressive LLMs with rectified flow for image understanding and generation.
证明整流流可在 LLM 架构内直接训练，无需复杂修改。 Demonstrates that rectified flow can be trained within LLM architecture without complex modifications.
提出解耦的视觉编码器用于理解和生成，防止任务干扰。 Proposes decoupled vision encoders for understanding and generation to prevent task interference.
引入表征对齐正则化以提升生成质量。 Introduces representation alignment regularization to improve generation quality.
在 1.3B 参数模型上，在多模态理解和文生图基准上达到最先进性能。 Achieves state-of-the-art performance on multimodal understanding and text-to-image generation benchmarks with a 1.3B model.
在标准基准上超越现有统一方法和多个专用模型。 Outperforms existing unified approaches and several specialized models across standard benchmarks.
局限 · Limitations
模型规模限于 1.3B 参数，未探索向更大模型扩展。 Model size limited to 1.3B parameters; scalability to larger models not explored.
需要三阶段训练，并仔细调整数据比例。 Requires three-stage training with careful data ratio tuning.
图像分辨率固定为 384x384，更高分辨率可能需要调整。 Image resolution fixed at 384x384; higher resolutions may require adaptation.
依赖预训练的 VAE 和 SigLIP 编码器，性能可能受其质量影响。 Relies on pre-trained VAE and SigLIP encoders; performance may depend on their quality.
评估限于英文基准，未评估多语言能力。 Evaluation limited to English-language benchmarks; multilingual capabilities not assessed.
论文章节 · Sections（共 16）
摘要Abstract
1 引言1 Introduction
2 相关工作2 Related Work
3 JanusFlow3 JanusFlow
3.1 背景3.1 Background
3.2 多模态理解与生成的统一框架3.2 A Unified Framework for Multimodal Understanding and Generation
3.3 训练方案3.3 Training Schemes
3.4 训练目标3.4 Training Objective
4 实验4 Experiments
4.1 实验设置与实现细节4.1 Experiment Setup and Implementation Details