In this work, we introduce Janus-Pro, an advanced version of the previous work Janus. Specifically, Janus-Pro incorporates (1) an optimized training strategy, (2) expanded training data, and (3) scaling to larger model size. With these improvements, Janus-Pro achieves significant advancements in both multimodal understanding and text-to-image instruction-following capabilities, while also enhancing the stability of text-to-image generation. We hope this work will inspire further exploration in the field. Code and models are publicly available.
核心贡献 · Key contributions
提出 Janus-Pro，通过优化训练策略、扩展数据和模型规模扩张至 7B 实现改进。 Proposes Janus-Pro with optimized training strategy, expanded data, and model scaling to 7B.
解耦多模态理解与生成的视觉编码，解决任务冲突。 Decouples visual encoding for multimodal understanding and generation to resolve task conflicts.
引入更长的第一阶段 ImageNet 训练和聚焦的第二阶段文生图数据训练。 Introduces longer Stage I training on ImageNet and focused Stage II training on text-to-image data.
加入 7200 万合成美学数据，实现稳定且高质量的文生图生成。 Incorporates 72M synthetic aesthetic data for stable and high-quality text-to-image generation.
在 MMBench（79.2）和 GenEval（0.80）上达到最优，超越此前统一模型。 Achieves state-of-the-art on MMBench (79.2) and GenEval (0.80), surpassing prior unified models.
验证视觉编码解耦方法从 1.5B 到 7B 的强可扩展性。 Demonstrates strong scalability of visual encoding decoupling method from 1.5B to 7B.
局限 · Limitations
输入分辨率限制为 384×384，影响 OCR 等细粒度任务。 Input resolution limited to 384×384, affecting fine-grained tasks like OCR.
低分辨率与视觉分词器重建损失导致图像细节不足，如小面部区域。 Low resolution and vision tokenizer reconstruction loss cause under-detailed images, e.g., small faces.
训练需要大量算力：在 16/32 节点（每节点 8 块 A100 GPU）上耗时 9/14 天。 Training requires significant compute: 9/14 days on 16/32 nodes with 8 A100 GPUs each.
依赖合成美学数据，可能无法完全覆盖真实世界的多样性。 Relies on synthetic aesthetic data which may not fully capture real-world diversity.