We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpassing a number of larger models. Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks. Pixtral uses a new vision encoder trained from scratch, which allows it to ingest images at their natural resolution and aspect ratio. This gives users flexibility on the number of tokens used to process an image. Pixtral is also able to process any number of images in its long context window of 128K tokens. Pixtral 12B substanially outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B). It also outperforms much larger open models like Llama-3.2 90B while being 7x smaller. We further contribute an open-source benchmark, MM-MT-Bench, for evaluating vision-language models in practical scenarios, and provide detailed analysis and code for standardized evaluation protocols for multimodal LLMs. Pixtral-12B is released under Apache 2.0 license.
核心贡献 · Key contributions
提出 Pixtral-12B，一个 120 亿参数的多模态模型，采用新型视觉编码器，支持原生分辨率和宽高比。 Introduces Pixtral-12B, a 12B-parameter multimodal model with a novel vision encoder supporting native resolution and aspect ratio.
在多模态基准上取得领先性能，超越 Llama-3.2 90B 和 Claude-3 Haiku 等更大模型。 Achieves leading performance on multimodal benchmarks, surpassing larger open and closed models like Llama-3.2 90B and Claude-3 Haiku.
保持强大的纯文本性能而不牺牲多模态能力，可作为两种任务的直接替代。 Maintains strong text-only performance without sacrificing multimodal capabilities, serving as a drop-in replacement for both tasks.
开源 MM-MT-Bench，一个实用的多模态基准，包含多轮对话，与 LMSys Vision ELO 高度相关。 Open-sources MM-MT-Bench, a practical multimodal benchmark with multi-turn conversations, correlating highly with LMSys Vision ELO.
提供标准化评估协议，包括显式提示和灵活解析，解决当前基准的不一致性。 Provides standardized evaluation protocols with explicit prompts and flexible parsing, addressing inconsistencies in current benchmarks.
在 Apache 2.0 许可下发布模型，包括推理和评估代码，确保可复现性。 Releases model under Apache 2.0 license, including inference and evaluation code for reproducibility.
局限 · Limitations
MM-MT-Bench 评估依赖 LLM 裁判，可能引入偏差或不一致性。 Evaluation relies on LLM judges for MM-MT-Bench, which may introduce bias or inconsistency.
视觉编码器消融仅与 CLIPA 比较，未涉及 SigLIP 等其他现代编码器。 Vision encoder ablation only compares with CLIPA, not other modern encoders like SigLIP.
在某些基准上的性能可能因提示设计而异，如显式与朴素提示的敏感性所示。 Performance on some benchmarks may vary with prompt design, as shown by sensitivity to explicit vs naive prompts.
模型大小（12B）可能限制在资源受限设备上的部署，相较于更小模型。 Model size (12B) may limit deployment on resource-constrained devices compared to smaller models.
MM-MT-Bench 仅覆盖五类图像，可能遗漏其他实际领域。 MM-MT-Bench covers only five image categories, potentially missing other practical domains.
论文章节 · Sections（共 8）
Pixtral 12BPixtral 12B
1 引言1 Introduction
2 架构细节2 Architectural details
3 MM-MT-Bench：多模态指令遵循基准3 MM-MT-Bench: A benchmark for multi-modal instruction following