We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades. For the vision component, we incorporate a dynamic tiling vision encoding strategy designed for processing high-resolution images with different aspect ratios. For the language component, we leverage DeepSeekMoE models with the Multi-head Latent Attention mechanism, which compresses Key-Value cache into latent vectors, to enable efficient inference and high throughput. Trained on an improved vision-language dataset, DeepSeek-VL2 demonstrates superior capabilities across various tasks, including but not limited to visual question answering, optical character recognition, document/table/chart understanding, and visual grounding. Our model series is composed of three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small and DeepSeek-VL2, with 1.0B, 2.8B and 4.5B activated parameters respectively. DeepSeek-VL2 achieves competitive or state-of-the-art performance with similar or fewer activated parameters compared to existing open-source dense and MoE-based models. Codes and pre-trained models are publicly accessible at https://github.com/deepseek-ai/DeepSeek-VL2.
核心贡献 · Key contributions
提出动态分块视觉编码策略，支持不同宽高比的高分辨率图像处理。 Introduces dynamic tiling vision encoding for high-resolution images with varying aspect ratios.
利用 DeepSeekMoE 和多头潜在注意力机制压缩 KV 缓存，实现高效推理。 Leverages DeepSeekMoE with Multi-head Latent Attention to compress KV cache for efficient inference.
在改进的视觉语言数据集上训练，在 VQA、OCR、文档理解和视觉定位等任务上表现优异。 Trains on improved vision-language dataset, achieving superior performance on VQA, OCR, document understanding, and visual grounding.
提供三种模型变体（激活参数 1.0B、2.8B、4.5B），平衡性能与效率。 Offers three model variants (1.0B, 2.8B, 4.5B activated parameters) balancing performance and efficiency.
以更少的激活参数达到与现有密集和 MoE 模型相当或最优的性能。 Achieves competitive or state-of-the-art results with fewer activated parameters than existing dense and MoE models.
开源代码和预训练模型，加速多模态研究。 Open-sources codes and pre-trained models to accelerate multimodal research.
局限 · Limitations
上下文窗口仅支持每轮对话少量图像，限制多图像交互。 Context window limited to few images per chat session, restricting multi-image interactions.
偶尔对模糊图像或未见物体处理困难，存在鲁棒性问题。 Occasional difficulties with blurry images or unseen objects, indicating robustness issues.
推理能力弱于视觉感知和识别，有待加强。 Reasoning capabilities weaker than visual perception and recognition, needing improvement.
处理超过两张图像时禁用动态分块，限制高分辨率处理能力。 Dynamic tiling disabled when processing more than two images, limiting high-resolution handling.
小模型（Tiny）在中文回答中存在语言混杂问题，需额外数据缓解。 Language mixing issue in smaller variant (Tiny) for Chinese responses, requiring additional data.
论文章节 · Sections（共 12）
摘要Abstract
1 引言1 Introduction
2 模型架构2 Model Architecture
3 数据构建3 Data Construction
3.1 视觉-语言对齐数据3.1 Vision-Language Alignment Data
3.2 视觉-语言预训练数据3.2 Vision-Language Pretraining Data
3.3 监督微调数据3.3 Supervised Fine-tuning Data
4.1 训练流程4.1 Training Pipelines
4.2 超参数与基础设施4.2 Hyperparameters and Infrastructures