GLM-4.5V 和 GLM-4.1V-Thinking:迈向具有可扩展强化学习的通用多模态推理

GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

唐杰 Jie Tang · Zhipu AI · 2025-07-01 · arXiv:2507.01006 ↗ · 被引 290

打开互动全文版(逐段中英对照 + 图/公式 + 论文问答)→

摘要 · Abstract

我们提出了 GLM-4.1V-Thinking、GLM-4.5V 和 GLM-4.6V 系列视觉语言模型(VLM),旨在推进通用多模态理解与推理。在本报告中,我们分享了以推理为中心的训练框架开发中的关键发现。我们首先通过大规模预训练开发了一个具有显著潜力的强大视觉基础模型,这可以说是最终性能的上限。然后,我们提出了课程采样强化学习(RLCS)以释放模型的全部潜力,从而在包括 STEM 问题解决、视频理解、内容识别、编码、定位、基于 GUI 的智能体以及长文档解读等多样化任务中实现全面的能力提升。在 42 个公开基准的综合评估中,GLM-4.5V 在几乎所有任务上均达到了同等规模开源模型的最先进性能,并在编码和 GUI 智能体等挑战性任务上展现出与 Gemini-2.5-Flash 等闭源模型竞争甚至更优的结果。同时,较小的 GLM-4.1V-9B-Thinking 仍然具有很强的竞争力——在 29 个基准上取得了优于更大模型 Qwen2.5-VL-72B 的结果。我们开源了 GLM-4.1V-9B-Thinking 和 GLM-4.5V。此外,我们推出了 GLM-4.6V 系列,这是具有原生工具使用和 128K 上下文窗口的开源多模态模型。简要概述请访问 https://z.ai/blog/glm-4.6v。代码、模型及更多信息发布在 https://github.com/zai-org/GLM-V。

We present GLM-4.1V-Thinking, GLM-4.5V, and GLM-4.6V, a family of vision-language models (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the reasoning-centric training framework. We first develop a capable vision foundation model with significant potential through large-scale pre-training, which arguably sets the upper bound for the final performance. We then propose Reinforcement Learning with Curriculum Sampling (RLCS) to unlock the full potential of the model, leading to comprehensive capability enhancement across a diverse range of tasks, including STEM problem solving, video understanding, content recognition, coding, grounding, GUI-based agents, and long document interpretation. In a comprehensive evaluation across 42 public benchmarks, GLM-4.5V achieves state-of-the-art performance on nearly all tasks among open-source models of similar size, and demonstrates competitive or even superior results compared to closed-source models such as Gemini-2.5-Flash on challenging tasks including Coding and GUI Agents. Meanwhile, the smaller GLM-4.1V-9B-Thinking remains highly competitive-achieving superior results to the much larger Qwen2.5-VL-72B on 29 benchmarks. We open-source both GLM-4.1V-9B-Thinking and GLM-4.5V. We further introduce the GLM-4.6V series, open-source multimodal models with native tool use and a 128K context window. A brief overview is available at https://z.ai/blog/glm-4.6v. Code, models and more information are released at https://github.com/zai-org/GLM-V.

核心贡献 · Key contributions

局限 · Limitations

论文章节 · Sections(共 20)

阅读逐段中英对照全文 →