ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→我们介绍了 ChatGLM，这是一个不断演进的大型语言模型家族，我们一直在持续开发。本报告主要关注 GLM-4 语言系列，包括 GLM-4、GLM-4-Air 和 GLM-4-9B。它们代表了我们在前三代 ChatGLM 中积累的所有见解和经验基础上训练出的最强模型。迄今为止，GLM-4 模型在数万亿个 token 上进行了预训练，这些 token 主要来自中文和英文，以及来自 24 种语言的小规模语料库，并主要针对中文和英文使用进行了对齐。高质量的对齐通过多阶段后训练过程实现，包括监督微调和从人类反馈中学习。评估表明，GLM-4 1) 在 MMLU、GSM8K、MATH、BBH、GPQA 和 HumanEval 等通用指标上接近或超越 GPT-4，2) 在 IFEval 衡量的指令跟随方面接近 GPT-4-Turbo，3) 在长上下文任务上与 GPT-4 Turbo（128K）和 Claude 3 相当，4) 在 AlignBench 衡量的中文对齐上超越 GPT-4。GLM-4 All Tools 模型进一步对齐以理解用户意图，并自主决定何时以及使用哪种工具——包括网络浏览器、Python 解释器、文本到图像模型和用户自定义函数——以有效完成复杂任务。在实际应用中，它在通过网页浏览获取在线信息和利用 Python 解释器解决数学问题等任务上匹配甚至超越 GPT-4 All Tools。在此过程中，我们开源了一系列模型，包括 ChatGLM-6B（三代）、GLM-4-9B（128K、1M）、GLM-4V-9B、WebGLM 和 CodeGeeX，仅 2023 年一年就在 Hugging Face 上吸引了超过 1000 万次下载。开源模型可通过 https://github.com/THUDM 和 https://huggingface.co/THUDM 访问。
We introduce ChatGLM, an evolving family of large language models that we have been developing over time. This report primarily focuses on the GLM-4 language series, which includes GLM-4, GLM-4-Air, and GLM-4-9B. They represent our most capable models that are trained with all the insights and lessons gained from the preceding three generations of ChatGLM. To date, the GLM-4 models are pre-trained on ten trillions of tokens mostly in Chinese and English, along with a small set of corpus from 24 languages, and aligned primarily for Chinese and English usage. The high-quality alignment is achieved via a multi-stage post-training process, which involves supervised fine-tuning and learning from human feedback. Evaluations show that GLM-4 1) closely rivals or outperforms GPT-4 in terms of general metrics such as MMLU, GSM8K, MATH, BBH, GPQA, and HumanEval, 2) gets close to GPT-4-Turbo in instruction following as measured by IFEval, 3) matches GPT-4 Turbo (128K) and Claude 3 for long context tasks, and 4) outperforms GPT-4 in Chinese alignments as measured by AlignBench. The GLM-4 All Tools model is further aligned to understand user intent and autonomously decide when and which tool(s) touse -- including web browser, Python interpreter, text-to-image model, and user-defined functions -- to effectively complete complex tasks. In practical applications, it matches and even surpasses GPT-4 All Tools in tasks like accessing online information via web browsing and solving math problems using Python interpreter. Over the course, we have open-sourced a series of models, including ChatGLM-6B (three generations), GLM-4-9B (128K, 1M), GLM-4V-9B, WebGLM, and CodeGeeX, attracting over 10 million downloads on Hugging face in the year 2023 alone. The open models can be accessed through https://github.com/THUDM and https://huggingface.co/THUDM.