The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advancements in open-source models has been challenging. This paper introduces InternLM2, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context modeling, and open-ended subjective evaluations through innovative pre-training and optimization techniques. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types including text, code, and long-context data. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages, exhibiting remarkable performance on the 200k ``Needle-in-a-Haystack" test. InternLM2 is further aligned using Supervised Fine-Tuning (SFT) and a novel Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF) strategy that addresses conflicting human preferences and reward hacking. By releasing InternLM2 models in different training stages and model sizes, we provide the community with insights into the model's evolution.
核心贡献 · Key contributions
开源了 InternLM2 模型（1.8B、7B、20B），在 30 个基准和 6 个维度上表现优异。 Open-sourced InternLM2 models (1.8B, 7B, 20B) with superior performance across 30 benchmarks and 6 dimensions.
设计了 200k 上下文窗口，在“大海捞针”测试中近乎完美。 Designed with a 200k context window, achieving near-perfect needle-in-a-haystack performance.
提供了预训练、SFT 和 RLHF 阶段的全面数据准备指导。 Provided comprehensive data preparation guidance for pre-training, SFT, and RLHF stages.
引入了带有条件奖励模型的 COOL RLHF，以调和冲突偏好并缓解奖励破解。 Introduced COOL RLHF with conditional reward model to reconcile conflicting preferences and mitigate reward hacking.
发布了不同训练阶段（RLHF 前和 RLHF 后）的模型，供社区分析。 Released models at different training stages (pre-RLHF and post-RLHF) for community analysis.
开发了 InternEvo 框架，在 1024 个 GPU 上实现 53% 的 MFU，并支持长达 1M token 的上下文。 Developed InternEvo framework achieving high MFU (53% on 1024 GPUs) and supporting long contexts up to 1M tokens.
局限 · Limitations
评估仅限于基准测试；实际部署性能可能有所不同。 Evaluation limited to benchmarks; real-world deployment performance may vary.
长上下文能力仅测试到 200k token；更长的上下文未验证。 Long-context capability tested only up to 200k tokens; longer contexts not verified.
COOL RLHF 需要多轮在线数据收集，增加了标注成本。 COOL RLHF requires multiple rounds of online data collection, increasing annotation cost.
模型大小仅限于 1.8B、7B 和 20B；未探索更大规模。 Model sizes limited to 1.8B, 7B, and 20B; larger scales not explored.
数据污染分析可能无法完全消除基准泄露。 Data contamination analysis may not fully eliminate benchmark leakage.
论文章节 · Sections（共 19）
摘要Abstract
1 引言1 Introduction
2 基础设施2 Infrastructure
2.1 InternEvo2.1 InternEvo
2.2 模型结构2.2 Model Structure
3 预训练3 Pre-train
3.1 预训练数据3.1 Pre-training data
3.2 预训练设置3.2 Pre-training Settings
3.3 预训练阶段3.3 Pre-training Phases
4 对齐4 Alignment
4.1 监督微调4.1 Supervised Fine-Tuning
4.2 基于人类反馈的 COOL 强化学习4.2 COOL Reinforcement Learning from Human Feedback