DeepSeek LLM：以长期主义扩展开源语言模型

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

深度求索 DeepSeek-AI · DeepSeek · 2024-01-05 · arXiv:2401.02954 ↗ · 被引 785

摘要 · Abstract

开源大型语言模型（LLM）的快速发展令人瞩目。然而，先前文献中描述的缩放定律呈现出不同的结论，给 LLM 的扩展蒙上了阴影。我们深入研究了缩放定律，并提出了独特的发现，这些发现有助于在两种常用的开源配置（7B 和 67B）中扩展大规模模型。在缩放定律的指导下，我们推出了 DeepSeek LLM，这是一个致力于以长期视角推进开源语言模型的项目。为了支持预训练阶段，我们开发了一个数据集，目前包含 2 万亿个 token，并且还在持续扩展。我们进一步在 DeepSeek LLM 基础模型上进行了监督微调（SFT）和直接偏好优化（DPO），从而创建了 DeepSeek Chat 模型。我们的评估结果表明，DeepSeek LLM 67B 在各种基准测试上超越了 LLaMA-2 70B，特别是在代码、数学和推理领域。此外，开放式评估显示，DeepSeek LLM 67B Chat 的性能优于 GPT-3.5。

The rapid development of open-source large language models (LLMs) has been truly remarkable. However, the scaling law described in previous literature presents varying conclusions, which casts a dark cloud over scaling LLMs. We delve into the study of scaling laws and present our distinctive findings that facilitate scaling of large scale models in two commonly used open-source configurations, 7B and 67B. Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source language models with a long-term perspective. To support the pre-training phase, we have developed a dataset that currently consists of 2 trillion tokens and is continuously expanding. We further conduct supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) on DeepSeek LLM Base models, resulting in the creation of DeepSeek Chat models. Our evaluation results demonstrate that DeepSeek LLM 67B surpasses LLaMA-2 70B on various benchmarks, particularly in the domains of code, mathematics, and reasoning. Furthermore, open-ended evaluations reveal that DeepSeek LLM 67B Chat exhibits superior performance compared to GPT-3.5.

核心贡献 · Key contributions

建立了超参数的缩放定律，为确定最优批大小和学习率提供了经验框架。
Established scaling laws for hyperparameters, providing an empirical framework for determining optimal batch size and learning rate.
引入非嵌入 FLOPs/词元作为模型规模表示，得到更精确的最优模型/数据分配策略。
Introduced non-embedding FLOPs/token as model scale representation, leading to more accurate optimal model/data allocation strategy.
发现预训练数据质量影响最优模型/数据缩放分配；质量越高越倾向于模型缩放。
Discovered that pre-training data quality influences optimal model/data scaling allocation; higher quality favors more model scaling.
在缩放定律指导下开发了 DeepSeek LLM 7B 和 67B 模型，在代码、数学和推理基准上超越 LLaMA-2 70B。
Developed DeepSeek LLM 7B and 67B models guided by scaling laws, surpassing LLaMA-2 70B on code, math, and reasoning benchmarks.
DeepSeek 67B Chat 在开放式生成中表现优异，在中英文评估中超越 GPT-3.5。
Achieved superior open-ended generation performance with DeepSeek 67B Chat, outperforming GPT-3.5 in Chinese and English evaluations.
通过全面评估展示了安全性和对齐性，包括在 Do-Not-Answer 数据集上获得 97.8 分。
Demonstrated safety and alignment through comprehensive evaluations, including Do-Not-Answer dataset with 97.8 score.

局限 · Limitations

预训练后缺乏持续知识更新；模型可能生成非事实信息。
Lack of ongoing knowledge updates after pre-training; model may generate non-factual information.
初始中文数据不全面，导致某些中文特定主题性能欠佳。
Initial Chinese data not exhaustive, leading to suboptimal performance on certain Chinese-specific topics.
模型在中文和英语以外的语言能力有限，应谨慎使用。
Model proficiency in languages other than Chinese and English remains limited and should be used cautiously.
倾向于产生幻觉和未经证实的建议，这是大语言模型的常见局限。
Tendency to produce hallucinations and unverified advice, a common limitation of LLMs.
较小模型（7B）难以理解系统提示，导致性能下降。
Smaller models (7B) struggle with system prompt understanding, causing performance degradation.

论文章节 · Sections（共 17）

摘要 Abstract
1 引言 1 Introduction
2.1 数据 2.1 Data
2.2 架构 2.2 Architecture
2.3 超参数 2.3 Hyperparameters
2.4 基础设施 2.4 Infrastructures
3 缩放定律 3 Scaling Laws
3.1 超参数的缩放定律 3.1 Scaling Laws for Hyperparameters
3.2 估计最优模型与数据缩放 3.2 Estimating Optimal Model and Data Scaling
3.3 不同数据下的缩放定律 3.3 Scaling Laws with Different Data
4 对齐 4 Alignment
5.1 公开基准评估 5.1 Public Benchmark Evaluation
5.2 开放式评估 5.2 Open-Ended Evaluation
5.3 留出评估 5.3 Held-Out Evaluation
5.4 安全评估 5.4 Safety Evaluation
5.5 讨论 5.5 Discussion
6 结论、局限与未来工作 6 Conclusion, Limitation, and Future Work

阅读逐段中英对照全文 →