In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.
核心贡献 · Key contributions
开发并发布了 Llama 2，一个包含从 7B 到 70B 参数的预训练和微调大语言模型集合。 Developed and released Llama 2, a collection of pretrained and fine-tuned LLMs from 7B to 70B parameters.
推出了针对对话优化的 Llama 2-Chat，在大多数基准测试中优于开源聊天模型。 Introduced Llama 2-Chat, optimized for dialogue, outperforming open-source chat models on most benchmarks.
提供了详细的微调和安全改进方法，包括基于人类反馈的强化学习（RLHF）和 Ghost Attention。 Provided detailed fine-tuning and safety improvement methodology, including RLHF and Ghost Attention.
展示了 Llama 2-Chat 70B 在人工评估中与 ChatGPT 等闭源模型具有竞争力。 Demonstrated that Llama 2-Chat 70B is competitive with closed-source models like ChatGPT on human evaluations.
发布了用于研究和商业用途的模型，并提供了负责任使用指南和代码示例。 Released models for research and commercial use, with responsible use guide and code examples.
分享了工具使用涌现和时间知识组织等新发现。 Shared novel observations such as tool usage emergence and temporal knowledge organization.
局限 · Limitations
人工评估仅限于 4k 个提示，未涵盖所有实际用例。 Human evaluations limited to 4k prompts, not covering all real-world use cases.
安全测试仅以英语进行，未覆盖所有场景。 Safety testing conducted only in English, not covering all scenarios.
预训练数据可能包含偏见和毒性，部署前需要进一步的安全调优。 Pretraining data may contain biases and toxicity, requiring further safety tuning before deployment.
Llama 2 70B 在编码基准测试上与 GPT-4 和 PaLM-2-L 相比仍有较大差距。 Llama 2 70B still has a significant gap compared to GPT-4 and PaLM-2-L on coding benchmarks.
如果不暴露于迭代更新的新样本分布，奖励模型准确性可能会下降。 Reward model accuracy may degrade if not exposed to new sample distribution from iterative updates.
论文章节 · Sections（共 22）
摘要Abstract
1 引言1 Introduction
2 预训练2 Pretraining
2.1 预训练数据2.1 Pretraining Data
2.2 训练细节2.2 Training Details
2.3 Llama 2 预训练模型评估2.3 Llama 2 Pretrained Model Evaluation
3 微调3 Fine-tuning
3.1 监督微调3.1 Supervised Fine-Tuning (SFT)
3.2 基于人类反馈的强化学习3.2 Reinforcement Learning with Human Feedback (RLHF)
3.3 多轮一致性的系统消息3.3 System Message for Multi-Turn Consistency
3.4 RLHF 结果3.4 RLHF Results
4 安全性4 Safety
4.1 预训练中的安全性4.1 Safety in Pretraining
4.2 安全性微调4.2 Safety Fine-Tuning
4.3 红队测试4.3 Red Teaming
4.4 Llama 2-Chat 安全性评估4.4 Safety Evaluation of Llama 2-Chat
5 讨论5 Discussion
5.1 经验与观察5.1 Learnings and Observations
5.2 局限性与伦理考量5.2 Limitations and Ethical Considerations