This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations.
核心贡献 · Key contributions
介绍基于 Gemini 研究的轻量级开放模型系列 Gemma。 Introduces Gemma, a family of lightweight open models based on Gemini research.
Gemma 在 18 项文本任务中的 11 项上优于同等规模开放模型。 Gemma outperforms similarly sized open models on 11 of 18 text tasks.
发布 2B 和 7B 参数模型的预训练和微调检查点。 Releases pretrained and fine-tuned checkpoints for 2B and 7B parameter models.
全面的安全与责任评估，并详细描述模型开发过程。 Comprehensive safety and responsibility evaluations with detailed model development.
在推理、数学和代码基准测试中表现强劲。 Achieves strong performance on reasoning, math, and code benchmarks.
强调负责任发布以提升前沿模型安全性并推动创新。 Emphasizes responsible release to improve frontier model safety and enable innovation.
局限 · Limitations
模型仅支持英语，不具备多模态能力。 Models are English-only and not multimodal.
测试无法覆盖所有应用场景，用户需自行进行安全测试。 Testing cannot cover all applications; users must conduct safety tests.
开放权重可能被滥用，尽管有禁止使用政策。 Open weights may enable misuse despite prohibited use policy.
存在低概率的个人数据记忆问题。 Memorization of personal data occurs at low rates.
在事实性、对齐和鲁棒性方面仍需进一步研究。 Further research needed for factuality, alignment, and robustness.
论文章节 · Sections（共 24）
摘要Abstract
1 引言1 Introduction
2 模型架构2 Model Architecture
3 训练基础设施3 Training Infrastructure
3.1 碳足迹3.1 Carbon Footprint
4.1 训练数据4.1 Training Data
4.2 过滤4.2 Filtering
5 指令微调5 Instruction Tuning
5.1 监督微调5.1 Supervised Fine-Tuning
5.2 过滤5.2 Filtering
5.3 格式化5.3 Formatting
5.4 基于人类反馈的强化学习5.4 Reinforcement Learning from Human Feedback