我们推出了 Kimi K3，一个拥有 2.8 万亿参数的混合专家模型，具备 1040 亿激活参数、原生视觉能力和 100 万 token 的上下文窗口。Kimi K3 基于 Kimi Delta Attention 和 Attention Residuals 构建，这两项技术改善了序列长度和模型深度上的信息流动。结合每 token 有效激活 896 个路由专家中的 16 个的 Stable LatentMoE，以及经过优化的训练和数据方案，这些进展使得整体扩展效率相比 Kimi K2 提升了约 2.5 倍。后训练阶段的核心包括在通用、智能体和编码领域以及多种推理努力水平上的强化学习，从而实现了组合泛化和稳健的长程执行能力。在 2.8T 规模下，Kimi K3 得到了多个领域基础设施进展的支持：针对 KDA 的算法-系统协同设计、完全均衡的专家并行训练与高效内存管理、支持持续 rollout 和沙盒状态的百万 token 智能体强化学习，以及部署创新。大量评估表明，Kimi K3 在长程编码、智能体、知识、推理和视觉任务上均达到前沿水平。尽管其整体性能仍落后于最强大的专有模型，即 Claude Fable 5 和 GPT-5.6 Sol，但 Kimi K3 在评估套件中的表现持续优于其他开放和专有模型。我们发布完整的 Kimi K3 模型权重，以促进未来研究，并加速前沿智能的更广泛部署与应用。
We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.
核心贡献 · Key contributions
提出 Kimi K3，一个 2.8T 参数原生多模态 MoE 模型，激活参数 104B，上下文窗口为 1M token。 Introduces Kimi K3, a 2.8T-parameter native multimodal Mixture-of-Experts model with 104B activated parameters and a 1M-token context window.
KDA、Attention Residuals 与 Stable LatentMoE 改善序列、深度和宽度维度的信息流，相对 Kimi K2 带来约 2.5× 的 Scaling（规模扩张）效率提升。 KDA, Attention Residuals, and Stable LatentMoE improve information flow and yield about 2.5x scaling efficiency over Kimi K2.
后训练在通用、智能体式和编码领域以多种推理力度进行强化学习，并通过多教师同策略蒸馏整合策略。 Post-training applies reinforcement learning across general, agentic, and coding domains at multiple reasoning-effort levels, then consolidates policies via multi-teacher on-policy distillation.
在长程编码、智能体式、知识、推理和视觉任务上达到前沿模型水平，优于其他被测模型。 Achieves frontier-level performance on long-horizon coding, agentic, knowledge, reasoning, and vision tasks, beating other evaluated models.
开放完整模型权重，以支持研究、部署和更广泛的前沿智能应用。 Releases full model weights to support open research, deployment, and broader adoption of frontier intelligence.
局限 · Limitations
在作者评测套件中，整体性能仍落后于最强的专有模型 Claude Fable 5 和 GPT-5.6 Sol。 Overall performance still trails the strongest proprietary models, Claude Fable 5 and GPT-5.6 Sol, on the authors' benchmark suite.
基准结果多为自报，未必能完全迁移到独立或真实世界评测。 Benchmark results are largely self-reported and may not fully transfer to independent or real-world evaluations.
尽管激活参数为 104B，但 2.8T 总参数和相关部署要求可能限制可及性。 The 2.8T total parameter count and associated serving requirements may limit accessibility despite 104B active parameters.
百万 token 智能体式强化学习依赖可恢复沙箱和同地部署 rollout 等专门基础设施，可能难以复现。 Million-token agentic reinforcement learning depends on specialized infrastructure such as resumable sandboxes and co-located rollout systems, which may be difficult to reproduce.
2.5× 的 Scaling（规模扩张）效率提升依赖于特定架构、数据和训练方案，能否迁移到其他模型尚未证实。 The 2.5x scaling efficiency gain is tied to specific architecture, data, and training recipes; transfer to other models is unproven.
论文章节 · Sections（共 24）
AbstractAbstract
1 引言1 Introduction
2 模型架构2 Model Architecture
2.1 混合注意力2.1 Hybrid Attention
2.2 注意力残差2.2 Attention Residuals
2.3 Stable LatentMoE2.3 Stable LatentMoE
2.4 原生视觉2.4 Native Vision
2.5 逐头 Muon2.5 Per-Head Muon
3.1 预训练数据3.1 Pre-Training Data
3.2 缩放定律3.2 Scaling Law
3.3 训练方案3.3 Training Recipe
3.4 长上下文扩展3.4 Long-Context Extension
4.1 方法4.1 Method
4.2 强化学习任务合成与智能体环境4.2 RL Task Synthesis and Agentic Environments
5 基础设施5 Infrastructure
5.1 KDA 的算法-系统协同设计5.1 Algorithm-System Co-Design for KDA
5.2 3T 级预训练的基础设施5.2 Infra for 3T-class Pre-Training
5.3 面向 1M 智能体式强化学习（Agentic RL）的基础设施5.3 Infra for 1M Agentic RL