We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.
核心贡献 · Key contributions
提出 MiMo-V2-Flash，一个 309B 总参数、15B 激活参数的 MoE 模型，用于快速推理和智能体任务。 Proposes MiMo-V2-Flash, a 309B MoE model with 15B active parameters for fast reasoning and agentic tasks.
引入混合注意力机制，交错滑动窗口注意力和全局注意力，窗口 128 词元、比例 5:1，KV 缓存减少 6 倍。 Introduces hybrid attention interleaving SWA and global attention with 128-token window and 5:1 ratio, reducing KV-cache by 6x.
开发多教师在线策略蒸馏（MOPD）用于高效后训练，通过词元级奖励整合领域专精教师。 Develops Multi-Teacher On-Policy Distillation (MOPD) for efficient post-training, integrating domain-specialized teachers via token-level rewards.
将多词元预测（MTP）重用作推测解码的草稿模型，实现最高 3.6 接受长度和 2.6 倍加速。 Repurposes Multi-Token Prediction (MTP) as a draft model for speculative decoding, achieving up to 3.6 acceptance length and 2.6x speedup.
在 SWE-Bench Verified（73.4%）和 SWE-Bench Multilingual（71.7%）上取得领先开源模型的最优结果。 Achieves state-of-the-art results on SWE-Bench Verified (73.4%) and SWE-Bench Multilingual (71.7%), leading open-source models.
开源模型权重和三层 MTP 权重，促进社区协作。 Open-sources model weights and three-layer MTP weights to foster community collaboration.
局限 · Limitations
与最强闭源模型（如 GPT-5-High）相比仍有性能差距。 Performance gap remains compared to strongest closed-weight models like GPT-5-High.
架构探索尚属初步，对设计权衡的分析有限。 Architectural exploration is preliminary with limited analysis of design trade-offs.
MOPD 迭代共进化算力未完全扩展，潜力未充分释放。 MOPD iterative co-evolution compute not fully scaled; potential not fully unlocked.
混合注意力发现基于经验，可能不泛化到其他规模或设置。 Hybrid attention findings are empirical and may not generalize to other scales or settings.
MTP 加速随批大小和任务熵变化，需针对部署进行最优调参。 MTP acceleration varies with batch size and task entropy; optimal tuning required per deployment.