We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most when building agents: sharp reasoning and fast, reliable execution. Step 3.5 Flash pairs a 196B-parameter foundation with 11B active parameters for efficient inference. It is optimized with interleaved 3:1 sliding-window/full attention and Multi-Token Prediction (MTP-3) to reduce the latency and cost of multi-round agentic interactions. To reach frontier-level intelligence, we design a scalable reinforcement learning framework that combines verifiable signals with preference feedback, while remaining stable under large-scale off-policy training, enabling consistent self-improvement across mathematics, code, and tool use. Step 3.5 Flash demonstrates strong performance across agent, coding, and math tasks, achieving 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6 (2024.08-2025.05), 88.2% on tau2-Bench, 69.0% on BrowseComp (with context management), and 51.0% on Terminal-Bench 2.0, comparable to frontier models such as GPT-5.2 xHigh and Gemini 3.0 Pro. By redefining the efficiency frontier, Step 3.5 Flash provides a high-density foundation for deploying sophisticated agents in real-world industrial environments.
核心贡献 · Key contributions
提出 Step 3.5 Flash，一种稀疏 MoE 模型，总参数量 196B，激活参数量 11B，实现高效的前沿级智能体智能。 Introduces Step 3.5 Flash, a sparse MoE model with 196B total and 11B active parameters for efficient frontier-level agentic intelligence.
提出 MIS-PO（Metropolis 独立采样过滤策略优化），用于 MoE 模型稳定的大规模离策略强化学习。 Proposes MIS-PO (Metropolis Independence Sampling-Filtered Policy Optimization) for stable large-scale off-policy RL in MoE models.
设计混合注意力机制，采用 3:1 滑动窗口/全局注意力比例并增加 SWA 查询头数，实现高效长上下文推理。 Designs a hybrid attention mechanism with 3:1 sliding-window/full attention and increased SWA query heads for efficient long-context inference.
开发统一的后训练方案，通过自蒸馏和可扩展强化学习交替进行领域特化与全局综合。 Develops a unified post-training recipe alternating domain-specific specialization and global synthesis via self-distillation and scalable RL.
仅用 11B 激活参数，在智能体、编程和数学基准上取得与 GPT-5.2 xHigh 等前沿模型相当的性能。 Achieves competitive performance on agent, coding, and math benchmarks with only 11B active parameters, comparable to frontier models like GPT-5.2 xHigh.
实现 EP 组平衡 MoE 路由和轻量级 MTP 头，降低多轮智能体交互的延迟和成本。 Implements EP-Group Balanced MoE Routing and lightweight MTP heads to reduce latency and cost in multi-round agentic interactions.
局限 · Limitations
评估限于基准测试；实际智能体部署可能暴露未捕获的额外失效模式。 Evaluation limited to benchmarks; real-world agentic deployment may reveal additional failure modes not captured.
混合注意力（含 SWA）在需要极长距离依赖的任务上可能仍不如密集注意力。 Hybrid attention with SWA may still underperform dense attention on tasks requiring very long-range dependencies.
训练稳定性依赖大量诊断和缓解措施；无法保证迁移到其他架构或规模。 Training stability relies on extensive diagnostics and mitigations; transferability to other architectures or scales is not guaranteed.
模型在非英语或低资源语言上的性能未得到充分评估。 The model's performance on non-English or low-resource languages is not thoroughly evaluated.
论文章节 · Sections（共 38）
摘要Abstract
1 引言1 Introduction
2.1 设计理念2.1 Design Philosophy
2.2 混合注意力的稀疏 MoE 主干2.2 Sparse MoE Backbone with Hybrid Attention