自 OpenAI 发布 o1 以来，已经过去了近两年，o1 是一个普及了基于 LLM 的推理模型理念的模型。大约四个月后，DeepSeek-R1 紧随其后，并提供了使用可验证奖励的强化学习（RLVR）配方来训练此类推理模型的细节。上周，OpenAI 发布了 GPT-5.6 模型系列。该系列包含三种尺寸，每种尺寸大约有五六种推理努力设置。图 1：GPT 5.6 Sol 模型在不同推理努力设置下的表现。（Ultra 的基准数字目前尚不可用，但应与 Max 相对相似，因为它使用类似的努力水平，但通过四个子代理加速工作。）
It has been almost two years since OpenAI released o1, a model that popularized the idea of LLM-based reasoning models. DeepSeek-R1 followed about four months later, together with details of a reinforcement learning with verifiable rewards (RLVR) recipe to train such reasoning models. Last week, OpenAI released the GPT-5.6 model family. It comes in three sizes, each with roughly five or six reasoning-effort settings. Figure 1: The GPT 5.6 Sol model with different reasoning effort settings. (Benchmark numbers for Ultra are currently not available but should be relatively similar to Max, since it uses a similar effort level but accelerates the work with four subagents.)
核心贡献 · Key contributions
引入推理努力控制作为现代 LLM 的关键机制，允许调整推理算力。 Introduces reasoning effort control as a key mechanism in modern LLMs, enabling adjustable inference compute.
综述实现方法：基于努力条件的微调、带长度惩罚的 RLVR 以及聊天模板切换。 Surveys implementation methods: effort-conditioned SFT, RLVR with length penalties, and chat-template toggles.
分析六个开放权重模型（DeepSeek V4、Nemotron 3 Ultra、Kimi K2.5、GLM-5、Qwen3、Inkling）的努力训练。 Analyzes six open-weight models (DeepSeek V4, Nemotron 3 Ultra, Kimi K2.5, GLM-5, Qwen3, Inkling) for effort training.
展示训练和推理缩放作为互补轴，成本-性能曲线重叠。 Shows training and inference scaling as complementary axes, with overlapping cost-performance curves.
讨论自动努力选择作为未来方向，指出 GPT-5 Auto 模式被移除。 Discusses automatic effort selection as future direction, noting GPT-5 Auto mode's removal.
局限 · Limitations
GPT-5.6 等专有模型的实现细节未公开，限制验证。 Implementation details of proprietary models like GPT-5.6 are not disclosed, limiting verification.
努力水平显示收益递减；更高努力可能变得不经济。 Effort levels show diminishing returns; higher effort may become uneconomical.
方法因模型而异，由于基础检查点和数据不同，难以进行受控比较。 Methods vary across models, making controlled comparison difficult due to differing base checkpoints and data.
自动努力选择仍未解决；手动选择可能并非所有任务的最优解。 Automatic effort selection remains unsolved; manual selection may not be optimal for all tasks.
推理努力控制可能无法跨领域或任务迁移，如 Kimi K2.5 的 Toggle 所示。 Reasoning effort control may not transfer across domains or tasks, as seen in Kimi K2.5's Toggle.
论文章节 · Sections（共 28）
LLM 如何学习低、中、高努力推理模式How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes
1. 推理模型的简要定义1. A brief definition of reasoning models
2. 训练与推理扩展推理模型概述2. A brief overview of training and inference scaling reasoning models
2.1 训练推理模型2.1 Training reasoning models
2.2 “顿悟”时刻2.2 “Aha” moments
2.3 推理时的规模扩展简述2.3 Inference scaling in a nutshell
3. 思考令牌3. Think tokens
4. 推理模式的开关4. Reasoning mode on and off switches
5. “推理努力”设置如何工作5. How “reasoning effort” settings work
5.1 推理努力与响应长度和质量5.1 Reasoning effort and response length and quality
5.2 可能的努力水平实现方式5.2 Possible effort level implementations
5.3 Inkling 案例研究5.3 Inkling case study
5.4 关于推理扩展与训练扩展的简短说明5.4 A short note about inference scaling versus training scaling
6. 补充：在旗舰开源权重 LLM 中实现推理努力的不同方式6. Bonus: Different ways to implement reasoning efforts (in flagship open-weight LLMs)
6.1 DeepSeek V4 训练专门的努力程度专家6.1 DeepSeek V4 trains separate effort specialists
6.2 Nemotron 3 Ultra 结合学习模式与硬预算6.2 Nemotron 3 Ultra combines learned modes with hard budgets
6.2.1 推理时使用 Nemotron 推理预算6.2.1 Using Nemotron reasoning budgets during inference
6.2.2 Nemotron 中的推理预算感知训练6.2.2 Reasoning budget-aware training in Nemotron
6.3 Kimi K2.5 交替进行预算约束与无约束强化学习6.3 Kimi K2.5 alternates budgeted and unconstrained RL
6.3.1 Kimi 的 Toggle 方法6.3.1 Kimi’s Toggle method
6.3.2 Toggle 在推理时的变化6.3.2 What Toggle changes at inference
6.3.3 Kimi K3（更新）6.3.3 Kimi K3 (Update)
6.4 GLM-5 通过 SFT 引入回合级和交错思考6.4 GLM-5 introduces turn-level and interleaved thinking through SFT
6.5 Qwen3 使用模式融合与推理时截断6.5 Qwen3 uses mode fusion and inference-time truncation
6.6 Inkling 条件：连续努力值上的强化学习6.6 Inkling conditions RL on a continuous effort value