GPT-6 Astra, Looped Transformers, and Hidden Reasoning
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→本文考察了 OpenAI 的 GPT-6 Astra、围绕其采用循环 Transformer 或循环深度的传闻，以及该模型隐藏其推理轨迹的说法。文章首先概述了 Astra 的性能，指出它是一个极其强大的模型，在数学、编程，尤其是图形与计算机使用任务上处于领先地位，同时它仍是一个通过可验证奖励强化学习训练的推理模型。随后，文章详细解释了什么是循环 Transformer，将这一思想追溯至 Universal Transformers 论文，并以开放权重的 Nanbeige4.2-3B 模型为例加以说明——该模型将隐藏状态多次通过同一组共享权重的 Transformer 块。在此基础上，文章评估了循环深度是否真的会掩盖思维链，结论是架构本身并不会内在地隐藏推理，并在结尾强调了近期关于循环 Transformer 研究的新见解。
This article examines OpenAI's GPT-6 Astra, the rumors surrounding its use of looped transformers or recurrent depth, and claims that the model hides its reasoning traces. It first summarizes Astra's performance, noting that it is an exceptionally strong model that leads on math, coding, and especially graphical and computer-use tasks, while remaining a reasoning model trained with reinforcement learning from verifiable rewards. The article then explains in detail what looped transformers are, tracing the idea back to the Universal Transformers paper and illustrating it with the open-weight Nanbeige4.2-3B model, which passes hidden states through the same stack of transformer blocks multiple times with shared weights. On this basis, the article assesses whether recurrent depth actually obscures chains of thought, concluding that the architecture itself does not inherently hide reasoning, and it closes by highlighting new insights from recent research on looped transformers.
过去几周发生了许多事情。我确信 OpenAI 的 GPT-6 Astra 目前是所有人关注的焦点。尤其是关于其性能、循环 Transformer/循环深度方面的讨论，以及有关 Astra 正在“隐藏”其推理轨迹（即思维链）的传言。
A lot has happened in the last few weeks. I am sure that OpenAI's GPT-6 Astra is top of mind for everyone right now. In particular, thoughts on its performance, the looped transformer/recurrent depth aspects, and rumors that Astra is "hiding" its reasoning trace (i.e., chain of thought).
因此，在本文中，我想先谈谈对 Astra 的一些初步印象，以及对未来走向的一些思考。然后，我将详细讨论什么是“循环 Transformer”，以及这与隐藏思维链有何关系（或者说，是否有关）。
So, in this article, I want to start with some brief impressions of Astra and some thoughts on where all this is headed. Then, I will discuss, in detail, what "looped transformers" are, and how (or rather, if) this relates to hiding chains of thought.
最后，在介绍循环 Transformer 的基础知识之后，我想重点介绍近期研究论文中关于该主题的一些新见解。
Lastly, after covering the basics of the looped transformer, I wanted to highlight some new insights from recent research papers on the topic.
先说重点。在深入探讨架构传闻和相关研究文献之前，让我简要总结一些关于 GPT-6 Astra 的观察和花絮。
First things first. Before getting into the architecture rumors and related research literature, let me briefly summarize some GPT-6 Astra observations and tidbits.
上周，OpenAI 的新模型 GPT-6 Astra 大张旗鼓地发布了。过去几天我一直在使用它，它是一个异常出色的模型，很可能是截至本文写作时我用过的最好的模型。但确切地说，它改进了什么，又是如何改进的？
Last week, OpenAI's new GPT-6 Astra was released with a big fanfare. I used it over the past couple of days, and it's an exceptionally good model, likely the best I've used as of this writing. But what, exactly, has it improved, and how?
Astra 是我迄今为止用过的最好的模型，并且它在 3D 渲染和动画任务上表现格外突出（相对于其他模型而言）。我的意思是，尽管它在几乎所有类别（写作、数学、编程等）上都超越了其前代 GPT-5.6，但在图形演示方面尤其如此。
Astra is the best model I've used so far, and it's disproportionately good at 3D rendering and animation tasks (relative to other models). With that, I mean that while it leapfrogs its GPT-5.6 predecessor in practically all categories (writing, math, coding, and more), it especially does so when it comes to graphical demos.
我们也可以在基准测试中看到这一点。例如，GPT-6 Astra 在数学和编程方面非常出色，如下所示。
We can see this also reflected in the benchmarks. For instance, GPT-6 Astra is really good at math and coding, as shown below.
图 1：选取了三个流行的编程基准测试和一个具有挑战性的数学基准测试。更多基准测试在 Astra 发布博客上分享：https://openai.com/index/gpt-6-astra/
Figure 1: Selection of three popular coding benchmarks and one challenging math benchmark. More benchmarks are shared on the Astra release blog: https://openai.com/index/gpt-6-astra/
其中一个亮点（未在图中显示）是 Astra 在 ARC-AGI-3 基准测试上也达到了 99.9%（GPT-5.6 Sol 仅为 7.8%），该基准测试衡量的是解决逻辑谜题和泛化能力的混合表现。然而，数学、编程和计算机使用基准测试更有趣，因为它们更接近实际应用。
One of the highlights (not shown in the figure) is that Astra also achieves 99.9% on the ARC-AGI-3 benchmark (GPT-5.6 Sol only 7.8%), which measures a mix of solving logic puzzles and generalization. However, the math, coding, and computer use benchmarks are more interesting because they are closer to real-world use.
回到 Artificial Analysis 编程智能体指数 v1.4（前图中的右下角），它融合了多个智能体式编程任务，GPT-6 Astra 显然处于前沿，但并没有大幅领先。这也可以从下面显示的通用 Artificial Analysis 智能指数中看出，该指数融合了不同类型的任务，而不仅仅是编程任务。
Coming back to the Artificial Analysis Coding Agent Index v1.4 (lower right in the previous figure), which blends several agentic coding tasks, GPT-6 Astra is clearly at the frontier, but it doesn't pull ahead by leaps and bounds. This can also be seen in the general Artificial Analysis Intelligence Index shown below, which blends different types of tasks, not just coding tasks.
图 2：Artificial Analysis 智能指数，来源：https://artificialanalysis.ai/#intelligence
Figure 2: Artificial Analysis Intelligence Index via https://artificialanalysis.ai/#intelligence
如今，Artificial Analysis 基准测试的一大优势在于其独立性，因此可能比模型开发者自我评估的基准测试更值得信赖。
Now, the big advantage of Artificial Analysis benchmarks is that they are independent and thus may be a bit more trustworthy than self-evaluated benchmarks by model developers.
测试框架的设置取决于具体的基准测试。例如，GDPval-AA 和 AA-Briefcase 在比较不同 LLM 时，均使用其开源的极简 Stirrup 测试框架。在上文展示的 Intelligence Index v4.2 中，Terminal-Bench v2.1 使用 Terminus 2，而 τ³-Banking 使用 τ-Bench 测试框架。单独的 Coding Agent Index 也比较了不同的编码智能体测试框架。
The harness setup depends on the benchmark. For example, GDPval-AA and AA-Briefcase use their open-source, minimal Stirrup harness across the different LLMs they compare. In the Intelligence Index v4.2 shown above, Terminal-Bench v2.1 uses Terminus 2, and τ³-Banking uses the τ-Bench harness. The separate Coding Agent Index also compares different coding-agent harnesses.
对于使用共享测试框架的评估，这使其更接近同类比较。同时，在模型训练过程中，模型通常主要针对一个主要测试框架进行开发（而在其他测试框架上的微调较少）。此外，主要测试框架往往是为了适配并放大模型的优势而开发的。
For evaluations that use a shared harness, this makes it more of an apples-to-apples comparison. At the same time, during model training, models are typically developed with one primary harness in mind (and fine-tuned less on other harnesses). Plus, the primary harness is often developed to suit and amplify a model’s strengths.
因此，一些智能体式评估可能低估了 Astra 在其主要测试框架中的表现。这对某智能指数得分的影响程度，需要通过在同一任务上跨测试框架比较 Astra 来检验。
So, some of the agentic evaluations might underestimate how well Astra performs in its primary harness. How much this affects its Intelligence Index score would need to be tested by comparing Astra across harnesses on the same tasks.
顺带一提，正如一位同事最近向我建议的（Claude Code 负责人也如此推荐），删除（/归档）你现有的一些 AGENTS.md 内容和 SKILL.md 文件或许并不是个坏主意，因为较新的 LLM 在理解提示词和解决手头问题方面已变得更加高效。额外的“手把手”指导可能会不必要地约束较新的模型，并导致更差的解决方案。
As a side note, as a colleague recently suggested to me (as also recommended by the Claude Code lead), it's maybe not a bad idea to delete (/archive) some of your existing AGENTS.md contents and SKILL.md files, as newer LLMs have become more efficient at understanding the prompt and solving the problem at hand. The extra hand-holding could unnecessarily constrain newer models and lead to worse solutions.
当然，我并不是建议永远不再使用 SKILL.md 文件，而是对于某些工作流而言，它们可以在复用时提高效率，因为模型不必重新发现它们。但我想说的是，有些工作流并不需要描述，而“旧的”描述可能已不再理想，LLM 或许能够提出更好的解决方案。因此，也许是时候更新或重新生成这些指令文件了。
Of course, I am not suggesting never using SKILL.md files again, but for some workflows, because they can improve efficiency upon reuse, since the model doesn't have to rediscover them. But what I am suggesting is that some workflows don't need describing, and "old" descriptions may no longer be ideal, and the LLM may be able to come up with better solutions. So, it's perhaps time to update or regenerate said instruction files.
GPT-6 Astra 在图像和渲染任务上似乎异常强大。当这些任务涉及与图形用户界面交互时，它们还展现出计算机使用能力，即模型通过 Codex/ChatGPT 应用在本地计算机上操作软件。
GPT-6 Astra seems to be exceptionally strong in image and rendering tasks. When these tasks involve interacting with graphical user interfaces, they also demonstrate computer-use capabilities, meaning the model operates software on your local computer through the Codex/ChatGPT app.
计算机使用是该模型真正优于其他模型的地方，任何与图形相关的内容也能在社交媒体平台上产生有趣且直观的演示。从在 Blender 中建模渲染纽约市到虚拟开放参观，有大量令人印象深刻的演示示例。
Computer use is where the model really shines compared to others, and anything graphic-related also makes for interesting and intuitive demos on social media platforms. There are tons of examples of impressive demos out there, from modeling rendering New York City in Blender to virtual open house tours.
举一个例子，下面是一个对比，我让 GPT-6 Astra Medium 和 High 在浏览器版 MS Paint 中用鼠标在我的计算机上重画我的照片（不是 Extra High 和 Max，因为我不想浪费所有 token :))。
To pick one example, below is a comparison where I had GPT-6 Astra Medium and High redraw a picture of me in a browser version of MS Paint using the mouse on my computer (not Extra High and Max, because I didn't want to waste all my tokens :)).
这不仅突出了模型的艺术能力，更重要的是，它展示了在个人计算机上使用工具的能力（在此例中是 Paint；你可以看到模型通过鼠标光标使用界面）。
This highlights not only the model's artistic capabilities but, more importantly, its ability to use tools on one's computer (in this case, Paint; you can see the model using the interface via the mouse cursor).
这并不是第一个在 harness 内具备通用计算机使用能力的模型。例如，自今年早些时候以来，我已成功使用 GPT 模型完成一些 UI 任务（例如 Excel 中与费用相关的任务）等。然而，计算机使用是一项相对较新的能力，由 harness 实现，通常感觉还不够成熟。这很合理。LLM 是文本模型，因此自然更容易实现的是写作、编码以及使用 API 和 CLI。
This is not the first model that, inside a harness, is capable of general computer use. For example, I successfully used GPT models for some UI tasks (e.g., expense-related tasks in Excel) and so on since earlier this year. However, computer use is a relatively new capability, enabled by the harness, and usually feels not quite as mature yet. This makes sense. LLMs are text models, so naturally the lower-hanging fruit is writing and coding and using APIs and CLIs.
与此同时，许多工具和软件（目前）并未提供 CLI，与其等待有人设计出这样的接口，为什么不改进模型以使用图形用户界面呢（而且，如前所述，这无论如何都能做出漂亮且令人印象深刻的演示）？这有点类似于新兴的人形机器人发展。诚然，人形机器人并非最高效的机器人，例如在装配线上，那里已有专用机器。但它们用途广泛。
At the same time, there are many tools and software that don't expose CLIs (yet), and instead of waiting until someone designs that interface, why not improve models to use graphical user interfaces (and, as mentioned before, this makes for pretty and impressive demos, anyway)? This is somewhat analogous to the emerging humanoid robot developments. Sure, humanoid robots are not the most efficient robots, for example, at the assembly line, where special-purpose machines exist. But they are versatile.
因此，我预计未来几个月（或几年）也将是 LLM 和智能体框架层在计算机使用方面不断精进的时代。也就是说，除了当前的能力，并扩展其数学和编码能力之外，模型在训练时将会越来越多地考虑计算机使用。这也将使 LLM 在技术世界之外的日常计算机任务中更易于使用（“嘿 ChatGPT，请帮我报税” :))
So, I expect the upcoming months (or years) also to be an era of computer use refinement on both the LLM and the agent harness layer. I.e., in addition to the current capabilities, and expanding their math and coding capabilities, models will be trained with an increasing amount of computer use in mind. And this will also make LLMs more accessible for everyday computer tasks outside the tech world ("Hey ChatGPT, please do my tax return" :))
计算机使用趋势也与近期报道一致，即 OpenAI 购买了数万台 Mac Mini 和 Mac Studio 用于强化学习。因此，这里的 Mac 并非字面意义上用于训练模型（训练模型最好使用 GPU），而是在模型训练期间暴露 macOS，让模型学习使用该操作系统及其中的工具。
The computer usage trend is also consistent with the recent reporting that OpenAI purchased tens of thousands of Mac Minis and Mac Studios for Reinforcement Learning. So, here the Macs are not used to literally train the models (it's better to use GPUs for that) but rather to expose macOS during the model training for the model to learn to use said operating system and the tools therein.
那么，在这些 Mac 上的计算机使用训练是如何进行的？简而言之，Mac（或更准确地说，其 macOS 操作系统）充当模型在训练期间可以交互的环境。
So, how does computer-use training on said Macs work? In short, the Macs (or their macOS operating system, to be precise) serve as an environment that the model can interact with during training.
1. 通过给模型一个任务来提示它，例如“打开应用 xyz 并执行 abc”。
1. Prompt the model by giving it a task, such as "open an app xyz and do abc".
2. 向它提供 macOS 界面的截图（这通常由测试框架完成）。
2. Provide it with screenshots of the macOS interface (this is usually done by the harness).
3. 然后 LLM 预测鼠标/键盘操作（点击、按键、滚动等）。
3. The LLM then predicts mouse/keyboard actions (click, key presses, scrolling, and so on).
4. 在 Mac 上执行这些动作（同样，这由 harness 完成）。
4. Execute those actions on the Mac (again, this is done by the harness).
5. 在执行上一步的动作后，输入更新后环境的新截图。
5. Feed new screenshots of the updated environment after performing the actions in the previous step.
6. 重复步骤 2-5，直到任务成功或失败。
6. Repeat steps 2-5 until the task succeeds or fails.
7. 将成功/失败信号与验证器（或评分器）用作训练反馈，包括后训练期间的强化学习；这类似于常规的带可验证奖励的强化学习（RLVR）。
7. Use success/failure signals and verifiers (or graders) as training feedback, including reinforcement learning during post-training; this is analogous to regular Reinforcement Learning with Verifiable Rewards (RLVR).
图 3：计算机使用训练工作流概览。
Figure 3: Overview of a computer-use training workflow.
同样，Mac 在这里主要充当环境，而不是在训练期间运行或更新模型的机器。模型很可能部署在 NVIDIA GPU 上，并通过 API 提供给上述 Mac。顺便提一下，NVIDIA 的 CEO 提到 GPT-6 Astra 正在约 100,000 块 Grace Blackwell GPU 上进行训练。
Again, the Mac is mostly the environment here and not the machine for running or updating the model during training. The model likely sits on NVIDIA GPUs and is fed via API to said Mac. By the way, NVIDIA's CEO mentioned that GPT-6 Astra was being trained on ~100,000 Grace Blackwell GPUs.
上一节讨论的以计算机使用训练为重点，并不是训练流程中的根本性范式转变。GPT-6 Astra（以及可预见的未来任何 LLM）仍然是一个推理模型。这意味着该 LLM 使用可验证奖励的强化学习（RLVR）进行训练，并产生中间推理轨迹（思维链）。
The focus on computer-use training discussed in the previous section is not a fundamental paradigm shift in the training pipeline. GPT-6 Astra (and likely any LLM in the foreseeable future) is still a reasoning model. This means the LLM is trained with reinforcement learning with verifiable rewards (RLVR) and produces intermediate reasoning traces (chains of thought).
但我将在本文稍后部分讨论 GPT-6 Astra 的推理模型方面（尤其是关于隐藏思维链的问题）。
But I will discuss the reasoning model aspects of GPT-6 Astra (especially regarding hiding chains of thought) a bit later in this article.
话虽如此，在官方模型发布大约两天前，新闻杂志《The Information》发表了一篇文章，报道称根据一些内部信息，Astra 正在使用一种称为“循环深度”或“循环 Transformer”的概念。
That being said, about two days before the official model, the news magazine The Information published an article reporting that, according to some inside information, Astra is using a concept called "recurrent depth" or "looped transformers."
图 4：引自《The Information》（来源：https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns）
Figure 4: Quote from The Information (Source: https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns)
由于 LLM 架构属于我的专业领域和热情所在，我制作了一个简短的讲座视频，解释一般的循环 Transformer 机制，并回应关于隐藏推理链的评论，你可以在下面找到。
Since LLM architectures are within my area of expertise and my passion, I created a short lecture video explaining the general looped transformer mechanism and addressing the comment about hidden reasoning chains, which you can find below.
在接下来的小节中，我将首先解释什么是循环 Transformer，并将在本文后面重新讨论关于隐藏思维链的评论。
In the following subsections, I'll first explain what looped transformers are, and I'll revisit the comment about the hidden chains of thought later in this article.
（循环 Transformer 的解释可能看起来有点长，但我真的认为它有助于建立对该技术的基础理解，这对于判断其掩盖推理轨迹或思维链的说法很有用。）
(The looped transformer explanation may seem a bit long, but I really think that it helps with establishing a foundational understanding of the technique, which is then useful to judging the claim that it obscures the reasoning traces or chains of thought.)
循环 Transformer 本质上是一种架构上的微调，其主要思想是让中间表示多次通过相同的 Transformer 块（而不是仅一次）。与单纯增加更多块相比，这里的“技巧”在于这些传递过程中权重保持不变。
A Looped Transformer is essentially an architectural tweak, with the main idea being to pass the intermediate representations through the same transformer blocks multiple times (instead of just once). Compared to just adding more blocks, the “trick” here is that the weights stay the same across these passes.
在本文中，我将使用以下术语：
Throughout this article, I’ll use the following terms:
* Transformer 块是一个包含注意力机制、前馈模块、归一化和捷径连接的单元。这些块在论文中通常被称为“Transformer 层”。
* A transformer block is a unit containing attention, a feedforward module, normalization, and shortcut connections. These blocks are often called “transformer layers” in papers.
* 堆叠是一系列 Transformer 块的序列。
* A stack is a sequence of transformer blocks.
* 块应用是指将输入通过一个 Transformer 块运行一次。
* A block application means running an input through a transformer block once.
循环 Transformer 并非新事物，其基本思想早在 2018 年的 Universal Transformers 论文中就已出现。但在讨论 Universal Transformer 之前，让我们从一个更简单的例子开始：Nanbeige4.2-3B，这是一个近期发布的开放权重 LLM，于 7 月问世，我今年夏天早些时候曾在 Substack Notes 和我的 LLM 架构画廊中介绍过它。
The looped transformer is nothing new, and the basic idea already appeared in the Universal Transformers paper from 2018. But before discussing Universal Transformer, let's start with a simpler example, Nanbeige4.2-3B, a recent open-weight LLM that came out in July and that I covered on Substack Notes and in my LLM Architecture Gallery earlier this summer.
如下图所示，Nanbeige 架构本质上看起来像一个常规的 Transformer。然而，请注意它有一个额外的（橙色）箭头循环回 Transformer 堆栈的起始处。
The Nanbeige architecture, shown below, essentially looks like a regular transformer. However, notice that it has an extra (orange) arrow looping back to the beginning of the transformer stack.
图 5. Nanbeige4.2-3B 将相同的 22 个 Transformer 块堆栈应用两次。橙色箭头显示了中间表示被传回堆栈的位置。
Figure 5. Nanbeige4.2-3B applies the same stack of 22 transformer blocks twice. The orange arrow shows where the intermediate representations are passed back into the stack.
让我们自下而上地梳理这个过程。首先，与任何其他基于 Transformer 的 LLM 一样，输入文本被分词并转换为嵌入向量。这些向量随后通过 22 个 Transformer 块，而这 22 个块中的每一个都有自己独立的权重。
Let's walk through this from the bottom up. First, as in any other transformer-based LLM, the input text is tokenized and converted into embedding vectors. These vectors then pass through 22 transformer blocks, and each of these 22 blocks has its own weights.
然而，这里的循环 Transformer 特性在于，在第一次传递之后，隐藏状态会再次通过相同的 22 个块。因此，块 1 被再次应用，接着是块 2，依此类推，直到块 22。
However, the looping transformer aspect here is that after the first pass, the hidden states are fed back through the same 22 blocks. So, block 1 is applied again, followed by block 2, and so on up to block 22.
如果我们将这一计算展开，就会得到 44 次 Transformer 块应用。然而，与拥有 44 个不同块的传统 Transformer 相比，第二组 22 次块应用复用了第一组的权重。例如，第 23 次块应用使用第 1 个块的权重，第 24 次块应用使用第 2 个块的权重，依此类推。
If we were to unroll this computation, we would have 44 transformer block applications. However, compared to a conventional transformer with 44 distinct blocks, the second stack of 22 block applications reuses the weights from the first stack. For example, block application 23 uses the weights of block 1, block application 24 uses the weights of block 2, and so on.
图 6. Nanbeige4.2-3B 展开为对同一 22 个 Transformer 块的两轮处理，共得到 44 次块应用。
Figure 6. Nanbeige4.2-3B unrolled into two passes through the same 22 transformer blocks, giving 44 block applications.
所以，这里的核心思路是：我们在不额外增加一组 Transformer 权重的情况下，将有效深度从 22 次块应用提升到 44 次块应用。
So, the whole idea here is that we increase the effective depth from 22 to 44 block applications without adding another set of transformer weights.
顺便问一下，为什么是 2 轮，而不是 3 轮、4 轮或更多？Nanbeige 论文中没有给出太多细节，但他们表示这基本上是最高效的配置。将循环次数从 2 增加到 3 可以提升建模性能，但额外的计算成本并不值得。
By the way, why 2 rounds, not 3, 4, or more? There are not many details in the Nanbeige paper, but they say that this was essentially the most efficient setup. Increasing the loops from 2 to 3 can increase modeling performance, but the extra computational cost wasn't worth it.
那么，我们为什么要普遍采用这种循环呢？这本质上是一种替代方案，用以取代通过增加更多 Transformer 块来扩大模型规模的做法。
So, why would we do this looping in general? This is essentially an alternative to just making the model bigger by adding more transformer blocks.
例如，一个将 22 个 Transformer 块使用两次的模型，其（Transformer 块）参数量大约是具有 44 个常规块的模型的一半。
So, for instance, a model that uses 22 transformer blocks twice has roughly half as many (transformer-block) parameters compared to a model with 44 conventional blocks.
这进而减少了存储权重所需的内存。顺便提一下，嵌入层和输出层通常很大，占总参数量的很大一部分，但它们不在此比较范围内。（在 Nanbeige 4.2 3B 的情况下，嵌入层和输出层约占 30 亿总参数量的 25%；如果在这两层之间共享权重，我们可以将其减少到 12.5%。）
This then reduces the memory needed to store the weights. As a side note, note that the embedding and output layers, which are usually large and make up a substantial portion of the total, are separate from this comparison. (In the case of Nanbeige 4.2 3B, the embedding and output layers make up ~25% of the total 3B parameters; with weight sharing between those two, we could reduce that to 12.5%.)
图 7：并排比较，展示传统场景与循环场景下所需的参数量。
Figure 7: Side-by-side comparison showing how many parameters would be required in traditional versus looping scenarios.
当然，在循环中重用相同的块仍然需要计算。更准确地说，在前向传播过程中，我们将中间输入通过 44 次块应用。并且，在训练期间，梯度会通过共享堆栈的两次重复进行反向传播。因此，与仅使用一次 22 个块相比，这增加了大量工作。实际上，其成本与使用 44 个不同的块相似（除了优化器需要更新的不同参数更少；反向传播仍然会运行所有 44 次块应用）。
Of course, reusing the same blocks in a loop still requires computation. More precisely, we pass the intermediate inputs through 44 block applications during the forward pass. And, during training, gradients flow backward through both repetitions of the shared stack. So, compared to using the 22 blocks only once, this adds substantial work. Actually, it’s similarly expensive as having 44 distinct blocks (except the optimizer has fewer distinct parameters to update; backprop still runs through all 44 block applications).
此外还有 KV 缓存，它存储先前 token 的注意力键和值，以便在常规和循环 Transformer 的每个下一个词生成步骤中复用。顺便说一句，如果对你有用，我这里有一篇关于 KV 缓存的独立文章：
There is also the KV cache, which stores the attention keys and values of previous tokens for reuse in conventional and looped transformers in each next-token generation step. By the way, I have a standalone article on KV caching here if useful:
现在，让我们回到通用 Transformer。在 Nanbeige 中，我们将 22 个 transformer 块组成的堆叠应用两次。在 2018 年的通用 Transformer 论文中，我们重复应用同一个 transformer 块，而不是重复一个 transformer 块堆叠。不过，主要思想是相似的。
Now, let's come back to Universal Transformers. In Nanbeige, we apply a stack of 22 transformer blocks twice. In the Universal Transformer paper from 2018, we repeatedly apply the same transformer block instead of repeating a stack of transformer blocks. The main idea is similar, though.
此外，步数可以是固定的，但论文也探索了自适应停止。例如，特定位置的一个 token 可能只经过一两次循环。另一个可能经过三四次循环，依此类推。这赋予了模型灵活性，可以将算力分配给那些能从额外计算中受益的 token。
Also, the number of steps can be fixed, but the paper also explores adaptive halting. For example, a token at a particular position may only go through one or two loops. Another may go through three or four loops, and so on. This gives the model flexibility to allocate the compute to those tokens that benefit from extra computation.
循环次数是如何决定的？这里，模型使用一个小型的、训练过的函数，为每个步骤的每个位置输出所谓的停止概率。它将这些概率在连续的循环中累加，然后一旦总和超过阈值，就在给定位置停止循环。此外，最大循环次数也限制了计算，以防万一。
How is the looping number decided? Here, the model uses a small, trained function that outputs a so-called halting probability for each position at each step. It adds up these probabilities over these successive loops and then stops looping at a given position once the sum exceeds a threshold value. In addition, a maximum loop count also limits the computation just in case.
图 8：通用 Transformer 中的自适应停止。
Figure 8: Adaptive halting in a Universal Transformer.
另一个循环 transformer 的例子是字节跳动的 Ouro，我在我的 LLM 架构画廊中也介绍过。例如，Ouro-Thinking 2.6B 将相同的 48 个 transformer 块堆叠应用四次。那就是 192 次块应用，同时存储 48 个不同块的权重。基本上，这比 Nanbeige 更极端。此外，一个学习到的退出门为不同的出口分配概率，累积概率的阈值决定哪一次传递提供输出。所以，它也借鉴了通用 Transformer 的自适应停止思想，而 Nanbeige 没有使用。（然而，这里有一个实际注意事项。发布的 Hugging Face 实现会在选择输出之前计算所有配置的传递，因此循环次数似乎实际上被硬编码为 4）。
Another example of a looped transformer is ByteDance's Ouro, which I also covered in my LLM Architecture Gallery. For instance, Ouro-Thinking 2.6B applies the same stack of 48 transformer blocks four times. That's 192 block applications while storing weights for 48 distinct blocks. Basically, that's a more extreme case than Nanbeige. Additionally, a learned exit gate assigns probabilities to the different exits, and a threshold on the cumulative probability determines which pass supplies the output. So, it's also borrowing the adaptive halting idea from Universal Transformer, which Nanbeige didn't use. (However, there is a practical caveat here. The released Hugging Face implementation computes all configured passes before selecting an output, so it seems like the number of loops is effectively hard-coded to 4).
另一种方法是 Mixture-of-Recursions，这是 2025 年的一篇论文，本质上是对之前讨论的 Universal Transformer 的更复杂版本。与 Universal Transformer 类似，单个 token 会一次或多次通过 transformer 块，如下图所示。然而，其创新之处在于如何基于每个 token 确定这个循环次数。
Another approach is Mixture-of-Recursions, a paper from 2025 that is essentially a more sophisticated version of the Universal Transformer discussed earlier. Similar to the Universal Transformer, individual tokens pass the transformer blocks one or more times as illustrated in the figure below. However, the innovation is how this looping number is determined on a per-token basis.
在论文的以下图中，循环（重复）堆叠在这里被称为递归块。它包含几个 transformer 块，位于单独的第一个和最后一个 transformer 块（标记为 Layer 0 和 Layer L-1）之间。
In the following figure from the paper, the looped (repeated) stack is called a recursion block here. This contains several transformer blocks, and it sits between separate first and last transformer blocks (labeled Layer 0 and Layer L-1).
图 9. Mixture-of-Recursions 在不同 token 位置对共享堆叠应用不同的次数。高亮文本显示了 1、2 或 3 次传递的示例。图改编自 Mixture-of-Recursions 论文。
Figure 9. Mixture-of-Recursions applies a shared stack a different number of times at different token positions. The highlighted text shows an example with 1, 2, or 3 passes. Figure adapted from the Mixture-of-Recursions paper.
模型如何决定一个 token 应该通过递归块多少次？在之前讨论的 Universal Transformer 中，这是基于每一步学习到的停止概率。这里的 Mixture-of-Recursion 方法使用一个小型的学习路由器。这类似于混合专家模型中的路由思想，不同之处在于这里的路由决策决定了应用共享堆叠的次数。
How does the model decide how many times a token should go through the recursion block? In the previously discussed Universal Transformer, it’s based on a learned halting probability at each step. This Mixture-of-Recursion approach here uses a small, learned router. This is similar to the routing idea in a mixture-of-experts model, except that here the routing decision determines how many times to apply the shared stack.
路由器作用于 token 的隐藏表示，其中也包含其上下文信息。因此，我们不应将其视为对特定 token 的每次出现都分配相同的传递次数（即上图中的单词“People”并不总是循环 3 次）。决策可能会根据该单词出现的位置及其前面的内容而变化。
The router operates on a token’s hidden representation, which also contains information about its context. So, we shouldn’t think of this as assigning every occurrence of a particular token the same number of passes (i.e., the word “People” in the figure above doesn’t always go through a loop of 3). The decision can change depending on where that word appears and what came before it.
那么，路由具体是如何工作的？论文探索了两种做出路由决策的方式，如下图所示。
Now, how does the routing work exactly? The paper explores two ways to make this routing decision, as illustrated below.
图 10. 选择递归深度的两种方式。左侧，路由器在每个步骤选择哪些 token 继续。右侧，单个路由器在开始时分配通过次数。图来自 Mixture-of-Recursions 论文。
Figure 10. Two ways to choose the recursion depth. On the left, routers select which tokens continue at each step. On the right, a single router assigns the number of passes at the beginning. Figure from the Mixture-of-Recursions paper.
在*专家选择路由*中（如上图左侧子图所示），每个递归步骤选择它将处理哪些 token。退出的 token 被排除在后续步骤之外。在* token 选择路由*中（如右侧所示），路由器在开始时做出一次决策，将每个 token 分配到具有一次、两次或三次通过的路径。
In *expert-choice routing*, which is shown in the left subpanel in the figure above, each recursion step selects which tokens it will process. Tokens that exit are excluded from later steps. In *token-choice routing*, shown on the right, the router makes one decision at the beginning, assigning each token to a path with one, two, or three passes.
在这两种情况下，Transformer 权重在多次通过中重复使用，类似于 Nanbeige 等。但额外的灵活性来自于选择每个 token 接收多少计算。模型及其路由器一起训练，因此模型在训练期间学会处理这些不同的路径。
In both cases, the transformer weights are reused across passes, similar to Nanbeige, etc. But the additional flexibility comes from choosing how much computation each token receives. The model and its routers are trained together, so the model learns to work with these different paths during training.
下面来自 Mixture-of-Recursions 论文的图比较了常规 Transformer（Vanilla）、固定递归的 Transformer（Recursive）以及 Mixture-of-Recursions（MoR）在不同模型规模和算力预算（x 轴）下的表现。
The plot from the Mixture-of-Recursions paper below compares a regular transformer (Vanilla), a transformer with fixed recursion (Recursive), and Mixture-of-Recursions (MoR) for different model sizes and compute budgets (x-axis).
图 11. 在四种模型规模和三种训练算力预算下的验证损失。图来自 Mixture-of-Recursions 论文。
Figure 11. Validation loss across four model scales and three training compute budgets. Figure from the Mixture-of-Recursions paper.
在最小的模型规模下，常规 Transformer 表现最佳。对于更大的模型，Mixture-of-Recursions 迎头赶上，并且通常表现更好，尤其是在较小的训练预算下。在最大的预算下，几条曲线非常接近。因此，优势取决于模型规模以及我们在训练上花费的算力。
At the smallest model scale, the regular transformer performs best. For the larger models, Mixture-of-Recursions catches up and often performs better, especially at the smaller training budgets. At the largest budget, several of the curves are very close. So, the advantage depends on the model size and how much compute we spend on training.
这里的另一个细节是，相同的训练算力并不一定意味着相同数量的训练 token。通过跳过一些计算，Mixture-of-Recursions 可以在相同的预算内处理更多的 token。
Another detail here is that equal training compute doesn’t necessarily mean an equal number of training tokens. By skipping some computation, Mixture-of-Recursions can process more tokens within the same budget.
我认为这是一个有趣的例子，因为它表明在循环 Transformer 的想法中存在多种选择，即每个位置有多少次循环以及如何决定。
I think this is an interesting example because it shows that there are several choices within the looped-transformer idea, that is, how many loops there are at each position and how that’s decided.
简而言之，我们可以说，如果模型足够大，那么在固定算力预算下，使用循环 Transformer 可以提高模型质量。（这也说明了在规模上运行一些实验的重要性；例如，仅看较小的 135M 参数模型，我们会得出相反的结论。）
So, in short, we can say that using looped transformers can improve model quality at a fixed compute budget if the model is large enough. (It also illustrates the importance of running some experiments at scale; e.g., just looking at the smaller 135M parameter model, we would have drawn the opposite conclusion.)
顺便说一句，如果你有深度学习（甚至是 20 世纪 90 年代的人工神经网络）的背景，那么循环或“循环深度”的想法应该有些熟悉。还记得循环神经网络（RNN）吗？RNN 的核心思想就是重用前一次迭代的层（权重）。
By the way, if you have a background in deep learning (or even artificial neural networks in the 1990s), the looping or “recurrent depth” idea should be somewhat familiar. Remember recurrent neural networks (RNNs)? The whole idea in RNNs is to reuse the layers (weights) from a previous iteration.
图 12：RNN 示意图（来自我 2022 年的《Machine Learning with PyTorch and Scikit-Learn》一书，https://amzn.to/3YzRnPR）
Figure 12: Illustration of an RNN (from my 2022 “Machine Learning with PyTorch and Scikit-Learn book”, https://amzn.to/3YzRnPR)
主要区别在于，RNN 在不同时间步上重用其权重。也就是说，隐藏状态从一个词元传递到下一个词元。而在循环 Transformer 中，词元的循环是沿着架构深度进行的。
The main distinction is that RNNs reuse their weights across time steps. That is, the hidden state is carried forward from one token to the next. In the looped transformer, the looping of a token is across the architecture depth.
或者换句话说，在传统的 RNN 中，每一步接收输入序列中的下一个元素以及上一步的隐藏状态。因此，当 RNN 处理一段文本时，它一次读取一个单词或词元，并通过隐藏状态将较早单词的信息向前传递。
Or, in other words, in a conventional RNN, each step takes the next element in the input sequence and the hidden state from the previous step. So, when the RNN is processing a chunk of text, it reads one word or token at a time and carries information from the earlier words forward in its hidden state.
在循环 Transformer 中，给定词元的中间表示会多次通过 Transformer 堆栈。模型仍然使用注意力机制在词元之间传递信息。
In a looped transformer, the intermediate representation of a given token goes through the transformer stack multiple times. The model still uses attention to pass information between tokens.
如果这个类比有点过于令人困惑，不必太在意。理解循环 Transformer 的一个或许更简单的方式，是将其视为对 Transformer 块的重复使用，类似于把模型做大，但采用权重共享。
If this analogy is a bit too confusing, don't worry about it too much. A perhaps simpler way to think about looped transformers is to think of them as reusing transformer blocks, similar to making the model bigger but with weight sharing.
图 13：RNN 与循环 Transformer 中“循环”的并排对比。
Figure 13: Side-by-side comparison of the "recurrence" in an RNN and a looped transformer.
在我们讨论循环 Transformer 机制是否如之前《The Information》引述中所传闻的那样掩盖了推理痕迹之前，GPT-6 Astra 真的使用了循环 Transformer 的概念吗？
Before we discuss whether the looped transformer mechanism obscured reasoning traces, as rumored in The Information quote from earlier, does GPT-6 Astra even use the looped transformer concepts?
图 14：《The Information》的引述（来源：https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns）
Figure 14: Quote from The Information (Source: https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns)
我们必须记住，这仍然只是传闻或独家报道，没有官方确认。如果该模型是开放权重的，我们当然可以自行核实，但在这种情况下，我们只能依赖未经证实的报道。
We have to keep in mind that this is still just a rumor or scoop, with no official confirmation. If the model were open-weight, we could double-check this ourselves, of course, but in this case we have to rely on unverified reporting.
然而，我认为 GPT-6 Astra 极有可能使用了循环 Transformer 的某些方面。首先，有上述报道。其次，这是一种在过往研究中已显示出前景的技术（如前所述），那为何不用呢？第三，OpenAI 的首席科学家说了以下内容。
However, I think it’s highly likely that GPT-6 Astra uses looped transformer aspects. First, there is the reporting mentioned above. Second, it’s a technique that has shown promise in past studies (as discussed earlier), so why not? Third, OpenAI’s chief scientist said the following.
然而，这并未明确证实循环 Transformer 架构，也可能只是意味着他们使用了双倍数量的常规 Transformer 块。
However, this doesn’t confirm the looped transformer architecture explicitly, and it could also just mean they use twice as many regular transformer blocks.
在我看来，Astra 背后的成功（即良好的建模性能）很可能主要归因于其他原因，即改进的训练配方和训练数据。
In my opinion, the success (i.e., good modeling performance) behind Astra is likely primarily due to other reasons, namely improved training recipes and training data.
循环 Transformer 的调整可能有一点帮助，但我认为 The Information 高估了它的贡献。
The looped transformer tweak might help a bit, but I think that The Information is overestimating its contribution.
接下来，我们终于要直面那个显而易见却避而不谈的问题：循环 Transformer 是否会掩盖推理轨迹？
Next, let's finally address the elephant in the room: does the looped transformer obscure the reasoning traces?
首先，自 OpenAI o1 起，OpenAI 从一开始就（大部分）对用户隐藏了推理轨迹。因此，对最终用户而言，不应有太大差别。
First, OpenAI has been hiding (most of) the reasoning traces from users from the very beginning, since OpenAI o1, anyway. So, for the end-user, there shouldn't be a big difference.
所以，可解释性方面的担忧主要针对模型开发者。
So, the interpretation concern is mostly with respect to the model developers.
无论如何，我认为循环 Transformer 并非隐藏或掩盖思维链的重要因素。为了解释我自己的推理（并非双关），让我们退一步，解释推理模型是如何工作的。
Either way, I don't think that looped transformers are significant contributors towards hiding or obscuring chains of thought. To explain my own reasoning (no pun intended), let's take a step back and explain how reasoning models work.
推理模型通常在生成最终答案之前会生成中间步骤。这些步骤使用常规文本 token（在某些用户界面中可选择对用户隐藏），被称为推理轨迹或思维链。
Reasoning models typically generate intermediate steps before producing a final answer. These steps use regular text tokens (that are optionally hidden from the user in some user interfaces) and are called a reasoning trace or chain of thought.
例如，假设我们要求找出两个数，其和为 10，乘积为 21。在下图中，模型首先尝试 5 和 5。虽然和是正确的，但乘积是 25，而不是 21。接着，它尝试 3 和 7，并再次检查这两个条件。
For example, say we ask for two numbers whose sum is 10 and whose product is 21. In the figure below, the model tries 5 and 5 at first. While the sum is correct, the product is 25, not 21. Next, it then tries 3 and 7 and checks both conditions again.
图 15. 一个示例性的 LLM 响应，标注了中间步骤、回溯和最终答案。
Figure 15. An illustrative LLM response annotated to show intermediate steps, backtracking, and the final answer.
该图展示了推理模型如何“推理”，包括回溯。也就是说，模型注意到错误，然后重新审视之前的选择，接着以不同的方法继续。
The figure illustrates how a reasoning model "reasons," including backtracking. That is, the model notices a mistake, then revisits an earlier choice, and then continues with a different approach.
注意，模型仍然一次生成一个 token，使用提示和之前的 token 作为上下文。因此，这些中间步骤充当草稿纸，在最终答案之前增加了计算。
Note that the model still generates one token at a time, using the prompt and previous tokens as context. So, these intermediate steps work as a scratch pad and add computation before the final answer.
如上例所示，最终答案可以比其之前的推理轨迹短得多。（OpenAI 倾向于对用户隐藏大部分推理轨迹。）
The final answer can then be much shorter than the reasoning trace that preceded it, as shown in the example above. (OpenAI tends to hide most of the reasoning traces from the users.)
关于理解和开发推理模型的更多细节，我推荐我的书《Build a Reasoning Model From Scratch》。
For more details on understanding and developing reasoning models, I recommend my book Build a Reasoning Model From Scratch.
图 16：我的《Build a Reasoning Model From Scratch》一书涵盖了推理模型的基础知识。
Figure 16: My Build a Reasoning Model From Scratch book covers the fundamentals of reasoning models.
现在，推理轨迹中额外的 token 会增加更多计算。循环 Transformer 会增加更多计算，因为 token 要经过更多的 Transformer 块。有人可能会认为，带有循环的模型在内部使用了更多计算，因此不需要那么多外部思考 token。
Now, extra tokens in the reasoning trace add more computation. Looped transformers add more computation, because the tokens go through more transformer blocks. One might argue that a model with looping uses more computation internally, so it doesn't need as many external thinking tokens.
下面是 GPT-6 基准测试的部分结果，x 轴为输出 token 数量。
Below is a selection of the GPT-6 benchmarks with the output token number on the x-axis.
图 17：来自 https://openai.com/index/gpt-6-astra/ 的选定 GPT-6 Astra 基准测试
Figure 17: Selected GPT-6 Astra benchmarks from https://openai.com/index/gpt-6-astra/
我们可以看到，GPT-6 Astra 在各个努力水平上整体而言并不一定比其前代 GPT 5.6 Sol 使用更少的 token。然而，在固定准确率下，GPT-6 Astra 确实比 GPT 5.6 Sol 使用更少的 token。
We can see that GPT-6 Astra doesn't necessarily use fewer tokens than its GPT 5.6 Sol predecessor across effort levels overall. However, at a fixed accuracy, it is true that GPT-6 Astra uses fewer tokens than GPT 5.6 Sol.
这对可解释性来说是个问题吗？不一定。使用更少的 token 可能只是意味着模型能力更强、犯的错误更少、回溯更少等等。也就是说，它可能只是第一次尝试就能做对更多事情。对我来说，这并没有立即引起关于可解释性的担忧。
Is this a concern for interpretability? Not necessarily. Using fewer tokens could just mean that the model is more capable and makes fewer mistakes, uses less backtracking, and so on. I.e., it might just get more things right on the first try. To me, that doesn't raise an immediate concern regarding interpretability.
我的意思是，之前的模型也是如此。我不认为有人会强烈担忧 GPT 5.6 Sol 的可解释性远低于更小的 GPT 5.6 Luna 模型，后者在相同任务性能下使用了多得多的 token，如下图所示。
I mean, the same is true for previous models. I don't think that anyone has strong concerns that GPT 5.6 Sol is so much less interpretable than the smaller GPT 5.6 Luna model, which uses many more tokens for the same task performance, as shown below.
图 18：Luna 和 Sol 在相似任务性能水平下的 token 使用量。数据来自 Artificial Intelligence Index v4.3。
Figure 18: Token usage in Luna and Sol at similar task performance levels. Numbers from the Artificial Intelligence Index v4.3.
事实上，我们可以看到，在相似的建模性能下，Luna 使用的 token 比 Sol 多 80%。这会让 Sol 的可解释性差那么多吗？
In fact, as we can see that Luna uses 80% more tokens than Sol at similar modeling performance. Does that make Sol that much less interpretable?
更合理的答案是，能力更强的模型（更大、训练良好、使用更多算力的模型）能更高效地解决问题，这里的“高效”意味着更少的 token。
Rather, the more plausible answer here is that more capable (bigger, well-trained models that use more compute) can solve problems more efficiently, where "efficient" here means fewer tokens.
同样值得记住的是，推理轨迹并不保证能忠实描述模型内部发生的一切。在我看来，唯一合理的担忧是，循环 Transformer 比传统 Transformer 更频繁地通过呈现“虚假”推理轨迹来故意误导用户。但我认为我们没有任何强有力的证据表明这种情况正在发生。
It's also worth keeping in mind that a reasoning trace is not guaranteed to faithfully describe everything that happens inside the model. In my view, the only valid concern is that looped transformers purposefully mislead users by presenting "fake" reasoning traces more often than conventional transformers. But I don't think we have any strong evidence that this is happening.
现在，Astra 的系统卡确实指出，也有证据表明其推理轨迹的可监控性降低，并且相对于 Sol 存在一些退化。这主要与更短、信息量更少的轨迹有关。但同样，这并不能确立循环是根本原因。它可能只是由于整体长度更短，类似于上面 Luna 与 Sol 的例子。
Now, Astra's system card does state that there is also evidence of reduced monitorability of their reasoning traces, and there is a bit of regression relative to Sol. It's mostly associated with shorter, less informative traces. But again, this doesn't establish looping as the root cause. It could just be due to the shorter length in general, similar to the Luna vs Sol example above.
在我分享关于循环 Transformer 与隐藏推理链相关的想法几小时后，Jakub Pachocki（OpenAI 的首席科学家）也分享了以下澄清：
A few hours after I shared my thoughts about looped transformers with respect to hiding reasoning chains, Jakub Pachocki (OpenAI's Chief Scientist) also shared the following clarification:
“混乱的报道”很可能指的是 The Information 前面提到的段落，暗示循环方面与思维链变化没有任何关系。
The "confused reporting" likely refers to The Information's aforementioned paragraph here, implying that the looping aspect does not have anything to do with chain-of-thought changes.
最后，我想分享一些与我们已讨论过的论文不同的、与循环 Transformer 架构相关的有趣论文。
Lastly, I want to share some interesting papers related to looped transformer architectures beyond the ones we already discussed.
与 Universal Transformer 相关，2025 年的论文《Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach》研究了模型如何在推理时使用额外的循环。为此，他们在 800B 个 token 上训练了一个相对适中但也不算特别小的 3.5B 参数模型。
Related to the Universal Transformer, the 2025 paper *Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach* studies how a model can use additional loops at inference time. For this, they trained a relatively modest but also not super tiny 3.5B-parameter model on 800B tokens.
与 Universal Transformer 中反复重用同一个块不同，它像 Nanbeige 那样重复一个堆叠；然而，与 Nanbeige 相反，它将这个由四个块组成的共享堆叠夹在 2 个初始块和 2 个最终块之间。
Instead of reusing the same block over and over again as in the Universal Transformer, it repeats a stack like in Nanbeige; however, in contrast to Nanbeige, it sandwiches this shared stack of four blocks between 2 initial and 2 final blocks.
此外，与 Nanbeige 的不同之处在于，共享堆叠在每次循环开始时接收初始块的输出，以及前一次循环的隐藏状态。这些被拼接起来，并在进入四个共享块之前通过一个学习到的线性投影。你可以把这理解为在每次传递时让堆叠访问相同的初始输入表示。整个布局总结在下图中。
Also, what's different from Nanbeige is that the shared stack receives the output of the initial blocks at the start of every loop, in addition to the previous loop's hidden state. These are concatenated and passed through a learned linear projection before entering the four shared blocks. You can think of this as giving the stack access to the same initial input representation on every pass. This whole layout is summarized in the figure below.
所以，简而言之，这是一个额外且有趣的循环 Transformer 变体。
So, in short, this is an additional and interesting looped transformer variant.
图 19. Geiping 等人提出的潜在推理模型的概念总结。
Figure 19. Conceptual summary of the latent reasoning model in Geiping et al..
一个有趣的细节是，研究人员在训练期间改变了循环次数。这使模型能够在推理时适应不同的计算量。
An interesting detail is that the researchers vary the number of loops during training. This prepares the model to work with different amounts of computation at inference time.
在这里，训练期间循环次数是随机采样的。在推理时，由运行模型的人选择一个固定的预算，例如 8、32 或 64 次循环。此外，他们基于下一个词的概率分布为每个 token 设计了一种自适应停止机制。如果连续两轮之间的 KL 散度低于某个阈值，即分布过于相似，则停止循环。
Here, during training, the loop count is randomly sampled. At inference, a fixed budget is chosen by whoever runs the model, such as 8, 32, or 64 loops. Additionally, they have an adaptive stopping mechanism for each token based on the next-token probability distribution. If the KL-divergence between 2 successive rounds is below a certain threshold, i.e., if the distributions are too similar, the looping is halted.
总体收益取决于任务。在他们的评估中，HellaSwag 的性能在大约八次循环后基本趋于平稳，而 GSM8K 和 HumanEval 则从更多循环中受益。
The overall benefit depends on the task. In their evaluations, HellaSwag performance largely levels off after about eight loops, while GSM8K and HumanEval benefit from more.
然而，尽管论文标题提到了“潜在推理”，该模型仍然可以生成文本思维链。循环只是在每个输出 token 之前为其提供了额外的计算。
However, while the title of the paper mentions “latent reasoning”, the model can still generate a textual chain of thought. Looping just gives it additional computation before each output token.
存储信息与利用信息解决问题之间存在一个有用的区分。例如，2025 年 6 月的论文《超越参数：探索缩放定律的虚拟逻辑深度》通过分别测量 LLM 中的记忆和推理来研究这一点。
There is a useful distinction between storing information and using it to solve a problem. For instance, the paper "Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws" from June 2025 investigates this by measuring memorization and reasoning in an LLM separately.
首先，在记忆实验中，当参数数量固定时，循环几乎不改变存储的信息量。增加不同参数的数量确实会增加这种容量。由此我们可以得出结论：循环不会增加或让模型检索更多知识。这是合理的。一旦信息被存储，信息检索是一个相对简单的任务。此外，循环本身是一种计算机制，而非“存储”机制。
First, in the memorization experiments, looping leaves the amount of stored information nearly unchanged when the parameter count stays fixed. Increasing the number of distinct parameters does increase this capacity. From this, we can conclude that looping does not add or let the model retrieve more knowledge. This makes sense. Information retrieval is a relatively simple task once the information is stored. Also, looping in itself is a computing, not a "storing", mechanism.
其次，在单独的推理实验中，重用模块可以在不增加参数的情况下提高多步数学问题的性能。这很有趣。在这里，我们可以得出结论：即使模型没有更多空间来存储信息，额外的计算也能帮助模型解决问题。但同样，更大的模型也能提高推理能力（尽管它们也增加了参数）。
Second, in separate reasoning experiments, reusing the blocks improves performance on multi-step math problems without adding parameters. This is interesting. Here, we can conclude that extra computation can help a model solve problems even when it does not have more space to store information. But again, bigger models can also improve reasoning (although they add parameters as well).
图 20。在这个记忆测试中，容量随参数数量增长，但随额外的模块应用变化很小。Zhu 等人标注的图。
Figure 20. In this memorization test, capacity grows with parameter count but changes little with additional block applications. Annotated figure from Zhu et al.
2026 年 9 月刚刚发布的《SMELT：计算匹配的 MoE 循环 Transformer 的缩放定律》论文回到了第 2.2 节中的成本比较。如果我们比较循环 Transformer 和传统 Transformer，在每 token 计算量、总非嵌入参数和 KV 缓存需求大致相同的情况下，会发生什么？
The just-released SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers paper from September 2026 comes back to the cost comparison from section 2.2. What happens if we compare looped and conventional transformers with approximately the same compute per token, total non-embedding parameters, and KV cache requirements?
研究人员使用混合专家架构，并将 Transformer 块的中间一半应用两次，有点类似于 Nanbeige，但在潜在推理中采用了夹层结构。
The researchers use a mixture-of-experts architecture and apply the middle half of the transformer blocks twice, kind of similar to Nanbeige except with the sandwiching in Latent Reasoning.
然而，他们缩小了隐藏维度，以补偿额外块应用所需的计算量。然后，由于这会使参数数量变小，他们又添加专家来恢复总参数数量。他们还调整了注意力头配置，以保持 KV 缓存相当。
However, they narrow the hidden dimension to compensate for the compute needed for the extra block applications. And then, because that makes the parameter count smaller, they then add experts to recover the total parameter count. They also adjust the attention head configuration to keep the KV cache comparable.
图 21. 来自 SMELT 第 3.2 节示例的 SMELT 概述。
Figure 21. SMELT overview from the example given in SMELT, section 3.2.
实验规模扩大到 54B 非嵌入参数等。然后，根据拟合的缩放曲线，研究人员估计，在研究计算范围内，SMELT 达到相同验证损失所需的训练计算量约少 6.8-18%。
The experiments scale up to 54B non-embedding parameters and so on. Then, from fitted scaling curves, the researchers estimate that SMELT requires about 6.8-18% less training compute to reach the same validation loss within the studied compute range.
因此，这回答了循环 Transformer 在算力上是否值得的问题：值得！在相同的算力预算下，它们能给我们一个略好的模型。
So, this answers the question of whether looped transformers are worth it computationally: Yes! They give us a slightly better model when using the same compute budget.
最后，同样非常近期的 2026 年 8 月的全带宽 Transformer 论文研究了跨 token 位置的循环。在每个解码步骤中，它通过一个学习到的门控将前一个 token 的最终隐藏状态与新采样 token 的嵌入相结合。这成为下一次前向传播的输入。
Finally, the also very recent Full-bandwidth transformer paper from August 2026 studies recurrence across token positions. At each decoding step, it combines the previous token's final hidden state with the newly sampled token's embedding through a learned gate. This becomes the input for the next forward pass.
因此，下一个 token 的计算可以访问来自堆栈底部的前一个 token 的最终表示，这有点类似于潜在推理。
So, the next token's computation has access to the previous token's final representation from the bottom of the stack, which is somewhat similar to Latent Reasoning.
当使用 1B 基础模型时，他们发现其潜在反馈方法在 MATH500 上输出更短的推理轨迹，同时保持或提高准确性。然而，在指令微调后，这种缩短效果消失了。
When using a 1B base model, they found that their latent feedback approach outputs shorter reasoning traces on MATH500 while maintaining or improving accuracy. However, the shortening effect disappears after instruction tuning.
图 22. 潜在反馈在基础模型中缩短了推理轨迹，但在指令微调后这种效果消失。改编自 Wang 等人，图 6，CC BY 4.0。添加了定义和注意事项。
Figure 22. Latent feedback shortens reasoning traces in the base model, but this effect disappears after instruction tuning. Adapted from Wang et al., Figure 6, CC BY 4.0. Definitions and caveats added.
无论如何，这很有趣，因为它直接联系到之前关于循环是否会导致更短推理轨迹的讨论。当然，结果取决于反馈机制和模型的训练方式。此外，该实验并未确定那些更短的轨迹是否更不忠实。
Anyway, this is interesting because this connects directly to the earlier discussion about whether looping results in shorter reasoning traces. The result depends on both the feedback mechanism and how the model is trained, of course. Also, the experiment doesn't establish whether those shorter traces are less faithful.
此外，该研究的一大局限在于，他们并未测试以传统方式扩大模型规模（即增加更多 Transformer 块而非循环）是否会对推理轨迹长度产生类似影响。
Also, the big caveat of the study is that they didn't test whether increasing the size of the model in conventional ways (adding more transformer blocks instead of looping) has a similar effect on the reasoning trace lengths.
总而言之，我们可以说，OpenAI GPT-6 Astra 确实是一个非常强大的模型。它在计算机使用方面取得了特别大的飞跃。我相信，计算机使用将成为未来几个月开源和专有工具链的下一个重点领域。我认为开源在计算机使用方面尤为重要，因为“能力越大，责任越大”，而且在让工具链访问我的主计算机之前能够对其进行审计是件好事。
To wrap it all up, we can say that yes, OpenAI GPT-6 Astra is a very strong model. And it's making a particularly large leap in computer use. I believe computer use will be the next big focus area for open-source and proprietary harnesses in the upcoming months. I find open-source especially important when it comes to computer use, as "with great power come great responsibilities", and it's nice to be able to audit the harness before giving it access to my main computer.
此外，GPT-6 Astra 很可能使用了循环 Transformer 的变体。在固定算力预算下，循环 Transformer 能带来更好的建模性能。
Besides, GPT-6 Astra is likely to use a variant of the looped transformer. Looped transformers simply give better modeling performance at a fixed compute budget.
同时，更好的建模性能可能会在更短的推理链中减弱。但这不是新趋势。我们一直看到，在包含不同规模模型的模型家族中（例如 GPT 5.6 Luna 与 Sol），情况就是如此。
Also, better modeling performance may decrease in shorter reasoning chains. But this is not a new trend. We have always seen that within a model family with models of different sizes (e.g., GPT 5.6 Luna versus Sol).
在我看来，更短的推理轨迹是更“智能”或能力更强的模型的副作用，这些模型犯的错误更少，并且能在其架构内部访问更多算力，而不是将推理轨迹用作草稿纸。从某种意义上说，人类也是如此。在面对面大学数学考试中，一个聪明且准备充分的学生可能更少使用草稿纸，也更少需要回溯，等等。
In my opinion, shorter reasoning traces are a side effect of more "intelligent" or capable models that make fewer mistakes and can access more compute internally inside their architecture versus using a reasoning trace as a scratchpad. In a sense, the same is true for humans. During an in-person college math exam, a smart and well-prepared student likely requires less use of the notepaper and needs to backtrack less often, and so on.
如果你想学习如何自己构建推理模型，请查看我的书《从零构建推理模型》。我们从预训练 LLM 开始，逐步添加推理能力，并提供可运行和实验的代码。这既有趣又有收获，而且是对未来自我的良好投资，可以建立跟上 AI 领域所需的基础知识。
If you'd like to learn how to build reasoning models yourself, check out my book Build a Reasoning Model (From Scratch). We start with a pre-trained LLM and add reasoning capabilities step by step, with code you can run and experiment with. It's both fun and rewarding, and a good investment in future-self to build the fundamentals to keep up with the AI field.
此外，如果你读过我的某本书，希望你能在 Amazon 上留下一段简短、真实的评论。评论能帮助其他读者判断一本书是否适合他们，也是支持作者的一种简单方式。
Also, if you've read one of my books, I'd appreciate a short, honest review on Amazon. Reviews help other readers decide whether a book is right for them and are a simple way to support authors.
图 23：我的《从零构建推理模型》（_Build a Reasoning Model (From Scratch)_）一书中的部分插图，涵盖推理时 Scaling（规模扩张）、蒸馏和强化学习。
Figure 23: Selected illustrations from my _Build a Reasoning Model (From Scratch)_ book, covering inference-time scaling, distillation, and reinforcement learning.
很棒的文章，写得也好。与我最近读的其他材料相比，它更易读易懂。
Great article and well written. Easy to read and understand, compared to the other materials I have been reading recently.
我肯定没有完全理解，但这是一篇引人入胜的文章。
I'm sure I didn't understand it all, but it was a fascinating article.