The Pacing of the Frontier
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→本文讨论了迫切需要对前沿 AI 能力的发展进行节奏控制以减轻生存风险，认为当前的发展轨迹可能导致灾难性后果。文章对比了主张建立未来减速能力的《前沿节奏》公开信与呼吁立即暂停的观点，并探讨了协调困难及潜在益处等反对意见。作者强调，可扩展的对齐对于递归自我改进至关重要，并警告狭隘的解决方案是不够的。结论是，虽然全面暂停尚不必要，但准备优雅地引导 AI 发展并根据风险升级调整节奏至关重要，因为不受控制的进步可能摧毁人类价值观和自由秩序。
The article discusses the urgent need to pace the development of frontier AI capabilities to mitigate existential risks, arguing that current trajectories could lead to catastrophic outcomes. It contrasts the Pacing the Frontier letter, which advocates for building the ability to slow down later, with calls for an immediate pause, and explores objections such as coordination difficulties and potential benefits. The author emphasizes the importance of scalable alignment for recursive self-improvement, warning that narrow solutions are insufficient. The piece concludes that while a full pause is not yet warranted, preparing to steer AI development gracefully and adjusting pace as risks escalate is critical, as uncontrolled progress could destroy human values and liberal order.
3. 关于为前沿发展设定节奏的支持声明。
3. Statements of Support For Pacing the Frontier.
10. 如果你从事的是常规对齐工作，请转向可扩展对齐。
10. If You Are In Mundane Alignment Pivot To Scalable Alignment.
关于特定能力水平会有多危险、或它将如何实际影响世界，也存在分歧；关于我们有哪些选择、各种协调机制或政府干预的性质，以及如何平衡不同的神圣价值，也存在分歧。人们往往只真正看到关键权衡的一半。
There are also disagreements about how dangerous a given level of capability would be, or how it would physically impact the world, and disagreements about what options we have, the nature of various coordination mechanisms or government interventions, and balancing different sacred values. Often people only properly see one half of a key trade-off.
人们常常真诚地只看到这类关键权衡的一半，主要原因在于他们预期的人工智能进展如此之少，以至于减轻生存风险或担心人类失去控制变得没有必要。
Mostly the reason people often sincerely only see one half of such key tradeoffs is that they anticipate so little AI progress that mitigating existential risks or worrying about humans losing control is unnecessary.
或者他们预期的人工智能进展如此之多，以至于减轻生存和灾难性风险、维持人类控制必须成为优先事项，而这必然意味着某个群体以某种方式集体选择并规划一条穿越因果空间的深思熟虑的路径，通往让我们得以生存的结果。
Or they anticipate so much AI progress that mitigating existential and catastrophic risks and maintaining human control has to be the priority, and that necessarily is going to mean some group of people collectively choosing and charting, in some way, a deliberate path through causal space towards outcomes that allow us to survive.
是的，这必然意味着让某个群体拥有某种集体机制来规划地球穿越因果空间的某些方面，而且确实有理由对此担忧，但正因为如此，我们才应该努力找出做到这一点的最佳方式。
Yes, that necessarily means enabling some group of people to have some collective mechanism to chart some aspects of Earth’s path through causal space, and yes there are reasons to worry about that, but that is why we should work to figure out the best way to do that.
那些反对《为前沿技术设定节奏》公开信的人并不想“放慢速度”，而是希望人工智能“加速”，但他们所设想的“快”，尽管按历史标准来看已是超快，却并非那么快。
Those opposing the Pacing the Frontier letter do not want to 'slow down' and instead want AI to go 'fast,' but their vision of fast is, while super fast by historic standards, not all that fast.
签署《为前沿技术设定节奏》公开信的人大多希望人工智能至少与反对者所期望的一样快。他们想要避免的是人工智能以“超级、超级、超级、超级快”的速度发展。
Those who signed the Pacing the Frontier letter mostly want AI to go at least as fast as the opposition. What they want to avoid is AI going super duper ultra hyper fast.
几乎所有签署这封信的人都希望有更好的聊天机器人、医生、科学以及其他形式的传播。他们不希望在 2027 年出现超级智能和奇点。
Almost everyone signing the letter wants better chatbots and doctors and science and other forms of diffusion. They don’t want superintelligence and a singularity in 2027.
我引用最后一点，是因为当前的情况是：反对《为前沿技术设定节奏》公开信的人声称他们不会等到二月，他们要求在 12 月 25 日就收到礼物，而且他们最好得到那把 BB 枪。
I quote that last one because what is happening is that the people who are arguing against the Pacing the Frontier letter are saying they won’t wait until February and demanding they get the presents on December 25 and they’d better get that BB gun.
而签署这封信的人则说，也许我们应该至少等到明天再订购更多礼物，因为家里已经没有更多空间了，而且也许应该让玩具保持合理，只给你那把会打瞎你眼睛的 BB 枪，孩子，而不是 AK-47、战术核弹或那个性感的冯·诺依曼探测器。
Whereas those signing the letter are saying maybe we should wait until at least tomorrow before we order even more presents and there is no more room left in the house, plus maybe keep the toys reasonable, and only get you the BB gun that’ll shoot your eye out kid and not an AK-47 or tactical nuke or that sexy Von Neumann probe.
另一位签署《为前沿设限》公开信的人士也发表了强有力的评论：
Another strong comment from one of those who signed the Pacing the Frontier letter:
塞缪尔·哈蒙德（Samuel Hammond）就我们可能即将采取的实际行动以及需要做好准备应对的情况，发表了一段非常精彩的陈述。如果你持续在对数图上画直线，并遵循其中涉及的逻辑，你就会得到这样的结果。你可以随意援引荒谬性启发式或“瓶颈”之类的说法，但这正是实验室们实际预期的。哈蒙德提出的变体显示事情加速了不少，但尚未完全以 ASI（超级智能）的视角来看待这一周期的最终结局：
And a very good statement about the actual thing we may be on the verge of doing, and need to prepare to handle, from Samuel Hammond. You get results like this if you keep drawing straight lines on logarithmic graphs and follow the logic of everything involved. You can invoke the absurdity heuristic or ‘bottlenecks’ or what not all you want, but this is what the labs actually expect. The variation Hammond offers has things accelerating quite a bit but is not even fully ASI (superintelligence) pilled about the ultimate ends of this cycle:
这里最重要的一点在最上面。如果事情如上所述加速发展，我认为人类很可能会完蛋。即使人类在这种情景下幸存下来，你的自由秩序也绝对会完蛋。在一个 AI 在几乎所有方面都比人类更有能力、且每天都在增强、却未受到有意义的约束、也无法被集体引导的世界里，提出相反的看法将显得荒谬或不合逻辑。
The most important point here is up top. If things accelerate as described above, I think humanity is probably toast. Even if humanity survives that scenario, your liberal order is most definitely toast. It will seem absurd or incoherent to suggest otherwise, in a world with AIs that are more capable than humans across basically everything and growing more so every day, that have not been meaningfully constrained and cannot be collectively steered.
你是否认为，通过确保没有人负责实验室的运作，当顶尖实验室比下一个实验室领先许多个当前周期，并且人工智能在所有方面都超越人类时，你会得到一个美好、友好、自由的人类秩序？
Do you think that by ensuring no person is in charge of what the labs do, when the top lab is many current cycles ahead of the next one and the AIs outperform the humans across the board, that you will get a nice, friendly, liberal order of humans?
我实际的理解是，这种想法是：‘我不需要去考虑那个，我只知道你提出的建议听起来很糟糕，所以我反对它。’
My actual read is that the thinking is ‘I don’t need to think about that, I just know that what you are proposing sounds bad, so I am against it.’
这种反应是对问题前提的拒绝。它拒绝被“ASI 化”，即使是在假设情境中也是如此。
That reaction is a rejection of the premise of the question. It is refusing to be ASI pilled, even within a hypothetical.
在这种情况下，人们应该声明他们拒绝前提，但仍然回答这个假设性问题。如果这样的人工智能确实相对较快地出现，你认为默认会发生什么？如果你认为答案仍然是‘美好、友好、自由的人类秩序’，那么你仍然依赖哪些限制来确保这一点？什么会导致它崩溃？
In which case, one should state they are rejecting the premise, but also still answer the hypothetical. If such AIs did come to pass relatively soon, what do you think would happen by default? If you think the answer is still ‘nice, friendly, liberal order of humans,’ then what limitations are you still counting on to ensure this? What would cause it to break down?
AI 未来项目（AI Futures Project）是 AI 2027 和 Plan A 的创造者，他们提出了一些关于如何为美国 AI 前沿的未来设定节奏的选项。
AI Futures Project, the creators of AI 2027 and Plan A, lay out some options for how one might pace the future of the frontier of American AI.
链接中有对这些选项的详尽分析。我理解这些提议的逻辑。我最感兴趣的是选项 4，即要求安全案例（safety cases），尽管它更难实施。选项 2 也很有吸引力（理想数字待定），即为对齐和安全工作设定最低算力分配，并且可以作为补充，但需要指出的是，明确其含义存在明显问题。
There is extensive analysis of these options at the link. I see the logic in these proposals. I am most interested in option 4, to require safety cases, even though it is harder to implement. Option 2 also appeals (with ideal numbers TBD), to have a minimum compute allocation for alignment and safety efforts, and can be a complement, with the caveat of obvious problems pinning down what that means.
还有一种立场，是《与前沿同步》公开信未采取的，即鉴于近期事件，前沿能力的发展现在就应该暂停。
One can also take the full position, not taken by the Pacing the Frontier letter, that given recent events the frontier capabilities development should be paused now.
这显然不是《与前沿同步》公开信所呼吁的。该信呼吁的是获得以后同步发展的能力。而那些呼吁暂停的人希望现在就发展速度设为零。
This is importantly not what the Pacing the Frontier letter calls for. The Pacing the Frontier letter calls for gaining the capability to pace development later. Those calling for a pause want to set the pace of development to zero, right now.
对此，一如既往地存在三个基本反对意见：
As always there are three basic objections to this:
1. 协调太难，激励机制不起作用，我们做不到。2. 想想潜力，我们承担不起这样做的代价。
1. Coordination is too hard, incentives do not work, we cannot do it. 2. Think of the potential, we cannot afford to do it.
One can also take the full position, not taken by the Pacing the Frontier letter, that given recent events the frontier capabilities development should be paused now.
3. 潜力不足，我们无需去做。
3. There is not enough potential, we do not need to do it.
我们很可能正在进入这样一个阶段：继续下去变得如此危险，以至于即使没有协调，也最好各自暂停。这反而会让协调变得容易得多。
Plausibly we are now entering the phase where it becomes so dangerous to continue that it is better to pause individually, even without coordination. Which would then make it far easier to coordinate.
在某个时刻，“无法承受继续”会压倒“无法承受停止”，即使两者都至关重要。现在更有理由认为我们正处于那个时刻。
At some point, 'you cannot afford to continue' overwhelms 'you cannot afford to stop,' even if both are importantly true. It is now a lot more arguable that we are there.
我还不认为我们已经到了同意欧文（Irving）观点的地步，但我对此的信心比两周前低得多。如果没有远超我们公开所见之外的保证，我不会同意在 OpenAI 恢复能力开发的工作，并且非常高兴他们有意放慢开发速度。
I do not think we are at that point yet where I agree with Irving, but I am a lot less confident about this than I was two weeks ago. I would not be okay working on resuming capabilities development at OpenAI without assurances well beyond what we have seen in public, and am very glad they are consciously slowing development down.
这与不发布已开发的东西（包括 GPT-5.6-Cyber 和“破晓计划”）不同，只要 GPT-5.6-Cyber 从未在留言板激活的情况下进行训练，不发布是可以的。
This is distinct from not releasing things already developed, including GPT-5.6-Cyber and Project Daybreak, which is fine so long as GPT-5.6-Cyber at no point was trained with the message board active.
参议员伯尼·桑德斯致信前沿实验室的负责人，直截了当地呼吁他们暂停 AI 开发，理由是他们曾承诺，如果无法控制 AI，就会这样做。他引用了 HuggingFace 攻击事件以及最近利用 AI 制造新病毒的例子。
Senator Bernie Sanders sends a letter to the leaders of the frontier labs, calling outright for them to pause AI development, on account of their promise to do so if they proved unable to control AI. He cites the HuggingFace attack and the recent use of AI to create new viruses.
OpenAI 实际上在几天前就因 Astra 触发了关键阈值。他们锁定了该模型，并正在采取一些加强的预防措施。很好。但他们不感兴趣的是延长暂停期，而且奥特曼明确表示，他仍打算推动尽快发布 Astra。
OpenAI actually invoked the critical threshold days ago with Astra. They locked down that model and are taking some enhanced precautions. Good. What they are not interested in is an extended pause, and Altman made clear he still intends to push to release Astra soon.
Dean Ball 澄清了他的观点，即“只要适度审慎，事情可能会进展得异常顺利”，但要真正做到适度审慎并不容易。适度审慎意味着整个组织和大量聪明人协调一致以确保事情顺利，而有些人认为人类甚至无法凝聚起这样微小的努力。
Dean Ball clarifies his view, that 'with even moderate prudence, things will probably go extraordinarily well,' but it is not so easy to have moderate prudence. Moderate prudence means entire organizations and tons of smart people coordinating to make things go well, and some believe humanity cannot muster even such modest efforts.
这是因为 Dean Ball 是 AGI 的信徒，但不是超级智能的信徒。他并不认为超级智能会以我和其他人所预期的那种形式出现。
This is because Dean Ball is AGI pilled but he is not ASI pilled. He does not expect superintelligence to take the form I and others expect it to take.
如果他对这一点判断正确，那么我不会像他那样乐观，但确实，适度的审慎会使我们处于有利地位。
If he is right about that, then I would not be as optimistic as he is but yes modest prudence would put us in a strong position.
我们首先需要克服实现适度审慎的所有障碍，因为我们目前并未朝着那个方向前进。相反，我们看到的是高层次的警报和各方面令人震惊的无能，且缺乏协调。
We would first need to overcome all the barriers to achieving modest prudence, as we are not currently on track to that. Instead, we are looking at high level fire alarms and staggering incompetence across the board, without much coordination.
Dean Ball 下面的陈述是在正确地说，许多人并不相信 AGI，这导致他们将 AI 视为“没什么新鲜事”，并且不愿意为“适度审慎”付出代价。
Dean Ball’s statement below is a way of saying, correctly, that many people are not AGI pilled, and that this causes them to dismiss AI as 'nothing new' and otherwise be unwilling to risk the cost of even 'moderate prudence.'
而那些信奉 AGI（通用人工智能）的人明白，我们至少需要适度审慎。
Whereas those who are AGI-pilled understand we need at least moderate prudence.
为前沿领域的发展节奏做好准备，是适度审慎的一个方面的例证。
Preparing to Pace the Frontier is an example of one aspect of moderate prudence.
这听起来对信奉 ASI（超级智能）的人来说同样不足，而对那些把头埋在沙子里、拒绝信奉 AGI 的人来说，这似乎又是不必要的。如果 Dean Ball 对超级智能的本质判断有误，那么适度的努力和审慎就不太可能足够。我们将需要至少非凡的努力，而且很可能需要闭嘴并去做不可能之事。那更难。
That sounds similarly insufficient to the ASI-pilled, as it seems unnecessary to those with their heads in the sand refusing to be AGI-pilled. If Dean Ball is wrong about the nature of superintelligence, then a modest effort and moderate prudence are unlikely to be sufficient. We will need at least an extraordinary effort, and likely will need to shut up and do the impossible. That is harder.
适度审慎无论如何都远胜于连这一点都没有。在任何我们可能面对的世界中，都没有理由不这样做，成本非常低，然而我们并未走上正轨，甚至这一点也面临激烈的反对。
Modest prudence is overdeterminedly far superior to not having even that. There is no reason not to have it, in any of the worlds we may face, the costs are very low, and yet we are not on track and even this faces fierce opposition.
而如果我们需要的远不止于此，我们将需要做出更艰难的选择，并在神圣价值之间面临冲突。
Whereas if we need a lot more than that, we will need to make harder choices, and face conflicts between sacred values.
如果你在做能力研究，也要转向对 RSI（递归自我改进）和 AI 研发自动化至关重要的可扩展对齐。
If you are in capabilities, also pivot to scalable alignment that matters for RSI (recursive self-improvement) and automation of AI R&D.
我假设 Mo 指的是旨在长期解决方案的对齐工作。显然，“常规对齐”工作一直都是有价值的。我认为尽早思考这类问题总是有价值的，因为其中许多问题周期长，无法并行处理，但我同意我们现在需要大力投入真正的 RSI 对齐。
I assume Mo means alignment work intended as a long-term solution. Obviously 'mundane alignment' work was always worthwhile. I think it was always valuable to get to thinking about such problems early, as much of this has long lead times and cannot be done in parallel, but I agree that we need to go heavily into real RSI-alignment now.
我强烈赞同 Yo Shavit 下面的陈述。你需要将你的 RSI 对齐工作（那些真正重要的）集中在完全 Scaling 化的通用解决方案上。狭窄的解决方案和渐进的局部步骤属于另一个部门，甚至可能掩盖问题。狭窄的解决方案只有在能让你获得可用于解决通用问题的模型时才有用。
I strongly endorse Yo Shavit's statement below. You need to focus your RSI-alignment efforts, the ones that count, on general solutions that are fully scaling pilled. Narrow solutions and incremental local steps are a different department, and if anything risk disguising the problem. The narrow solutions are useful insofar as they get you models you can use to solve the general problems.
你可以而且应该从当前的训练运行中剔除黑客模式，但你也需要一个能够在存在潜在黑客模式的情况下存活的解决方案，因为足够先进的 AI 会找到黑客模式。
You can and should prune the hack patterns out of the current training runs, but also you need a solution that survives there being potential hacking patterns, because a sufficiently advanced AI will find hacking patterns.
各大实验室的许多人认为，事情即将以更快的速度升级。几乎没有人期望在未来五年内出现真正的奇点，就其短期经济影响而言。
Many at the major labs think things are about to escalate even more quickly. Almost no one else expects a true singularity in the next five years, in terms of its short-term economic impacts.
对 AI 资本支出也有很高的期望。这里的尾部效应有点疯狂，但确实，我们应该预期数字会上升。
There are also high expectations for AI Capex. The long tail here is kind of crazy, but yes we should expect Number Go Up.
然而，实验室里的人大多在想什么呢？
Whereas what do those at the labs largely think?
这里的“真正好的模型”指的是 AI 研发自动化的开端、递归自我改进，以及基本上就是奇点。
‘Really good models’ here is code for the start of automation of AI R&D, recursive self-improvement and basically a singularity.
我确实同意，我们应当预期的一个迹象是大量的效率提升。你知道，就像 OpenAI 在 Luna 和 Sol 上报告的那样，并且每隔几周就不断发布新模型。我确信这没什么。
I do agree that one sign we should expect is a bunch of efficiency gains. You know, like what OpenAI is reporting with Luna and Sol, and constantly getting model releases every few weeks. I’m sure it’s nothing.
大约一天后，OpenAI 宣布它解决了 10 个重大的开放数学问题。
About a day later OpenAI announced it had solved 10 major open math problems.
这就是罗宾·汉森对“人们正确地注意到他们默认都会死去，他们珍视的一切都将被摧毁，而他们不喜欢这样”的总结：
This is how Robin Hanson summarizes 'people correctly notice they by default are all going to die and everything they value will be destroyed, and they don’t like that':
我就是那些人中的一员。我不期望喜欢不受控制的达尔文式选择前进过程的结果。我也不期望喜欢在当前条件下构建超级智能 AI 可能发生的各种其他事情。
I am one of those people. I do not expect to like the results of an uncontrolled forward process of Darwinian selection. I also do not expect to like various other things that might happen if we build superintelligent AIs under current conditions.
我认为目前没有必要完全暂停，但这种情况可能很快改变。我确实认为这是一个重新评估很多事情的机会，而且我们发现迫切需要整理我们内部的各个方面，这可能会延迟短期能力进展。我认为这些行动必须采取，OpenAI 原则上同意。
I do not think a full pause is warranted at this time, but that could quickly change. I do think this is an opportunity to reassess quite a few things, and we have found an urgent need to get various parts of our house in order, in ways that will likely delay short-term capabilities progress. I think those actions need to be taken, and OpenAI agrees in principle.
因此，我们需要集体评估我们的处境，提高我们优雅地引导的能力，并准备好调整我们的步伐。如果我们不准备优雅地做到这一点，那么要么我们会等待太久而完全失败，要么更可能的是，我们会试图以非常不优雅的方式去做，没有合适的工具，使用当时手头的一切，就像我们人类经常处理这类事情一样。让我们不要走到那一步。
Thus, we collectively need to assess our situation, increase our ability to steer gracefully, and be ready to adjust our pace. If we do not prepare to do it gracefully, then either we will wait too long and fail to do it at all, or more likely we will attempt to do it highly ungracefully, without the right tools, using whatever is handy at the time, as we humans often deal with such matters. Let’s not let it come to that.