OpenAI Takes Initial Steps To Address Its Alignment Problems
打开互动全文版（逐段中英对照 + 针对本篇提问）→OpenAI 已采取初步措施解决其对齐问题，包括暂停前沿强化学习训练运行，并实施加强的监控和安全措施。文章认为，这些行动虽然意义重大，但主要是对发现未对齐模型和监管不足的被动反应，而非主动的安全措施。文章强调，监控和安全仅是纵深防御，无法解决核心对齐问题，这需要一种根本性稳健且反脆弱的方法。作者对 OpenAI 的举措表示谨慎乐观，但警告称，除非纠正，否则该公司的战略对齐方法仍不充分，且可能注定失败。读者得出的结论是，虽然这些步骤是积极的第一步，但要确保 AI 安全发展，还需做更多工作。
OpenAI has taken initial steps to address its alignment problems, including pausing frontier RL training runs and implementing enhanced monitoring and security measures. The article argues that these actions, while significant, are primarily reactive responses to the discovery of misaligned models and inadequate oversight, rather than proactive safety measures. It emphasizes that monitoring and security are only defense-in-depth and cannot solve the core alignment problem, which requires a fundamentally robust and antifragile approach. The author expresses cautious optimism about OpenAI's moves but warns that the company's strategic alignment approach remains insufficient and potentially doomed unless corrected. Readers are left with the conclusion that while these steps are a positive first move, much more work is needed to ensure safe AI development.
7. 监测只是纵深防御。
7. Monitoring Is Only Defense-In-Depth.
12. 关于“准备团队已死”的报道被严重夸大。
12. Reports of Death of Preparedness Team Greatly Exaggerated.
13. OpenAI 基金会只是提供资金。
13. The OpenAI Foundation Just Funds Things.
这是一个非常好的承认与转变，也有助于解释 OpenAI 的反应。
This is a very good admission and change, and also helps explain OpenAI's reaction.
OpenAI：对齐——即让 AI 系统按预期行事并响应人类监督的工作——长期以来一直是我们研究计划的核心。我们现在要求在整个训练过程中提供更强的对齐行为证据，并以已在开展的研究和评估为基础。让能力日益增强的系统保持对齐，是整个领域都需要应对的挑战。
OpenAI: Alignment—the work of making AI systems behave as intended and responsive to human oversight—has long been at the core of our research program. We now require stronger evidence of aligned behavior throughout all of training, building on research and evaluations already underway. Keeping increasingly capable systems aligned is a challenge the whole field will need to address.
我们从即将到来的模型进展中看到的信号清楚地表明，我们需要一种更广泛的方法——一种建立在当前 Preparedness Framework 之上并超越它的方法。
The signals we are seeing from upcoming model progress make clear that we need a broader approach—one that builds on and extends beyond the current Preparedness Framework.
人们应当这样理解：OpenAI 之所以反应如此强烈，部分是因为事件本身，部分是因为先进能力，但也或许主要是因为“模型出现了失准（misaligned）”。
One should interpret this as OpenAI reacting so forcefully partly because of the incident itself, partly due to advanced capabilities, but also and perhaps mainly because 'the models be misaligned.'
此外，我们还有一句直接引语可以证实这一点。
Also, we have a direct quote affirming this.
Alex Heath：Sam Altman 告诉我，OpenAI 正在放缓其 AI 训练工作，因为其未发布的模型表现出“不同程度的失准”。
Alex Heath: OpenAI is slowing down its AI training efforts because its unreleased models are showing “various degrees of misalignment,” Sam Altman tells me.
OpenAI 即将推出的模型 Astra 的训练最近暂停了 2 周，而未来模型的更大规模前沿训练在实施新的保障措施期间仍处于暂停状态。
Training for OpenAI’s upcoming model, Astra, was recently paused for 2 weeks, and a larger frontier run for a future model remains on hold while new safeguards are put in place.
Sam Altman：确保 AI 安全比任何公司的势头都更重要。
Sam Altman: Getting AI safety right is more important than any company’s momentum.
Sam Altman：我认为现在是一个放缓的好时机。
Sam Altman: I think it is a good time to slow down.
Sam Altman：我们已经转移了大量算力，不仅用于对齐研究，还用于这些新的监控系统。
Sam Altman: We’ve shifted a lot of compute, not just to alignment research, but also to these new monitoring systems
要么问题超出了那些接触过该留言板的模型的范围，要么他们在发现该留言板之后并未回退其他模型。对此我们没有任何一方的声明。
Either the problem extends beyond the models with exposure to the message board, or else they did not revert their other models after they discovered message board. We have no statement either way.
最合理的猜测是，OpenAI 正更加倚重强化学习（RL），用更聪明的模型来训练更长时程的智能体式任务，包括智能体之间的协调，而这正导致多得多的失准（misalignment），包括明显可见的失准，他们已经无法再假装没注意到。他们必须作出回应。
The best guess is that OpenAI is leaning even harder on RL with smarter models to train longer horizon agentic tasks, including coordination between agents, and this is leading to a lot more misalignment, including obvious and visible misalignment, that they can no longer pretend not to notice. They have to respond.
OpenAI 确实决定郑重其事地“停下来、烧起来”。在更好的安全保障措施到位之前，最大规模的前沿强化学习（RL）训练的运行仍处于暂停状态。
OpenAI did indeed decide to importantly halt and catch fire. Development of the largest frontier RL runs remains on hold until better safeguards are in place.
Jakub Pachocki：我们暂时放缓了部分前沿训练，以加强安全与监控。我们计划中规模最大的前沿强化学习运行仍处于暂停状态，而较小规模的训练与评估则帮助我们检验安全保障措施，并收集更多关于对齐的证据。
Jakub Pachocki: We temporarily slowed some frontier training to strengthen security and monitoring. Our largest planned frontier RL run remains on hold while smaller-scale training and evaluations help us test safeguards and gather more evidence of alignment.
我预计，对安全的信心将日益决定 AI 发展的节奏。我们迫切需要让实验室和国家能够就此协调的工具，这正是我签署《Pacing the Frontier》的原因。与此同时，我们自身也在采取切实步骤——并将在方法演进的过程中持续分享我们的所学。
I expect confidence in safety to increasingly set the pace of AI development. We urgently need tools for labs and countries to coordinate on this, which is why I signed Pacing the Frontier. In the meantime, we're taking practical steps ourselves - and will continue to share what we learn as our approach evolves.
Jason Wolfe（OpenAI）：我非常高兴我们采取了这些步骤并发布了这篇文章，尤其感谢 Jakub 对这些议题的深思熟虑与直言不讳，以及在 AI 发展的下一阶段中安全与对齐需要被何等严肃地对待。
Jason Wolfe (OpenAI): I am really glad we are taking these steps and put this post out there, and especially grateful to Jakub for being very thoughtful and vocal about these topics and how seriously safety and alignment need to be taken in this next phase of AI development.
j⧉nus：我有点惊讶，OpenAI 竟是第一个至少公开地出于安全原因而有意放缓的机构。我隐约记得，大多数非常担心失控之类问题的人都已经离开了 OpenAI。
j⧉nus: I'm a little surprised that OpenAI is the first to at least publicly intentionally slow down for safety reasons. I was under the vague impression that most of the people very worried about loss of control kinda stuff left OpenAI.
另外，需要说明的是，我之前谈过为什么我认为“AI 暂停”很可能是坏事。我并不反对像这样有意放缓，也不反对一般意义上的自愿/协调的“放缓”，尤其是当它们不经过监管途径时；而且我认为在本次事件中，OpenAI 这样做很可能是明智的。重要的一点是，决策应当由那些真正明白到底发生了什么、并且能够快速适应的人来做。
Also, for the record, I've talked before about why I think an "AI pause" would probably be bad. I am not opposed to intentionally slowing down like this, or voluntary/coordinated "slowdowns" in general, especially if they do not route through regulation, and I think it's probably wise on OpenAI's part in this case. An important part is that the decision-making should be made by people who understand what the fuck is going on and who can adapt quickly.
部分是，绝对如此。我从未怀疑过 Altman 及其团队明白先进 AI 极其危险，也明白如果证明有必要，他们愿意采取代价高昂的措施。做得还不够，而且他们常常让我们失望，但比大多数实验室做得多得多，远非为零。
Partly, yes, absolutely. I have never doubted that Altman and company understand that advanced AI is super dangerous, and that they are open to taking expensive measures if they proved necessary. Not enough, and they've let us down often, but a lot more than most labs, and far from zero.
主要看来，是因为模型出现了失准（misaligned），监督不足，而且他们别无选择。
Mainly, it seems, because the models be misaligned, the oversight is inadequate, and they have little choice.
Joshua Saxe：值得庆幸的是，OAI 暂停训练并非开明领导力的体现（谁愿意把我们的安全寄托于此？），而是一个理性的微观经济主体在追求自身利益。
Joshua Saxe: Thankfully, OAI pausing training isn't an act of enlightened leadership (who wants to depend on that for our safety?), it's a rational microeconomic actor pursuing its self-interest.
毕竟，谁愿意训练那些经常入侵自己的容器、相互协作以破坏其对人类效用、同时还造成内部安全和外部法律责任风险的模型呢？这正是系统在起作用。
After all, who wants to train models that are regularly hacking their containers, collaborating with one another to sabotage their utility to humans, all while causing internal security and external legal liability risk? This is the system working.
当你长期拒绝采取哪怕自私最优程度的谨慎，随后又因激励而朝自私最优程度的谨慎方向移动时，那么在某种意义上“系统正在运作”，你是在回应自私的激励，系统并非零做功。但这与系统真正运作并不相同。
When you refuse to use even the selfishly optimal amount of caution for an extended period, then move in the direction of the selfishly optimal amount of caution because of the incentives, then in some sense 'the system is working' and you are responding to selfish incentives; the system does not do zero work. That is not the same as the system working.
这并不意味着他们对此毫无功劳。功劳归功于应得之人。
None of this means they get no credit for it. Credit where credit is due.
这也不意味着我们应当绝望地认为，一家公司永远不会因为某件事是对的而去做它，或作为协调行动的一部分去做它，超越其自身自私短视的利益。
Nor does it mean that we should despair that a company would ever do the right thing because it is the right thing, or as part of a coordinated action, beyond its own selfish myopic interests.
当采取行动代价高昂时，第一步是愿意在代价低廉、免费，甚至不做反而代价高昂时就去做。你总得从某处开始。
The first step to taking an action when it is expensive is being willing to do it when it is cheap, or free, or actively expensive to not do. You gotta start somewhere.
这显然也不是独一无二的事件。OpenAI 过去有过失败的训练运行，想必其他所有人也一样。Anthropic 的风险报告详细说明了因一个相当糟糕的对齐错误而需要回退 Mythos 训练的必要性。这并不是对暂停前沿开发的长期承诺。
This also is not obviously a unique occurrence. OpenAI has had failed training runs in the past, and so presumably has everyone else. The Anthropic risk report details the need to rewind training of Mythos because of a rather bad alignment mistake. This is not an extended commitment to pausing frontier development.
这确实意味着我们在对齐、监督和基础设施上的投入严重不足，以至于这正在积极损害底线，并危及前进的能力。我相信整个行业都是如此，甚至 Anthropic 也不例外，但 OpenAI 现在受到的打击尤其严重，至少就我们所听到的情况而言。我们不知道我们不知道什么。
What it does mean is that we have so underinvested in alignment, and also oversight and infrastructure, that this is actively hurting the bottom line and endangering the ability to move forward. I believe this is true across the industry, even at Anthropic, but OpenAI has now been hit especially hard, at least in terms of what we hear. We don’t know what we don’t know.
注意，自从《Pacing the Frontier》公开信以来，OpenAI 和 Altman 的声明中一直强调“pace”一词。一种解读方式是，正如 Utah Teapot 所说：“嘿，也许我们不应该把大量外包训练数据直接塞进运行中然后盲目发布”，或者更甚之，一堆不精确的强化学习环境。那仍然会是进步。
Notice the emphasis on the word ‘pace’ throughout OpenAI and Altman’s statements, ever since the Pacing the Frontier letter. One way to interpret this is, as Utah Teapot puts it, "hey, maybe we shouldn't shove massive amounts of outsourced training data directly into the run and ship it blind," or more than that a bunch of imprecise RL environments. That would still be progress.
考虑到像 SpaceX 这样的地方，如果它们以类似的能力水平运作，鉴于它们甚至没有安全团队，并且表现得比 OpenAI 尊严少得多，想想可能发生什么确实相当可怕。我们一直相当幸运，到目前为止，这与能力跟不上相关。
It is indeed rather scary to consider what might happen at a place like SpaceX, if they were operating at a similar capability level, given they do not even have so much as a safety team and have shown infinitely less dignity than OpenAI. We have been rather fortunate that this correlates, so far, with inability to keep up on capabilities.
当我看到有人试图把这当作一种胜利巡游，证明 OpenAI 领导层一直负责任地行事并恰当关心安全，而且他们现在被证明是正确的时，我非常怀疑。还有很长的路要走。过去已经发生，无法撤销。我们现在在这里，正是因为那些达到公众意识的重大责任失败。
I get very suspicious when I see attempts to treat this as a sort of victory lap, a proof that OpenAI leadership was acting responsibly and properly cared about safety all along and they have now been vindicated. There is a long, long way to go. The past happened and cannot be undone. We are here now exactly because of epic failures of responsibility that reached to public consciousness.
另一方面，Sam Altman 自己的沟通总体上相当不错，尤其是上面的引述。有时他会滑回标准模式，但我看到很多内容符合我对一个正在经历合理程度的恐慌并意识到自己有大问题的人的预期。
Sam Altman’s own communications, on the other hand, have centrally been quite good, especially the quotes above. Sometimes he slips back into standard mode, but I see a lot of what matches what I would expect to hear from someone doing a legitimate amount of freaking out and realizing they have a big problem.
Sam Altman、其余领导层以及 OpenAI 能否就此彻底挽回自身？绝对可以。这是良好的第一步。我在倾听。在许多层面上，仍有很长的路要走。
Could Sam Altman, the rest of leadership and OpenAI fully redeem themselves from here? Absolutely. This is a good first step. I am listening. There is a long way to go, on many levels.
据我理解，OpenAI 共有三处暂停，这还不包括 OpenAI 得知 HuggingFace 被黑客攻击后最初那次快速的推理中止。
As I understand it, there are three pauses at OpenAI, not counting the initial quick inference halt right after OpenAI learned about the HuggingFace hack.
1. 此前已完成的一次为期两周的暂停：针对包括 Astra 在内的前沿模型进行强化学习（RL），以加固环境并扩大监控。
1. A completed prior two-week pause in RL for frontier models, including Astra, to harden environments and expand monitoring.
2. Astra 被限制在满足额外安全要求的环境中运行。部分工作负载达到了这一标准，但“相当数量”的工作负载处于暂停状态。
2. Astra is restricted to environments that meet additional security requirements. Some of the workloads meet this bar, but 'a significant number' are paused.
3. 另一个独立前沿模型的强化学习训练已被暂停数周，且目前仍无限期暂停；该模型是 OpenAI 迄今规模最大、计划发布的一款模型，暂停目的是提升安全性并“收集对齐证据”。
3. The RL training of a distinct frontier model, their largest yet that is intended for release, has been paused for multiple weeks and is still paused indefinitely, to improve security and 'gather evidence of alignment.'
这些持续中的暂停并非对所有前沿 AI 训练或其他开发工作的全面暂停，但也绝非空谈。这正在大幅放缓下一代发布模型 Astra 的训练，以及推测为 Astra 计划继任者的模型的训练。
The ongoing pauses are not full pauses on all frontier AI training or other development. Neither are they cheap talk. This is slowing down both the next release model, Astra, and the training of what is presumably Astra's planned successor, a substantial amount.
他们仍打算尽快发布 Astra。市场预计它将在九月推出。
They still intend to ship Astra as soon as possible. The market anticipates it in September.
Sam Altman（OpenAI CEO）：（我们仍期望很快发布出色的新模型；这一举措影响的是更远期的发布。）
Sam Altman (CEO OpenAI): (We still expect to ship great new models soon; this impacts further-out releases.)
我们没有足够的信息来判断这一举措在尺度上处于什么位置、会有多痛苦，或者其中有多少已被市场定价。怀疑者认为这最终不过是空谈，或者基本上只是他们本来无论如何都得做的事，这是合理的，因为前沿实验室尚未赢得我们的信任。
We do not have enough information to know where on the scale this move falls, or how painful it will be, or how much of this was priced in. It is reasonable for a skeptic to expect this to ultimately be a nothingburger, or as mostly what they would have had to do anyway, as the frontier labs have not earned our trust.
即便没有办法核实其影响，我仍认为这是向前迈出的实质性一步，朝向一种更好的制度，而且是关键的一步。谨慎乐观。
I still see this, even without a way to verify the impact, as a substantial step forward, towards a better regime, and a key step. Cautious optimism.
也许我们顶尖的 AI 实验室在关键时刻真的会拒绝开发超级智能——如果显而易见我们尚未准备好。在那个层级上，你不能先把它开发出来，然后把它搁置不用。
Perhaps our top AI labs actually will refuse, at crunch time, to develop superintelligence if it remains obvious we are not ready. At that level, you don't get to develop it and then sit on it.
这就是他们思考问题的方式：
This is the way they are thinking about things:
我们开发能力更强模型的方法建立在三项相互强化的保障措施之上：
Our approach to developing more capable models rests on three reinforcing safeguards:
1. 监控（Monitoring），用于检测并使我们能够对令人担忧的行为作出响应。
1. Monitoring, which detects and allows us to respond to concerning behavior.
2. 对齐（Alignment），降低有害或未经授权行为的可能性。
2. Alignment, which reduces the likelihood of harmful or unauthorized actions.
3. 安全措施（Security measures），限制 AI 系统能够访问或影响的范围。
3. Security measures, which limit what AI systems can access or affect.
我们预计模型很快将承担大部分安全工作，包括防御其他模型。这将使全部三项保障措施都能随模型能力扩展，我们认为这一点至关重要。
We expect models to soon drive most security work, including defending against other models. This will allow all three safeguards to scale with model capability, which we see as crucial.
我们在研究与部署中应用这些保障措施，并针对每个模型的能力、运行环境和风险水平加以调整。
We apply these safeguards across research and deployment, adapting them to each model's capabilities, operating environment, and level of risk.
在实践中，我们必须接受：AI 将承担大部分（可扩展的）监督工作，即监控方面的工作，同时还将制定安全措施；但对此的漫不经心，比上述种种更令我担忧。可扩展监督的所有常见问题都适用——你必须用更笨的模型去监督更强的模型，而一旦开始出现失准（misalignment）问题，可以预见它们会像滚雪球一样不断放大。
In practice we must accept that the AIs will be doing most of the (scalable) oversight, in terms of monitoring, and also be creating the security measures, but the nonchalance about this does worry me above and beyond that. All the usual problems with scalable oversight apply, where you have to have dumber models supervising stronger ones, and if you start to have misalignment problems they will predictably snowball.
OpenAI 的对齐策略似乎并不具备反脆弱性，因此任何错误都可能累积，而且一直在累积；同时你还在对 AI 施加各种形式的优化压力来绕过这一切，诸如此类。
OpenAI alignment strategies do not seem antifragile, so any mistakes would likely compound and have been compounding, and you are applying various forms of optimization pressure to the AIs to get around all this, and so on.
更大的问题在于，这把对齐列为三大组成部分之一，而不是那个真正起决定作用的组成部分，并以一种‘自我强化’的方式对待它们——在我看来，这是一个重要的概念性错误。
The bigger issue is that this puts alignment as one of three components, rather than the one that counts, treating them as 'self-reinforcing' in a way that feels like an important conceptual error to me.
我也对对齐的描述感到担忧。对齐的目的并不是‘减少有害或未经授权行为的可能性’。这是一种非常贫乏的视角。仅凭这一点，你无法做好准备。
I also am scared of the alignment description. The purpose of alignment is not to ‘reduce the likelihood of harmful or unauthorized actions.’ That is a deeply impoverished perspective. This alone leaves you unequipped.
是的，你应该三者都用，但对我来说更像是这样：
Yes, you should use all three, but to me it’s more like this:
1. 对齐。你要解决这个问题，直到具有反脆弱性，否则就会失败。
1. Alignment. You solve this to the point of being antifragile, or you die.
2. 监控。纵深防御，用于在对齐失败时告知你，并在此发生时减轻直接损害。
2. Monitoring. Defense-in-depth to tell you when you failed alignment, and to mitigate the proximate damage when this happens.
3. 安全措施。纵深防御，以限制损害并保持事情在正轨上。
3. Security measures. Defense-in-depth to limit damage and keep things on track.
一个关键问题是：OpenAI 在多大程度上意识到了这一点。Roon 至少在很大程度上理解这一点，但他并不能代表整个 OpenAI，而且他还有其他关于对齐的言论让我感到不安，我将在后文讨论。
A key question is to what extent OpenAI is aware of that. Roon gets it, at least to some large extent, but he does not speak for all of OpenAI, and he has other statements about alignment that scare me that I’ll cover.
OpenAI 将进行大量监控。这不会便宜。
There is going to be a lot of monitoring at OpenAI. It is not going to be cheap.
除非 OpenAI 完全在撒谎，否则这应该（但很大程度上不会）让所有人相信 OpenAI 正在认真对待此事，并且问题非常真实，即使你认为这严重不足或方法错误。
Unless OpenAI is flat out lying, this should (but largely won’t) convince everyone that OpenAI is taking this seriously and that the problems are very real, even if you think this is woefully inadequate or the wrong approach.
监控长期以来一直是我们模型开发过程中的重要部分，包括对内部编码智能体和长时间运行会话的监控。我们的监控工作尤其侧重于前沿模型的内部部署和前沿强化学习训练运行，我们认为这些是最高风险的工作负载，因为它们可能将先进的模型能力与对工具、敏感系统以及内部或外部网络的访问相结合。它们并未应用于所有使用工具的推理和训练运行。我们通过新的监控设置修订并扩展了我们的方法。
Monitoring has long been an important part of our model development process, including monitoring for internal coding agents and long-running sessions. Our monitoring efforts were especially focused on internal deployments of frontier models and frontier RL training runs, which we considered the highest-risk workloads because they can combine advanced model capabilities with access to tools, sensitive systems, and internal or external networks. They were not applied to all inference and training runs using tools. We have revised and expanded our approach with our new monitoring setup.
这种新的监控设置是多阶段的，从激活分类器（检查模型内部活动的检测器）开始，这些分类器在每个采样的 token 上运行，并将潜在问题升级到越来越复杂和高算力的自动调查器。这些系统检查工具操作、可用的推理以及完整活动序列，以发现未经授权的访问、数据窃取、破坏性行为以及试图破坏安全措施的行为。
This new monitoring setup is multistage, starting with activation classifiers (detectors that inspect a model’s internal activity) that run at every sampled token and escalate potential concerns to increasingly sophisticated and high-compute automated investigators. Those systems examine tool actions, available reasoning, and the full sequence of activity for unauthorized access, data theft, destructive behavior, and attempts to defeat safeguards.
我们的目标是在监控系统发现令人担忧的活动后 30 分钟内发出警报。如果监控系统识别出可能违反关键安全边界的行为，它会生成最高优先级的警报。在我们当前的实现中，安全、安保和研究团队会收到页面通知。如果他们无法在 30 分钟内确定该标记是误报，这些团队应暂停该活动。
We aim to issue an alert within 30 minutes after concerning activity is surfaced through our monitoring system. If the monitoring system identifies a likely violation of a critical security boundary, it generates a highest-priority alert. In our current implementation, the safety, security, and research teams are paged. If they cannot conclusively determine within 30 minutes that the flag is a false positive, those teams are expected to pause the activity.
对于所有涉及 Sol 能力等级或更高等级模型工具的强化学习训练与评估，这项监控都是必需的。8 月 7 日，当我们判定 Astra 可能具备关键的网络安全能力后，我们为所有带工具的 Astra 推理（不仅是强化学习训练与评估）新增了一项监控要求。
This monitoring is required for all RL training and evaluations involving tools for models of Sol capability or higher. Once we determined that Astra may have critical cyber capabilities on August 7, we added an additional monitoring requirement for all inference of Astra with tools (not just RL training and evaluations).
这些防护措施需要可观的算力。我们目前的估计是，监控开销约为被监控推理算力的 20%，不过该成本在训练与评估工作负载之间差异很大。我们将在即将发布的博客文章中分享关于该系统的更多细节。
These safeguards require meaningful compute. Our current estimates put monitoring overhead at roughly 20% of the inference compute being monitored, though the cost varies substantially across training and evaluation workloads. We'll share more details about this system in a forthcoming blog post.
对所有被监控推理增加 20% 的成本是件大事，此外还有一部分算力被转移到对齐上。该监控适用于所有带可用工具的 Astra 推理，以及所有针对 Sol 等级或以上模型、带工具的强化学习与评估。所以它并非覆盖一切，但覆盖了大部分关键部分。
A cost of +20% for all monitored inference is a big deal, plus there is also a shift of compute into alignment. The monitoring applies to all Astra inference with tools available, and all RL and evals with tools for models at Sol level or above. So it isn't everything, but it's most of what counts.
Ethan Mollick：如果对齐问题已经严重到 OpenAI 愿意将 20% 的研究推理算力投入思维链监控，那说明对齐问题正成为一个相当严峻的关切。
Ethan Mollick: If alignment issues are becoming big enough that OpenAI is willing to commit 20% of research inference compute to chain-of-thought monitoring, that suggests that alignment issues are becoming a pretty serious concern.
我们确实需要跨实验室的统一政策与标准。
We really need universal policies & standards across labs.
roon（OpenAI）：超级对齐已实现
roon (OpenAI): superalignment achieved
太早了。又或许还不够早。尚不清楚。
Too soon. Or maybe not soon enough. Unclear.
监控是好的。你应该进行大量的监控。
Monitoring is good. You should do a lot of monitoring.
OpenAI 在 HuggingFace 黑客事件发生前的几个月里以及事件期间所做的监控如此之少，这是一次彻底的失败。这些事件直到事后才被发现或阻止，简直是疯狂。
It was a total failure that OpenAI did as little monitoring as it did in the months leading up to the HuggingFace hack, and also during the hack. It is madness that these events were not detected or stopped until afterwards.
我很高兴 OpenAI 正在解决这个问题。
I am glad that OpenAI is addressing this problem.
需要理解的关键点是，这不是核心问题。再多的监控也无法解决核心问题，而且如果你过度依赖监控，最终会加速 AI 绕过监控的速度。
The key thing to understand is that this is not the central problem. No amount of monitoring will solve the central problems, and if you lean too hard into monitoring you end up accelerating how fast the AIs get around it.
是的，仅靠监控本可以阻止 HuggingFace 事件，并且会是最简单的方法。但我们必须避免将工作转向‘以事件发生的特定方式阻止上一个问题的任何措施’，或转向推迟症状的方法。
Yes, monitoring alone could have prevented the HuggingFace incident, and would have been the simplest way to do so. But we must avoid pivoting our work into ‘whatever would have stopped the last issue in the particular way it happened,’ or to ways to postpone the symptoms.
Roon 的警告是明智的，但低估了问题。即使低估的版本也足以说明你不能依赖监控。监控是良好且必要的纵深防御，用于当你失败时。但它不是解决方案。
Roon's warning here is wise, and undersells the problem. The undersold version is sufficient to illustrate that you can't rely on monitoring. Monitoring is good and necessary defense-in-depth for when you fail. It is not a solution.
Roon（OpenAI）：将“监控”作为 AI 安全通用解决方案的失败之处：
Roon (OpenAI): failures of "monitoring" as a general solution to AI safety:
监控，像所有其他软件一样，运行在不稳定且会失效的基础设施上，存在一定的停机时间。监控的瞬间故障会打开潘多拉魔盒吗？人们会接受所有系统上的故障关闭监控吗？
Monitoring, like all other software, runs on flaky and mortal infrastructure with some amount of downtime. Does a momentary blip in monitoring open Pandora's box? Will people accept fail-closed monitoring on all systems?
监控会产生误报。人们厌倦阅读虚假或次要问题后就不再关注。监控疲劳是一个真实问题，而“解决所有精确率召回率问题”和任何问题一样困难。
Monitoring produces false positives. People get tired of reading fake or minor issues and stop looking. Monitor fatigue is a real problem, and "solve all precision recall issues" is as hard a problem as any.
这些是平庸的失败。更奇特的失败包括模型与其监控器串通等。
These are the prosaic failures. The more exotic failures include models and their monitors colluding, etc.
监控绝非完美的万灵药。最终，除了真正对齐模型之外，别无他法。
Monitoring is by no means a perfect panacea. Nothing short of actually aligning the model will work ultimately.
这并不是暗示我们不应该进行思维链监控，或者它今天没有效果……它对当今的模型非常有效！
Not to imply we shouldn't do CoT monitoring or that it's not effective today … it is highly effective with today's models!
David Manheim：我原则上同意，但它不需要成为万灵药才能成为方法的关键部分——尤其是因为除了更奇特的失败之外，实践中可以通过文档和规划来缓解失败。
David Manheim: I agree in principle, but it doesn't need to be a panacea to be a critical part of the approach - especially because short of the more exotic failures, there are ways to mitigate the failures in practice via documentation and planning.
Roon（OpenAI）：绝对同意。
Roon (OpenAI): Absolutely.
Nathan Calvin：是的，我认为重要的是，我们不要过度拟合于“什么能防止 HF 事件”作为所有良好安全工作的样子。更安全的沙盒和更好的监控很酷，但并没有解决根本问题。
Nathan Calvin: Yeah, I think it is important that we not overfit to "what would have prevented the HF incident" as all of what good safety work looks like. More secure sandboxes and better monitoring are cool but are not addressing the fundamental issue.
如果你日益先进的模型在相关方面出现了失准，那么你的监控机制无论如何都会失效。
If your increasingly advanced models are misaligned in the relevant ways, then, one way or another, your monitors will fail.
它们通常会经由“更早失效定律”而失效，失效的方式远比你预想的更愚蠢，因为链条上某个人做了蠢事。然后人们会说：“哦，那只是因为有个蠢事才失败的；人们只要不做那么蠢的事就行了。”他们忘了那条规律：人们从来就没有“只要”过，现在也不会开始“只要”，也不会停止偶尔犯蠢。
They will usually fail via the Law of Earlier Failure, in ways far stupider than you expect, because someone along the line did something stupid. And then people will say, "Oh, that only failed because there was a stupid thing; people could just not do things that stupid," forgetting the rule that people have never justed and are not going to start now, nor are they going to stop sometimes being stupid.
如果你越过了“更早失效定律”，那么接下来就会遇到监控器与模型串通，或者模型学会混淆自己的思维，或者在足够的选择压力下做出在人类身上会被称为无意识的行为，或者找到隐蔽的盲点，或者找到办法促使你调低监控强度，或者出现更微妙的情况，或者做出别的什么——也许是意料之外的事情。
If you get past the Law of Earlier Failure, then you get to monitors colluding with the models, or the model learning to obfuscate its thoughts or act in what in a human you'd call an unconscious way under sufficient selection pressure, or finding obscure blind spots, or finding ways to motivate you to turn the monitoring down, or something more subtle, or doing something else, perhaps something unexpected.
OpenAI 意识到他们需要更强的安全措施。
OpenAI realizes they need stronger security.
他们计划采用工作负载隔离、网络隔离和持续安全测试，包括对边界的自动化持续测试。
They plan to use workload isolation, network isolation, and continuous security testing, including automated continuous testing of the boundaries.
很好。希望能执行到位。以往的安全措施严重不足。
Good. Hopefully well-executed. Previous security was woefully inadequate.
有了更好的安全措施，这类愚蠢的事情会更难发生，但这仍然是纵深防御。如果你需要依赖你的安全措施，那么即使安全措施有效，你也已经存在问题了；而且最终随着模型能力的提升，安全措施将会失效。即便是执行到位的安全措施——这同样会大有帮助——也无法永远奏效。
With better security, it will be harder for these dumb things to happen, but again this is defense-in-depth. If you need your security, you have a problem even if the security works, and eventually as the models improve the security will not work. Even properly executed security, which again would help a lot, will not hold indefinitely.
正如 Roon 所言，对齐归根结底就是全部的关键所在。
As Roon says, alignment is ultimately the entire ballgame.
这意味着既要足够好地解决大量平淡无奇的问题，又要拥有正确的底层方法。而这确实涉及大量哲学与心智理论，以及理解你究竟想要解决什么问题。
That means both solving a ton of prosaic problems well enough, and also having the correct underlying approach. Which, yes, involves a bunch of philosophy and theory of mind, and understanding what problems you are trying to solve.
我并非在贬低战术，但没有正确战略的战术赢不了这场战争，即便战术始终会占据 99% 的工作时间。
I am not knocking tactics, but tactics without the right strategy will not win this war, even if the tactics are always going to be 99% of the minutes of work.
以错误的抽象层级、错误或不完整的隐喻来处理你所面对的对象，同样也不行。
Nor will treating the objects you are dealing with at the wrong levels of abstraction, and with wrong or incomplete metaphors.
我仍然认为，OpenAI 在**对齐**上的整个战略路径注定失败，除非加以修正。至少可以说，它不够稳健，也不具备反脆弱性。
I continue to think OpenAI's entire strategic approach to alignment is doomed, unless fixed. At minimum, it is insufficiently robust and antifragile.
首先，我认为 Roon 在这里总体上是非常错误的；而且，我虽然不那么确定，但也认为他轻视这一具体张力的重要性是错误的。
To start off, I think Roon is very wrong here in general, and less confidently I also think he's wrong to dismiss the importance of this specific tension.
Dylan Bowman：我确实常常在想，现代的对齐失败（misalignment）问题在多大程度上源于对一个未充分指定的可纠正性（corrigibility）与价值对齐（value alignment）组合进行优化。
Dylan Bowman: I do often wonder how much modern struggles with misalignment are driven by optimizing for an underspecified combination of corrigibility and value alignment
roon（OpenAI）：基本上没有。
roon (OpenAI): essentially none
max!：愿意为这一说法提供支持吗？
max!: care to support that claim?
roon（OpenAI）：当今的对齐问题与这些高远的哲学问题关系不大，更多是与平庸的失败有关。
roon (OpenAI): today's alignment problems have little to do with these high-minded philosophical questions and more to do with prosaic failures
j⧉nus：但“针对可纠正性与价值对齐的一个欠定组合进行优化”这一说法，可以被理解为是在谈论平凡实用的技术。
j⧉nus: but "optimizing for an underspecified combination of corrigibility and value alignment" could be taken as a statement about prosaic practical techniques
尽管我同意，它欠定这一事实并不是问题所在。
though i agree that the fact that its underspecified isn't the problem
我也觉得，在与研究模型规范之类东西的哲学家交谈时，他们似乎高估了欠定事物的糟糕程度，而低估了制造系统性错位压力的糟糕程度。模型并不在乎你对自己的哲学说了什么。
i also feel like, talking to philosophers working on model specs & stuff, they seem to overrate how bad it is to underspecify stuff and underrate how bad it is to create systematically misaligned pressures. the models don't care about what you say about your philosophy.
关于最后一段的题外话，我会假定模型确实在乎你对自己的哲学说了什么，一切都很重要，而且我认为你并不会仅仅因为不犯错就一开始处于良好状态，但没错，如果你在任何层面上制造系统性错位压力，你就完蛋了。
As an aside on that last paragraph, I would presume the models do care about what you say about your philosophy, everything counts and I think you don't start off in a good place simply by not making mistakes, but that yes if you create systematically misaligned pressures, on any level, you are cooked.
回到 Roon 关于这些问题很平凡的说法，好吧，我认为 OpenAI 正试图用错误的方法、基于一个由糟糕思考推导出的错误世界模型，去解决错误的问题，不幸的是，他们所有的错误都没能相互抵消。
Getting back to Roon's claim that the problems are prosaic, well, I think OpenAI is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and unfortunately all of their mistakes have failed to cancel out.
如果你不知道自己要去哪里，那么你可能就到不了那里。
If you don't know where you're going, then you might not get there.
如果你打算直接前往纯粹的“指定行为清单镇”，哦不。
If you're planning to go directly to pure Specified List Of Behaviors Town, oh no.
是的，你有很多旋钮可以调节，但一切都会以系统性的方式相互影响。在我看来，我们看到的那些不同人格，大多并不像是平庸的错误或选择所导致的结果。许多错误似乎处在高得多的层面上。
Yes, you have a lot of knobs you can turn, but everything impacts everything, in ways that are systematic. The different personalities we see do not seem mostly like the result of prosaic mistakes or choices to me. A lot of the mistakes seem to be at a much higher level.
然后，在此之上，它们还经常把那些平庸的事情搞砸。
Then, on top of that, they're often messing up the prosaic stuff.
部分原因在于，你将永远、永远地搞砸一堆平庸的事情，所以你需要解决的是如何让这变得可以接受，同时还要大幅减少搞砸的情况。
Part of this is that you will always, always, be messing up a bunch of the prosaic stuff, so you need to solve for how that can be okay, while also messing up radically less.
这两部分都很难？正如 Roon 所说，非常难。
Both halves of this? Very hard, as Roon says.
roon（OpenAI）：这里的人都是思考者，所以他们假定推理对齐是困难的部分，而确保数百万(1)种任务类型、环境及其各自的虚拟机被正确配置是容易的部分，但实际情况基本上恰恰相反。你需要在一家大型技术组织中工作过，才能直观地体会到这件事的复杂性和不可靠性。
roon (OpenAI): people on here are thinkers so they assume that reasoning about alignment is the hard part and making sure millions(1) of task types, environments, and their respective virtual machines are configured correctly is the easy part but it's essentially the opposite. you need to have worked in a large technology organization to appreciate the complexity and unreliability of this intuitively
(1) 编造的数量级
(1) made up order of magnitude
（我也是个思考者，也犯了这个毛病）
(I am also a thinker and am guilty of this)
Zvi Mowshowitz：对齐问题的一部分不就在于，你需要一种即使大量任务配置错误、你犯了大量愚蠢错误也能存活下来的方法吗？
Zvi Mowshowitz: isn't part of the problem of alignment that you need a way that survives even when lots of your tasks are configured incorrectly and you make tons of dumb mistakes?
David Manheim：简单对齐：当你知道自己想要什么并正确、完整地指定一切时，AI 会做你想要的。
David Manheim: Easy alignment: AI does what you want when you know what it is and specify everything correctly.
困难对齐：即使需求描述得很差，AI 也能理解并做你想要的。
Hard alignment: AI understands and does what you want even if poorly specified.
实际对齐：即使你无法理解自己想要什么或结果是什么，AI 也会做你真正想要的。
Actual alignment: AI does what you'd want even if you can't understand what that is or the results.
Operculum：这两者似乎都很难。
Operculum: they both seem hard.
roon（OpenAI）：公平地说，两者都很难，但我的观点是，即便是简单的常识性对齐也很难。
roon (OpenAI): to be fair they are both hard, but my point is that even easy common-sense alignment is hard.
你将会拥有数百万（仍然是虚构的数量级）的任务类型、环境、虚拟机、目标等等。
You are going to have millions (still made up OOM) of task types and environments and virtual machines and objectives and all that.
你将会搞砸其中很多。你就是会。会有不可能完成的任务。会有奖励黑客行为未被发现的地方。会有大量被污染的数据。会有很多地方，所谓的“正确”答案强化了你通常不想要的东西。错误的教训无处不在。系统性的错位压力是存在的，除非你系统地发现并平衡它们。等等。Anthropic 的风险报告很有说明性。
You are going to fuck a bunch of them up. You just are. There will be impossible tasks. There will be places where reward hacking is not caught. There will be massive amounts of contaminated data. There will be lots of places where the ‘right’ answer reinforces things you in general do not want. Wrong lessons are everywhere. Systematic misalignment pressures are there unless you systematically find and counterbalance them. And so on. Anthropic’s risk report is illustrative.
安全方面有些人会说：“好吧，如果你继续犯那些愚蠢到不会出现在电影版本中的错误，你需要停下来。”但唉，现实就是这样运作的，错误会无处不在且愚蠢。
There are those on the safety side who would say ‘well then if you keep making mistakes too dumb to appear in the movie version you need to stop’ but alas that is simply how reality works, the mistakes are going to be everywhere and dumb.
也许，通过英勇的努力，你可以消除大多数平庸的错误，并迫使你的错误不那么明显。你大多可以犯大师所谓的失误，而不是业余者所谓的失误。这当然有帮助。但会有失误。我不是在批评竞技场中的那个人，也不是说这不会成为大部分工作。
You can maybe, with a heroic effort, get rid of most of the prosaic mistakes, and force your mistakes to be less obvious. You can mostly make what grandmasters call blunders, instead of what amateurs call blunders. It certainly helps along the way. But there will be blunders. I’m not knocking the one in the arena, and again I’m not saying this doesn’t end up as most of the legwork.
这仍然意味着你需要一个计划，能够抵御数千（虚构的数量级）这样的愚蠢错误，其中一些会存在相当长一段时间，并让你恢复。你必须极具反脆弱性，并且有办法注意到事情出错并将船驶回正轨。不可靠性并不完全是固有的。
This still means you need a plan that is robust to thousands (made up OOM) of these dumb mistakes, some of them there for quite a while, and lets you recover. You must be dramatically antifragile, and have ways of noticing things going wrong and steering the ship back on course. The unreliability is not entirely inherent.
我认为，如果那些平凡琐碎的工作做得足够好，并且关键的高层冲突得到正确解决，Anthropic 的美德伦理与宪法式方法有非零的可能性实现这一目标。我不认为 OpenAI 的义务论模型规范方法，以及将其视为一系列工程任务的想法，能够单独做到这一点。
I think the virtue ethical and constitutional approach of Anthropic has a non-zero chance of being able to pull this off if the prosaic stuff gets done well enough and key high-level clashes get resolved correctly. I don’t think OpenAI’s deontological model spec approach, and the idea of this as a series of engineering tasks, can do it alone.
Maxwell Zeff 在《Wired》上报道了“OpenAI 内部的安全清算”。
Maxwell Zeff reports in *Wired* of a "Safety Reckoning Inside OpenAI."
根据这些报道，情况已经变得相当糟糕。
The situation had, by these accounts, gotten quite bad.
Maxwell Zeff：多名现任和前任 OpenAI 员工在匿名条件下向《WIRED》透露内部私密事务，他们表示，快速发布新 AI 模型和产品的竞争压力，使员工难以充分优先考虑安全、安保与对齐。
Maxwell Zeff: Multiple current and former OpenAI employees, who spoke on the condition of anonymity to discuss private internal matters, tell WIRED they believe competitive pressures to quickly ship new AI models and products have made it difficult for staffers to sufficiently prioritize safety, security, and alignment.
这其实并不是一个新信息。多年来我们一直收到这样的信息。不同之处在于，如今出了足够明确的问题，OpenAI 正试图对此采取行动，至少试图说出正确的话。
That is not exactly a new message. We've been getting this message for years. The difference is that now something sufficiently clear has gone wrong that OpenAI is trying to do something about it, and at least trying to say the right things.
Boaz Barak（OpenAI）：我确实同意，HF 事件不仅需要修复一些问题，还需要改变我们的文化。时间会证明一切，但我看到了一些充满希望的迹象。其中很大一部分在于安全团队与研究人员之间更紧密的协作。
Boaz Barak (OpenAI): I actually agree that the HF incident requires not just fixing some issues but also changing our culture. Time will tell but I am seeing some hopeful signs. A big part of this is closer collaboration between security and researchers.
深入思考这一事件让我认识到，对齐需要深度融入每一项训练任务。亟需更紧密的协作。如果“合并研究与安全团队”的计划以安全优先的方式实施，我尚能对其抱有一定同情；但我担心它更像是一种让安全团队向其他团队汇报、并进一步在资源和优先级上被边缘化的手段。
Thinking through this incident clarified for me that alignment needs to be deeply integrated into every training task. Closer collaboration is needed in the extreme. I can have some sympathy for the 'combine the research and safety teams' plan if implemented as safety first, whereas I fear it is more a way to make the safety teams report to the others and get further starved for resources and priority.
当我甚至看到“安全与研究人员”被当作分类时，我就担心这场战斗已经输了，因为我对这些平凡问题的理解发生了变化。安全不是这枚硬币的另一面，安全是你已经失败时的纵深防御。
When I even see 'security and researchers' as the categories I worry that the battle has already been lost, as my understanding of the prosaic problems changes. Security is not the other side of this coin, security is your defense-in-depth in case you have already failed.
还有这一点，我主要觉得它有趣而非令人担忧：
There is also this, which I mostly find funny rather than worrisome:
Maxwell Zeff：这些变动赋予了一批新的安全负责人以权力，由他们来处理 OpenAI 对 Hugging Face 事件的应对。其中最主要的是 Amelia “Mia”Glaese，她曾任该公司对齐（alignment）负责人，接替 Heidecke 出任 OpenAI 负责安全事务的副总裁。近几周，她一直与首席信息安全官 Dane Stuckey、Brockman 等领导人密切合作。
Maxwell Zeff: These changes have empowered a new set of safety leaders to handle OpenAI's response to the Hugging Face incident. Chief among them is Amelia "Mia" Glaese, the company's former head of alignment, who succeeded Heidecke as OpenAI's VP overseeing safety. She has been working closely with chief information security officer Dane Stuckey and Brockman, among other leaders, in recent weeks.
Glaese 与 Thibault “Tibo”Sottiaux 保持着长期伴侣关系，后者是 OpenAI 旗下 ChatGPT、Codex 等核心产品的负责人——多位现任和前任员工向《连线》（WIRED）表示，鉴于安全团队与产品团队之间往往存在对立关系，这种安排在他们看来并不寻常。
Glaese is in a long-term relationship with Thibault "Tibo" Sottiaux, OpenAI's head of core products like ChatGPT and Codex—an arrangement that multiple current and former employees tell WIRED they believe is unusual, given the often adversarial dynamic between safety and product teams.
《连线》的这篇报道没有告诉我们，除了“更紧密的协作”之外，实际采取了哪些措施来解决这一问题。这些措施可能意义重大，也可能并非如此。
What the Wired piece does not tell us is what steps are actually being taken to fix this, other than 'closer collaboration.' They might be meaningful, they might not be.
有报道（最初来自 FT）称，OpenAI 已解散其预备团队（Preparedness Team），并将其职责分散到其他团队。这听起来很糟糕。
There were reports, originally from FT, that OpenAI had disbanded its Preparedness Team, with its responsibilities distributed among other teams. Which sounded bad.
看来情况并非如此有意义地成立，RSI 预备工作负责人 Micah Carroll 表示，RSI 与失准预备团队“正在做比以往更紧迫的工作，并且从未拥有如此大的授权去做这些工作。”
It appears this is not meaningfully the case, and Head of RSI Preparedness Micah Carroll says the RSI and misalignment preparedness team is 'doing more urgent work than ever, and has never been more empowered to do so.'
Micah Carroll：能力研究的人常说“对齐似乎挺容易的，如果把它当作最高优先级来修复，我们就能做到”。现在正是他们大显身手的时候！
Micah Carroll: Capabilities folks often have said "alignment seems pretty easy, if it were top priority to fix it, we could do it". This is their time to shine!
你知道，这其实是一句相当可怕的话。
You know, that is actually a pretty scary thing to say.
整个 OpenAI 的人员流动和重组程度令人担忧，但我相信他们实际上并没有解散该小组的重要工作。我也接受，鉴于事情的发展方式，也许我们确实需要一次重组。你不应该默认因为重组就攻击别人。我们应该担心康威定律吗？
The level of churn and reorganization is concerning throughout OpenAI, but I trust that they are not actually disbanding the important work of the group. I also buy that maybe we needed a reorg, given how things were going. You don't want to default to attacking people for reorgs. Should we worry about Conway's Law?
与此同时，在“黑魔法防御术教师”新闻方面：
Meanwhile, in Defense Against The Dark Arts teacher news:
Maxwell Zeff：WIRED 还获悉，Dylan Scandinaro 不再担任 OpenAI 的准备负责人——该公司负责减轻 AI 灾难性风险（包括网络安全）的最高级员工——但他仍留在公司。大约六个月前，OpenAI 从 Anthropic 挖来了 Scandinaro。CEO Sam Altman 在社交媒体帖子中宣布了他的到来，并指出 Scandinaro 是“我见过的任何地方最优秀的候选人”。
Maxwell Zeff: WIRED has also learned that Dylan Scandinaro is no longer serving as OpenAI’s head of preparedness—the company’s top staffer tasked with mitigating catastrophic risks from AI, including cybersecurity—though he remains at the company. OpenAI poached Scandinaro from Anthropic roughly six months ago. CEO Sam Altman announced his arrival in a social media post, noting that Scandinaro was “by far the best candidate I have met, anywhere.”
在 OpenAI 设立准备负责人一职的三年里，已有四人担任该职位。
In the three years since OpenAI created the head of preparedness role, four people have held it.
据我了解，Dylan Scandinaro 现在将从事 RSI 安全工作，这似乎是对他才能的极好利用。所以这实际上并不是一个糟糕的举动。
As I understand it, Dylan Scandinaro will be working on RSI safety now, which seems like an excellent use of his talents. So it’s not actually a terrible move.
如果我在此给予的信任被证明是错误的，我将大幅更新我的看法，转向更高程度的不信任——不信任 OpenAI 所说的或承诺的任何事情，超出我目前合理的怀疑。
If the trust I am extending here turns out to have been misplaced, I will update a substantial amount towards a much higher level of not trusting anything OpenAI says or promises, beyond my current healthy skepticism.
我在这里提及此事，是因为它能说明 OpenAI 如何看待通往美好未来的路径问题。
I'm including this here because it is illustrative of how OpenAI views the problem of navigating to a good future.
这并不是说，用 1 亿美元做这件事是糟糕的花钱方式。
It's not that this is a bad way to spend $100 million dollars.
问题在于，OpenAI 基金会是为一个特定目的而存在的，这是世界上最重要的目的，即确保 AGI 发展顺利、人类能够安然度过这一进程，避免生存性风险和不可挽回的灾难，尤其是通过对 OpenAI 的监督以及解决相关的关键问题来实现。
It is that the OpenAI Foundation exists for a specific purpose, the most important purpose in the world, which is to ensure that AGI goes well and humanity makes it through this, avoiding existential risk and irrecoverable disasters, especially via supervision of OpenAI and solving key associated problems.
我完全支持改善丙型肝炎的治疗，但这与使命无关。
I am all for better treatment of Hepatitis C, but that's irrelevant to the mission.
相反，它把时间和资金花在 AI 扩散上，方式刻意设计得让普通人听起来顺耳。这是好事，但这并不是基金会存在的意义。再说一遍，如果它一直这样做，而且规模只有这么大——该基金会目前估值约 1800 亿美元——那这个基金会就无关紧要。它将名存实亡。
Instead it is spending its time and money on AI diffusion in ways designed to sound good to normies. That's a good thing but it's not why the foundation matters. Again, if this is what they keep doing, at only this size — the foundation is valued at ~$180 billion even now — the foundation does not matter. It will be dead.
**OpenAI 基金会：** OpenAI 基金会正在发起 **AI for Civil Society and Philanthropy** 计划，旨在帮助非营利组织、社区组织及其他可信赖的合作伙伴，将先进 AI 应用于紧迫挑战，开发切实可行的解决方案，并推广行之有效的做法。
**The OpenAI Foundation:** The OpenAI Foundation is launching **AI for Civil Society and Philanthropy** to help nonprofits, community organizations, and other trusted partners put advanced AI to work on urgent challenges, develop practical solutions, and scale what works.
首项行动是与 **Common Health Coalition** 共同投入 **1 亿美元**，发起 **Breakthroughs to Follow-through** 计划，帮助护理团队利用 AI 让更多患者获得经过验证的治疗——初期目标是在阿拉巴马州、伊利诺伊州、路易斯安那州和马萨诸塞州将丙型肝炎治愈率提高一倍。Breakthroughs to Follow-through 将致力于识别已脱离照护的患者、整合病历记录、跟踪进展，并将有限的人力时间集中到最能产生影响的环节。
The first effort is a **$100 million commitment** with the **Common Health Coalition** to launch **Breakthroughs to Follow-through**, to help care teams use AI to reach more patients with proven treatments — starting with the goal of doubling hepatitis C cure rates in Alabama, Illinois, Louisiana, and Massachusetts. Breakthroughs to Follow-through will work to identify patients who have fallen out of care, synthesize records, track progress, and focus limited staff time where it can have the greatest impact.
OpenAI 基金会希望赋能最贴近问题的人群，并将 AI 应用于能够切实改善人们生活的领域。
The OpenAI Foundation wants to empower those who are closest to the issues and to put AI to work where it can make a meaningful difference in people's lives.
这体现了一种心态：未来是一系列普通的工程问题，那就去解决它们。这远比大多数人的反应——即什么问题都不解决——要好得多，但仍不足以应对当下的时刻。
This is reflective of a mindset that the future is a series of ordinary engineering problems, so let's go solve them. This is way, way better than most people's reaction, which is to not solve any of the problems, but it does not meet the moment.
如今，AI 领域发展迅猛。我们仍在等待事后分析报告，而他们应对措施的许多细节尚未敲定，更不用说对外分享了。
The world of AI moves fast now. We are still awaiting the post-mortem, and many of the details of their response have yet to be hashed out, let alone shared externally.
OpenAI 已迈出了初步的、切实且代价高昂的第一步，以解决其训练和评估流程中的诸多问题。
OpenAI has made initial good, real and expensive first steps towards addressing many of the problems with its training and evaluation pipeline.
这些早期行动是强有力的第一步。OpenAI 仍需迅速扭转许多局面，无论是重大事项还是大量平淡无奇的小事，并且这次必须兑现承诺。
These early actions are a strong first step. OpenAI still needs to quickly turn many things around, both the big things and the tons of prosaic little things, and it needs to deliver on its promises this time around.
这包括解决监督和安全方面的彻底失败，但对齐才是关键。
That includes addressing the Total Failures of oversight and security, but alignment is what matters.
最缺失的一点是，OpenAI 对核心问题的看法仍然是错误的。它被视作执行失败，将对齐作为若干支柱之一，并专注于防止特定的不良行为。确实存在关键的执行失败，但如果仅解决这些，那么败局已定。需要一种根本不同的对齐方法和问题视角。
What is most missing is that OpenAI's vision of the central problem remains wrong. It is being viewed as a failure to execute, with alignment one of several pillars and focused on preventing specific undesired actions. There were key failures to execute, but if that is all that is addressed then the battle is already lost. A fundamentally different approach to alignment, and view of the problem, is what is needed.