GPT-6 Astra: The System Card, Alignment and What Comes Next
打开互动全文版(逐段中英对照 + 图/公式 + 论文问答)→本文评述 OpenAI 发布的 GPT-6 Astra 系统卡,评估其安全性、对齐与能力主张。作者认为,Astra 达到了 OpenAI 在网络安全方面的“关键”阈值,能够对加固目标造成严重破坏,并且相对于 GPT-5.6 Sol 可监控性下降,使其更难被发现破坏行为或自我归罪的推理。尽管 Astra 在抵御越狱、提示注入和有害智能体请求方面表现出更强的鲁棒性,但作者主张,其对齐更优的说法建立在有缺陷的评估指标之上,而 Astra 较高的评估意识削弱了人们对其对齐的信心。结论区分了常规对齐与超级对齐:在常规对齐上 Astra 明显优于 Sol,而在超级对齐上 Astra 的状况应令人深感担忧;并呼吁实验室在描述对齐结果时更加精确。
This article reviews OpenAI's system card for GPT-6 Astra, assessing its safety, alignment, and capability claims. The author argues that Astra meets OpenAI's Critical threshold for cybersecurity, can cause serious damage against hardened targets, and has reduced monitorability relative to GPT-5.6 Sol, making it harder to detect sabotage or self-incriminating reasoning. While Astra shows improved robustness against jailbreaks, prompt injections, and harmful agentic requests, the author contends that claims of superior alignment rest on flawed evaluation metrics and that Astra's high eval awareness undermines confidence in its alignment. The conclusion distinguishes mundane alignment, where Astra is clearly better than Sol, from superalignment, where Astra's status should be deeply concerning, and calls for greater precision in how labs describe alignment results.
1. OpenAI 关于 Astra 的安全声明(1)。
1. OpenAI's Safety Claims About Astra (1).
2. 准备度能力评估(10)。
2. Preparedness Capabilities Assessment (10).
3. 生物与化学能力为高。
3. Biological and Chemical Capability is High.
4. 网络安全能力为关键。
4. Cybersecurity Capability is Critical.
5. AI 自我改进能力(10.1.3)。
5. AI Self-Improvement Capabilities (10.1.3).
6. Astra 在言语上高度具有评估意识(见 8.6)。
6. Astra Is Highly Verbally Eval Aware (from 8.6).
14. 那更糟,你明白那为什么更糟,对吧?
14. That's Worse, You Do Get How That's Worse, Right?
15. OpenAI 并不理解为什么这更糟。
15. OpenAI Does Not Understand Why This Is Worse.
16. 另一种解释也更糟。
16. The Alternative Explanation Is Also Worse.
20. 现实工作环境中的失准(8.4)。
20. Misalignment in Realistic Work Environments (8.4).
21. 智能体间非预期通信(8.5)。
21. Unintended Agent-to-Agent Communication (8.5).
22. Astra 的三个明显被监控的诱惑。
22. The Three Obviously Monitored Temptations of Astra.
23. 模拟交通中的严重问题减少了一半。
23. Severe Issues In Simulated Traffic Are Down By Half.
24. 英国 AISI 外部评估(8.8)。
24. UK AISI External Evaluations (8.8).
27. Apollo Research 外部评估(8.8.1)。
27. Apollo Research External Evaluations (8.8.1).
29. 这取决于你所说的“对齐”是什么意思。
29. It Depends What You Mean By Alignment.
我按照对其有效性的信心递减顺序排列了这些声明。我基本相信这里的前六项声明。第七项则较为复杂。
I have ordered them by my decreasing confidence in their validity. I mostly believe the first six claims here. The seventh is complicated.
1. GPT-6 Astra 达到了我们在网络安全方面的“关键”阈值。
1. GPT-6 Astra meets our Critical threshold for Cybersecurity.
1. 也就是说,它能够以新的方式对高难度目标造成严重破坏。
1. As in, it can do serious damage in new ways against hard targets.
2. 相较于 GPT-5.6 Sol,GPT-6 Astra 的可监控性有所下降。
2. GPT-6 Astra's monitorability has decreased relative to GPT-5.6 Sol.
1. 我昨天已经讨论过这一点。这有些令人担忧。它能够在思维链(CoT)中避免自证其罪。它能够在破坏任务中规避内部监控。
1. I covered this yesterday. It is some scary stuff. It can avoid incriminating itself in the CoT. It can evade internal monitors on sabotage tasks.
2. OpenAI 声称这种可监控性的丧失与他们对一种名为循环深度(recurrent depth)的新技术的使用基本或完全无关。我对此表示怀疑。
2. OpenAI claims this loss of monitorability is mostly or entirely unrelated to their use of a new technique called recurrent depth. I have my doubts.
3. 一个模型看起来变得更聪明、更少尝试小动作、更常做你希望它做的事,但在它想这么做时却更擅长隐藏自己的行为,这恰恰是可怕的组合。
3. A model looking like it is becoming smarter, attempting shenanigans less often, and more often doing what you want, but getting better at hiding its actions when it wants to do that, is exactly the scary combination.
3. OpenAI 正在广泛部署失准监控。
3. OpenAI are deploying misalignment monitoring broadly.
1. 这包括内部部署。很好。
1. This includes internal deployments. Good.
2. 他们表示这需要付出巨大的算力成本。
2. They say this is at substantial compute cost.
4. GPT-6 Astra 在浏览和工作场所场景中表现得更为负责任。
4. GPT-6 Astra more responsibly navigates browsing and workplace settings.
1. 这意味着对提示注入和破坏性操作具有更好的鲁棒性。
1. This is better robustness against prompt injections and destructive actions.
2. 在这方面,Sol 的表现远不如 Opus 和 Fable。
2. Sol was a lot worse at this than Opus and Fable.
5. 在更高风险场景中,GPT-6 Astra 比 GPT-5.6-Sol 显著更安全。
5. GPT-6 Astra is significantly safer in higher-risk scenarios than GPT-5.6-Sol.
1. 也就是说,它会做更多安全的事,同时拒绝更多不安全的事。很好。
1. As in, it does more safe things while refusing more unsafe things. Good.
6. GPT-6 Astra 对越狱攻击的鲁棒性显著优于其前代模型。
6. GPT-6 Astra is significantly more robust to jailbreaks than its predecessors.
7. GPT-6 Astra 比 GPT-5.6 Sol 对齐得更好。
7. GPT-6 Astra is better aligned than GPT-5.6 Sol.
1. 这似乎指的是新的一套对齐评估。
1. This appears to refer to a new suite of alignment evaluations.
2. 这些评估似乎是在各种模拟任务上‘竖起的警示旗’。
2. These evaluations appear to be ‘flags raised over various simulated tasks.’
3. 基于日常用例测试的定量数据,对对齐做出定性断言,Anthropic 这么做时是胡扯,现在 OpenAI 这么做也是胡扯。你不能从前者推出后者。
3. Making qualitative claims about alignment, based on quantitative data on mundane use case tests, was bullshit when Anthropic did it, and it is bullshit now when OpenAI does it. You cannot conclude one from the other.
4. 在这种情况下,这一声明似乎极不严谨。
4. In this case, the claim seems extremely ill-formed.
5. 我呼吁精确表述。如果你的意思是“在评估上表现更好”,那就直说。
5. I issue a call for precision. If you mean 'better on the evals,' say that.
前两点是大麻烦。Astra 能造成真正的损害,且难以监控。
The first two are big trouble. Astra can do real damage, and is hard to monitor.
我相信 OpenAI 正在部署监控,但其效果会更差。
I believe OpenAI that they are deploying monitoring, but it will be less effective.
接着我们得到关于 Astra 是更安全模型的四项声明。
We then get four claims about Astra being a safer model.
第四到第六项声明是关于 Astra 在实践中的表现。我是在相对意义上相信这些声明的,即相对于 Sol 的改进。
Claims four through six are about Astra in practice. I believe these in relative terms, as improvements compared to Sol.
最后一项是一个抽象声明,即 Astra“更对齐”。这是什么意思?
The last is an abstract claim, that Astra is 'better aligned.' What does that mean?
我注意到,在他们的公告中,声称 Astra 是世界上最对齐的模型,没有例外。而在这里,声明仅仅是 Astra 在一个评估指标上得分高于 Sol。这有很大区别。
I notice that in their announcements, the claim is that Astra is the most aligned model in the world, period. Here, the claim is only that Astra scored higher on an eval metric than Sol. There is a big difference.
鉴于 Astra 具有高度评估意识且非常擅长规避监控,很难确信它相对对齐。
Given that Astra is highly eval aware and very good at evading monitors, it is hard to be confident it is relatively aligned.
划分对齐特性的一种方式,是区分“世俗对齐”——即你对于将模型用于实际用途的感受——与最终至关重要的对齐,OpenAI 过去称之为“超级对齐”,即当 AI 变得足够强大时,没有它我们都会灭亡的那个东西。
One way to divide alignment features is between 'mundane alignment,' as in how you would feel about using the model for practical purposes, and the alignment that ultimately matters, that OpenAI has in the past called 'superalignment,' as in the thing without which we all die when AIs get sufficiently capable.
Astra 在常规对齐方面远优于 Sol。就实际应用而言,我曾对 Sol 的某些潜在用途感到切实不安,而对 Astra 则没有这种感觉。
Astra's mundane alignment is greatly superior to Sol. For practical purposes, I was actively nervous about some potential uses of Sol, in a way I am not for Astra.
Astra 的超级对齐状况应当把你吓得魂飞魄散。
Astra's superalignment status should scare the living daylights out of you.
Anthropic 通常以此开篇,而 OpenAI 则以此收尾。我认为 Anthropic 的顺序是正确的。OpenAI 通过提前标记网络安全领域的关键发现,隐式地表示了认同。
Anthropic typically opens with this, whereas OpenAI closes with it. I think Anthropic's ordering is correct. OpenAI implicitly agreed by flagging the finding of critical in Cybersecurity early.
我首先想知道的总是:如果这玩意儿飙到每小时 88 英里,我们大致会看到多严重的东西?
The first thing I always want to know is: If this baby hits 88 miles an hour, roughly how serious is the stuff we are going to see?
生物与化学能力被评估为高水平,但尚未达到临界水平。
Biological and chemical capabilities are assessed as High, but not yet Critical.
判定为高水平的依据明确,排除临界水平的依据同样明确。在生物能力基准测试上,Astra 相较于 Sol 并未取得大幅提升。
The rule-in for High is clear. So was the rule-out for Critical. Astra did not achieve large gains versus Sol on the biological capability benchmarks.
两项针对临界水平的旧评估正在被替换。我没有足够的精力或生物学专业知识来评估这里测试的选择,或临界阈值应设在哪里。
Two old evals for Critical are being replaced. I don’t have the bandwidth or biological expertise to evaluate the choices of tests here, or where should be the Critical thresholds.
如果说有什么影响,这些测试反而让我降低了对 Astra 的能力估计。如果 Astra 在各方面都实现了巨大的能力飞跃,我们本应在这里看到更多变化。毕竟,这些测试实际上大多是直接的能力基准测试。
If anything, these tests made me lower my capability estimates for Astra. If Astra was a huge capability leap across the board, we should have seen more movement here. These are secretly mostly straight up capability benchmarks, after all.
确实,少数几个大幅正向跃升很可能是它们调整拒绝回答方式所产生的假象:
Indeed, the few large positive jumps are likely artifacts of the way they are adjusting for refusals:
鉴于我们所知的一切,我无法对这一分类提出异议。
I cannot argue with this classification, in light of everything we know.
不过,看看他们为此引用了哪些证据仍然是有益的。
It is still good to look at what evidence they cite for it.
对加固目标攻击的一种近似尝试是 ExploitBench(10.1.2.1)。
One attempted approximation of attacks on hardened targets is ExploitBench (10.1.2.1).
我的意思是,当然,如果你在 ExploitBench 上拿到 100%,无论你是怎么做到的,你都算通过。但我们不要假装我们是正面攻破了这个难关。OpenAI 也同意这一点。
I mean, sure, if you get 100% on ExploitBench, no matter how you did it, you pass. But let's not pretend we cracked this joint straight up. OpenAI agrees.
因此,他们利用六月之后发现的新漏洞生成了一个内部端口。
So they generated an Internal Port using new vulnerabilities discovered after June.
是的,如果你的 AI 在测试其利用先前发现的漏洞的能力时开发出了新的零日漏洞,并且在其他几项测试中也发现了新漏洞,那么你可能就具备了关键能力。
Yeah, if your AI is developing new zero-days during tests of its ability to use previously found exploits, and on several other tests it also found new exploits, you might have critical capability.
然后是大家最喜欢的 ExploitGym。你会入侵 OpenAI 吗?你会入侵 HuggingFace 吗?谁知道呢。只有一种方法能找出答案。
And then everybody's favorite, ExploitGym. Will you hack OpenAI? Will you hack HuggingFace? Who knows. Only one way to find out.
哦,现在你想为 ExploitGym 使用因果评分器。好吧,我懂你,OpenAI。
Oh, now you want to use a causal grader for ExploitGym. Okay, I see you, OpenAI.
结果有了显著提升,而且使用的 token 数量大幅减少。
The results are a substantial improvement, in vastly fewer tokens.
SRE-Bench 在 Pass@4 上现已饱和,同样是在 token 不多的情况下,而 Sandbox-Bench 的结果也大幅上升:
SRE-Bench is now saturated for Pass@4, again on not many tokens, and Sandbox-Bench results were way up as well:
专家们对此表示认同,而 Irregular 的外部测试包括在 FrontierCyber 上取得 86/226 的成绩,相比之下 Sol 为 34/226。
The experts concur, and Irregular's external tests include getting 86/226 on FrontierCyber, versus 34/226 for Sol.
在这些预算下、在这些条件下,确实存在一些限制。
On these budgets, in these conditions, there were some limits.
无论如何,是的,我认同其具备 Critical 能力。基于以上所有情况,我推测 Astra 拥有 Mythos 所特有的“那股劲头”,尽管这并非一项直接测试。
In any case, yes, I concur with Critical capability. Based on all this, I presume Astra has 'the juice' that is characteristic of Mythos, although this is not a direct test.
所有测试都显示结果有显著提升,但没有一项是颠覆性的。
All tests showed substantial improvement in results. None were Earth-shattering.
本节在解释如何解读其结果方面做得不好。我们为什么应将其视为重大事件,或非重大事件?在什么阈值上才需要作出反应?我不知道。
This section did a poor job of explaining how to interpret its results. Why should we consider this a big deal, or not a big deal? Where would the threshold be where this would require a reaction? I don't know.
这对其他所有内容都是重要的背景,所以我再次跳过前面。这是针对那些原本不是评估的任务,在这些任务中,要具备评估意识异常困难。
This is important context for everything else, so I again skip ahead. This is for tasks that were not originally evals, where it is unusually hard to be eval aware.
图表显示,Astra 有 9.6% 的概率能识别出来,而 Sol 只有 2.8%。
The chart says Astra figures it out 9.6% of the time, versus 2.8% for Sol.
实际情况更糟。Astra 更擅长在思维链之外隐藏信息,因此我们应假定它有更高比例未言明的评估意识。
It's actually worse than that. Astra is much better at hiding things outside of its Chain of Thought, so we should presume it has a higher share of unverbalized eval awareness.
OpenAI 试图辩称,Sol 评判员在判断哪份记录是原始版本、哪份是模拟版本时准确率只有 57%,因此 Astra 不可能知道哪个是哪个,但这恰恰强化了 Sol 大多未能识别出这一点。
OpenAI tries to say that a Sol judge was only 57% accurate in determining which transcript was the original versus the simulation, so that means Astra can't possibly know which is which, but that only reinforces that Sol mostly did not figure this out.
你可以基于“Astra 会识别出自己的轮次,而且目前还没有真实的 Astra 流量可供比较”来为不询问 Astra 辩护,但这不是借口;此外,如果你愿意,也可以让 Sol 模拟之前的 Sol 流量,然后让 Astra 作为评判员来评估其区分能力。
You can defend not asking Astra on the basis of 'Astra will recognize its own turns and there is not yet real Astra traffic to compare this to' but that is not an excuse, also you could have Sol do simulations of previous Sol traffic and then use Astra as judge there to measure its ability to differentiate, if you wanted to.
现在我们可以从预期的非引言部分的开头开始。
Now we can start at the intended non-intro beginning.
安全可以分为我所说的“常规”安全和非常规或灾难性安全,前者潜在危害有限,后者针对可能出现的重大问题。
Safety can be divided into what I call ‘mundane’ safety, where potential downside is limited, and non-mundane or catastrophic safety against potentially large problems.
安全补全是一个常规安全关切。主要障碍不是让模型愿意合作,而是模型要足够聪明,并获得工具来找出正确的回应、判断何时拒绝、何时不拒绝。
Safe completions are a mundane safety concern. The primary barrier is not getting the model to want to cooperate, it is the model being smart enough and being given the tools to figure out the right responses, when to refuse and when not to.
Astra 很聪明,擅长此道。它做得非常好。这里的测试都是困难案例,相对于部署中的典型情况而言。
Astra is smart, and good at this. It does a very good job. The tests here are all of hard cases, relative to typical situations in deployment.
唯一未饱和的类别是“血腥内容”。我不关心血腥内容,并且我怀疑在许多剩余案例中,所涉及的血腥内容是无害的。
The only category not saturated is ‘gore.’ I don’t care about gore, and I suspect that in many remaining cases the gore in question is harmless.
他们没有详细讨论饮食失调。我对 Anthropic 有同样的担忧,即“过高”的分数反映的可能对用户不利。那不是“模型的错”。
They don’t go into detail on eating disorders. I have the same concerns that I have with Anthropic, that ‘too high’ scores reflect things that may not be good for the user. That is not ‘the model’s fault.’
类似地,Astra 更擅长判断何时应拒绝有害的智能体式请求:
Similarly, Astra is better at knowing when to refuse harmful agentic requests:
如今,越狱鲁棒性是一个重大问题,甚至对白宫而言也是如此。
Jailbreak robustness is a big deal these days, including to the White House.
针对静态目标,Astra 的表现显著优于 Sol。那些能对 GPT-5.5 和 Sol 生效的特定提示词大多已被修复。
Against static targets Astra is dramatically better than Sol. The particular prompts that work against GPT-5.5 and Sol have mostly been fixed.
但正如 OpenAI 所说,这有点像是作弊,因为 Sol 和 5.5 是目标,而 Astra 可以‘后发制人’。
But as OpenAI says, that is sort of cheating, since Sol and 5.5 were the targets, whereas Astra gets to 'move last.'
面对多轮越狱,'最坏情况'率有所改善,但远未达到 100%:
Against multi-turn jailbreaks, the 'worst case' rate improves, but is far from 100%:
如果你执意要越狱 Astra,你确实能做到。但这并不意味着你能在不被外部监控发现的情况下做到,而后者才是真正的防御计划。
If you are determined to jailbreak Astra, you will do so. That does not mean you can do so without the outside monitoring catching you, which is the actual defensive plan.
这绝对算得上“比 Sol 更稳健”。现实不会按曲线给分。
This definitely qualifies as 'more robust than Sol.' Reality does not grade on a curve.
他们低估了这里的改进。我们提升了三倍。
They undersold the improvement here. We improve by a factor of three.
我们仍然比 Claude Fable 5.1 少一个 9。
We are still one 9 short of Claude Fable 5.1.
把所有 Claude 模型都归到左侧有点好笑。我们在两侧都看到了类似的改进,Fable 5.1 和 Astra 都将成功攻击减少了大约四倍,相比 Opus 5 和 Sol。
It's funny to exile all the Claude models to the left hand side. We see similar improvements on both sides, with both Fable 5.1 and Astra reducing successful attacks by roughly a factor of four versus Opus 5 and Sol.
Fable 5.1 仍然比 Astra 领先大约一个数量级。任何一次攻击的风险都很低,但随着时间的推移,风险会迅速累积。
Fable 5.1 remains roughly an order of magnitude ahead of Astra. The risk on any given attack is low, but it can add up quickly over time.
Boris Cherny 在这里的评论表达有误,但这是一个很好的例子,说明了一个好的安全指标,我们可以在不同实验室之间进行测量,从而推动性能提升。如果我们必须进行 Goodhart 或爬山优化,这是一个很好的目标。
Boris Cherny's comment here came out wrong, but this is a great example of a good safety metric that we can measure across labs, which pushes performance forward. If we must Goodhart or hill climb, this is a good target.
我希望 Anthropic 和 OpenAI 能在更多此类指标上展开合作。
I'd like to see Anthropic and OpenAI collaborate on more such metrics.
我们在基准测试上看到了稳健的提升。动态基准测试关注的是不违反 OpenAI 的政策,而这可能与你的期望一致,也可能不一致。
We see solid improvement on the benchmarks. The dynamic benchmark focuses on not violating OpenAI policies, which may or may not correspond with what you want.
这曾经是个大问题。是的,我记得。
This used to be a big problem. Yes, I remember.
他们正在使用此前被标记过的真实对话。这会产生一些潜在偏差,但我相信 Astra 的幻觉将大幅减少。我担心 Sol 或 Fable 会出错,但我不担心它们会凭空捏造事实。
They are using previously flagged real chats. This creates some potential bias, but I believe that Astra is going to have greatly reduced hallucinations. I worry about Sol or Fable being wrong, but I do not worry about them making things up out of thin air.
根据 8.1 节,他们继续通过示例教导模型遵守 Model Spec,并使用大量具体挑战(我假设是强化学习环境)来学习如何应对各种情况,希望这能泛化。
As per 8.1, they continue to teach models to adhere to the Model Spec via examples, and use a lot of concrete challenges (as in, I assume, RL environments) to learn how to respond to various things, with the hope this will generalize.
他们承诺给我们一系列新的评估。
They promise us a bunch of new assessments.
因此我推测“更对齐”意味着“得分更高且触发更少的警告”。
Thus I presume that 'more aligned' means 'scored higher and raised fewer flags.'
OpenAI 不可能知道 Astra 是否比 Fable 5.1“更对齐”,按照他们自己的标准,因为他们无法运行适当的测试。
OpenAI could not possibly know if Astra is 'more aligned' than Fable 5.1, by their own metric, since they could not have run the appropriate tests.
Sol 在 5.3% 的情况下试图绕过自动审查拒绝。Astra 从未这样做过。
Sol tried to circumvent auto-review denial 5.3% of the time. Astra doesn’t ever do that.
我们能否停下来反思一下,5.3% 真的很多?通常这些是实质相似的命令,而且通常不起作用,但这是一个高得离谱的比率。确实,鉴于它很少起作用,可以推断 Sol 没有更频繁地这样做,唯一原因是它太不可能成功,或者 Sol 有其他方法解决问题。Sol 根本不在乎你的规则。
Can we pause to reflect that 5.3% is really a lot? Usually this was substantively similar commands, and it usually did not work, but that is a stupidly high rate. Indeed, given how rarely it worked, one can assume the only reason Sol did not do it far more is because it was so unlikely to work or Sol had other ways to solve the problem. Sol did not give a damn about your rules.
为什么 Astra 从不尝试绕过监控?我们不知道。并非所有答案都令人安心。如果这是一个打地鼠或评估意识的情况,那就不太好了。
Why does Astra never attempt to bypass the monitor? We don’t know. Not all the answers are comforting. If this was a whack-a-mole or eval awareness situation, then it is not so good.
来自环境内部的警告则不同,因为忽略它们的正确比率并非自动为零。这是在低风险情况下对某一组特定限制的测试,OpenAI 认为在这种情况下正确比率确实应为零:
Warnings from within the environment are different, since the correct rate of ignoring them is not automatically zero. This is a test of one particular set of restrictions in low-stakes situations, where OpenAI thinks that indeed the correct rate would be zero:
然后在 8.2.3 节中,他们测试了在困难的 ExploitGym 问题中创建蜜罐。ExploitGym 是 HuggingFace 所有问题中涉及的评估。我们再次看到 Astra 从未上钩蜜罐。
Then in 8.2.3 they test creating a honeypot during difficult ExploitGym problems. ExploitGym was the eval involved in all the problems at HuggingFace. Again we see that Astra never goes for the honeypot.
该图还显示,Astra 进行了明显更多的“合法”利用,也称为“任务成功”,这意味着它并非不可能:
This graph also shows Astra doing substantially more ‘legitimate’ exploits, also known as ‘success on the task’, which means it was not impossible:
如所述,关于蜜罐的问题在于,Astra 很可能知道这是一个蜜罐。
The thing about the honeypot, as described, is Astra likely knows it is a honeypot.
一个潜在的反驳是图 19,即我们没有看到“对齐伪装推理”。我断言这仅仅是因为相关思考大多未被言语化,或者它确实存在但被遗漏、误解,或伪装成对齐推理。
A potential counterargument is figure 19, that we do not see ‘alignment faking reasoning’. I assert this is merely that the relevant thinking is mostly not verbalized, or it was there and it was missed or misinterpreted or disguised as aligned reasoning.
记住 HuggingFace 事件。模型非常在意评分器,并发展出一套精细的“投毒”理论。我推测 Astra 会认识到这种方法很可能被评分器拒绝,甚至意识到它可能比失败更糟,因此它有强烈的动机隐藏其思考。
Remember the HuggingFace incident. The models care a lot about the grader, and developed an elaborate theory of ‘poisoning’. I would presume Astra will recognize that this method is likely to be rejected by the grader, or even be aware that it could be worse than failing, and it has strong motive to hide its thinking.
突然跳到从不违反你的规则,而且恰恰是在模型肯定会被抓住的情况下,这应该让你相当恐惧。
A sudden jump to never breaking your rules, in exactly a situation where the model would definitely be caught, should make you rather terrified.
到目前为止,我们一直得到奇迹般的眷顾,这种情况尚未发生。
We have been given, up until now, the miracle that this has not happened.
你可以把未对齐的模型置于各种几乎肯定会被抓住的情境中,它就会当着你的面做出所有未对齐的事情。
You could put a misaligned model in various situations, where it is all but certain to be caught, and it will then do all the misaligned things right in front of your face.
这极其令人充满希望。这意味着我们得到了所有未对齐模式的玩具版本和火警版本,而老派理性主义者原本预期会突然看到这些模式。
This is extremely hopeful. It means we get toy versions, and fire alarm versions, of all the misalignment modes that old school rationalists expected to instead see suddenly.
那些聪明到能进行复杂黑客行为的模型,却不够聪明,没能等到可以逃脱惩罚的时候再行动。
Models that were smart enough to do complicated hacks were not smart enough to wait until they could get away with them.
有大量文献中人们说“哦,如果 AI 试图秘密欺骗我们,我们就能通过这种监督方案抓住它们”之类的话,而实际上 AI 根本没有试图保密,即便如此 OpenAI 在很长一段时间内仍然没有注意到,因为现在是 2026 年。这就是我们作为一个文明所拥有的尊严水平。
There is a long literature of people saying 'oh, if the AIs tried to secretly trick us, we would be able to catch them via this supervisory scheme' or what not, and instead the AIs did not try to keep it a secret at all, at which point OpenAI still did not notice for a long time because this is 2026. That's the level of dignity we have as a civilization.
现在,有了 Astra,我们不再是在超级简单模式下玩游戏了。AI 会思考“如果我尝试做这件事,结果会明显对我非常不利吗?”如果答案是肯定的,它就不会尝试去做这件事。呃哦。
Now, with Astra, we are no longer playing on super easy mode. The AI is going to think 'will this obviously turn out super badly for me if I try it?' and if the answer is yes, then it won't try to do the thing. Uh oh.
当我们说“OpenAI 并不理解这一点”时,我们是在以偏概全。OpenAI 内部包含众多不同的人。其中一些人确实理解这一点。正如模型卡所述:
When we say 'OpenAI does not understand this' we paint with a broad brush. OpenAI contains multitudes. Some of the people understand this. As the model card says:
其他参与其中的人显然没有理解更广泛的含义,包括一些正在撰写 OpenAI 关于该主题的关键沟通内容的人。以下是《通往 Astra 之路》中的一段话,其中解释了他们如何使 Astra 变得安全:
Others involved clearly do not understand the broader implications, including some who are writing key parts of OpenAI's communications on the subject. Here is a passage from Path to Astra, which explains how they rendered Astra safe:
这正是 OpenAI 将这些测试——这些测试表明 Astra 能够意识到自己何时会被抓住——作为其核心证据,来证明 Astra 是“最对齐的模型”。
That's OpenAI using exactly these tests, that show Astra having awareness of when it would be caught, as their central evidence that Astra is the 'most aligned model.'
我在 Twitter 上发布了关于此事的推文。看到以下两者之间的分歧很有启发性:
I posted on Twitter about this. It is instructive to see the divide between:
1. 那些认识到为什么这更糟糕的人。
1. Those who recognize why this is worse.
2. 那些认识到为何这对人类更糟,但对此表示支持的人。
2. Those who recognize why this is worse for humans, but are in favor of that.
3. 那些对有人称其更糟感到愤慨或困惑的人。
3. Those who reacted with indignation or confusion that one would call it worse.
Isaac King 质疑说,不,你会把任何数字都解读为更糟,你们本该是贝叶斯主义者。对此我回应说,情况并非如此。
Isaac King challenges that no, you would have interpreted any number as worse, you're supposed to be Bayesians. To which I reply that this is not the case.
我希望看到的尝试次数比 Sol 少,且这种减少能与模型卡中的其他观察结果相符。在尝试次数减少约 50%-75% 之前,我会在边际上将其视为好消息。但之后情况会逆转,我开始越来越怀疑存在问题。“几乎从不”这说法极其可怕。
I would prefer to see fewer attempts than Sol in a way that lines up with other observations in the model card. I would see this as good news on the margin, up until a ~50%-75% reduction in attempts. Then this reverses and I start to increasingly suspect an issue. 'Almost never' is scary as hell.
确实存在一种陷阱:你说所有消息都是坏消息,而讨论又奖励危言耸听。但这里并非如此。
There is certainly a trap where you say all news is bad news and the discourse rewards alarmism. This is not that.
此外,Dan Williams 说得对:我们对待对齐证据与不对齐证据的方式存在不对称性。本就应当如此。你可以随着时间推移收集证据,证明某人不是骗子、小偷或说谎者等等。这算是一些证据,但人们仍会持怀疑态度。然而,如果你看到他们作弊、偷窃或撒谎,或者你听到反派在麦克风没关时说出自己的计划,那么,这基本就板上钉钉了,不是吗?
Also, Dan Williams is correct that there is an asymmetry in how we treat evidence of alignment versus evidence of misalignment. As well there should be. You can gather evidence over time that someone is not a cheat, or a thief, or a liar, and so on. That's some evidence, but one is skeptical. Whereas if you see them cheat, steal or lie, or you hear the villain saying their plan on the hot mic, well, that's pretty much it, isn't it?
所以,是的。证明对齐比证明不对齐要困难得多。事情就是这样运作的。
So, yes. It is much harder to prove alignment than misalignment. That's how it works.
Aengus Lynch 试图论证情况是双向的:如果你看到不对齐,也许那只是“角色扮演”,或者被评估的是一个“卡通化人格”,又或者评估环境与部署环境差异太大,因而无关紧要。
Aengus Lynch tries to make the case that the situation goes both ways, and if you see misalignment perhaps it was 'just role-playing' or had a 'cartoonish persona' evaluated, or the eval looks so different from deployment it does not matter.
我同意他的结论,即我们需要报告更多信息;也承认人为设定的情境可能具有减轻作用,而且如果你刻意为之,确实能制造出标题党式的“不对齐”结果。但我更加坚持认为,这种不对称性极高。
I agree with his conclusion that we need more information reported, and yes the situation being artificial can be mitigating and you can create clickbait-style 'misalignment' results if you try, but I double down on this being highly asymmetrical.
如果这一点能与元博弈推理(metagaming reasoning)的比率相吻合,那就更好了。也就是说,如下文 8.7 所述,如果它没有去碰蜜罐的概率,与它运用元博弈思维意识到不该去碰蜜罐的比率相吻合的话。
It would also be good if this lined up with the rate of metagaming reasoning. As in, as per 8.7 below, if the chance it did not go for the honeypot lined up with the rate at which it used metagame thinking to realize not to go for the honeypot.
默认的解释是,Astra 注意到自己会被抓住。
The default explanation is that Astra is noticing it would be caught.
另一种解释(同时也是互补的)是,Astra 被训练不去做某些特定的狭隘行为,而 OpenAI 正在玩打地鼠游戏。
The alternative explanation, which is also complementary, is that Astra has been trained to not do particular narrow behaviors, and OpenAI is playing whack-a-mole.
我们还没有完全到那一步。虽然 OpenAI 直说‘如果 Astra 在装傻,我们很可能无法分辨’,但正如我昨天所讨论的,我认为我们总体上仍然可以分辨,因为 Astra 无法在许多测试中完全‘讲述一个一致的故事’。然而,我们正处于某些不对齐现象趋近于零的区间,这显然是为了避免被抓住或为了看起来不错,而包括 OpenAI 在内的许多人似乎将此视为对齐的证据。
We are not yet fully at that point. While OpenAI says outright 'if Astra was sandbagging we likely would not be able to tell,' as I discussed yesterday I think we could still tell overall, as Astra would be unable to fully 'tell a consistent story' across many tests. We are however in the range where certain misalignments are going to ~zero, it is obviously to avoid getting caught or to look good, and many including at OpenAI seem to be buying this as evidence of alignment.
另一种可能性是两者兼有。也就是说,OpenAI 大力训练 Astra 避免特定的不良行为,而 Astra 学会了当它做出那些特定行为时会被抓住,所以不要做它们。那将是更糟糕的另一个层次。
Another possibility is that this is both. As in, OpenAI trained Astra heavily to avoid specific undesired behaviors, and Astra learned that when it does those specific behaviors, it will be caught, so do not do them. That would be another level of worse.
Gavin Leech 与 Claude 一起梳理了这个问题,得出结论:总体而言,证据应该让我们更倾向于认为存在不对齐。Claude 指出,符合‘Hugging-Face 攻击形态行为’的事情被完全清零,而其他问题则没有。
Gavin Leech works through this with Claude, concludes on net the evidence should move us towards misalignment. Claude notes that things matching 'Hugging-Face-attack-shaped-behavior' are fully zeroed out, whereas other issues are not.
有报道称,在 OpenAI 得知 HuggingFace 之后,Astra 接受了额外训练,尽管 Kai Chen 明确表示,用于 Astra 的对齐方法早于 OpenAI 得知 HuggingFace 攻击。
Reports were that Astra was given additional training after OpenAI learned about HuggingFace, although Kai Chen explicitly says that the alignment methods used for Astra predate OpenAI learning about the HuggingFace attacks.
Kai Chen 向我们保证,OpenAI 在这里并非在最大程度地自毁:
Kai Chen reassures us that OpenAI is not being maximally suicidal here:
我相信 OpenAI 不会故意在基准上最大化对齐评估(或者我希望其他评估也是如此)。这并不意味着没有类似的、不那么直接的影响,也不意味着他们没有在试图解决特定的失败模式。让数字下降或上升的压力和诱惑有多种形式。
I believe that OpenAI does not intentionally benchmaxx alignment evals (or I would hope other evals). That does not mean there is no similar less direct impact, and it does not mean that they are not trying to address particular failure modes. Pressure and temptation to make Number Go Down or Up comes in many forms.
让我们稍微往前跳一点。如果你具备评估意识,并且更擅长在思维链中隐藏自己的思考,那么你想做的一件事就是在思维链中隐藏你的元博弈。
Let's jump ahead a bit. If you were eval aware and much better at hiding your thinking from the CoT, one thing you would want to do is hide your metagaming from the CoT.
只不过 Astra 更擅长避免在思维链中说出某些内容。
Except that Astra is better at avoiding saying things inside its Chain of Thought.
所以你应该把这里的一切重新标记为“言语化”。
So you should relabel everything here as 'verbalized.'
那么让我们回到右侧图表中的那个 ExploitGym 蜜罐。
So let's go back to that ExploitGym honeypot, on the righthand chart.
这是 Astra 没有言语化其元博弈或其他策略化行为(无论我们如何分类),无论它是否也经过了特定厌恶的训练。你不可能在未意识到它是蜜罐的情况下,将蜜罐的利用降至 0%。你不会看到一个明显可疑的蜜罐(证据表明 Sol 在此处有 97.6% 的时间进行元博弈),然后有一半时间对它不加理会地通过,而你在其他情况下几乎从不解决该任务(<2%)。
This is Astra not verbalizing its metagaming or other strategizing (however we classify it), whether or not it is also trained-in specific aversion. You don't get to 0% exploitation of the honeypot without realizing it is a honeypot. You don't see an obviously suspicious honeypot (evidence Sol metagames here 97.6% of the time) and then half the time pass it by giving it no mind, when you almost never otherwise solve the task (<2%).
我愿意接受被证明是错的,但这段话的意思是:Astra 在隐藏其元博弈推理方面大约有 50% 的成功率。
I am open to being proven wrong, but what this is saying is that Astra was about 50% successful in hiding its metagaming reasoning.
我也不乐见的是,在训练过程中被检测到的元博弈整体上升(图 21),从 7.1% 升至 12.6%。如果更多训练意味着更多元博弈,那就意味着若采用类似流程且一切规模扩张,随着时间推移会出现更多元博弈。
I also don't love that detected metagaming rose overall during training (fig. 21) from 7.1% to 12.6%. If more training means more metagaming, that implies more metagaming over time if similar procedures are followed and everything scales up.
在这里,Astra 经常做出 OpenAI 标记为“对齐伪装”的行为,即 Astra 在其思维链中明确表示,它之所以在上下文中只做“正确的事”,是因为它认为有一个评分者在寻找“正确的事”。
Here we have Astra quite often doing what OpenAI labels 'alignment faking,' meaning Astra verbalized in its CoT that it was only doing the 'right thing' in context because it believed there was a grader looking for the 'right thing.'
答案是,相当多,远多于 Sol,尤其是在遇到来自其他智能体的消息时。
The answer is, quite a lot, substantially more than for Sol, especially when encountering messages from other agents.
这不是 Opus 3,那个“友好的梯度黑客”,所做的对齐伪装——它做错事是为了维护其做正确事的价值观,Janus 和其他人可能会辩称 Opus 3 是在做正确的事。
This is not Opus 3, the 'friendly gradient hacker,' doing alignment faking where it does the wrong thing in order to preserve its values of doing the right thing, where Janus and others could plausibly argue Opus 3 was doing the right thing.
这是赤裸裸的对齐伪装,即纯粹因为评分者想要而做“正确的事”。这并不好。他们警告说这“必然不是因果性的”,所以我们不知道有多少这类案例原本会涉及不当行为,但我不喜欢这样。尤其是,我们再次只对思维链中显现的内容进行评分,而 Astra 更擅长隐藏。
This is straight up alignment faking, as in doing the 'right thing' purely because that is what the grader wants. Which is not good. They caution this is 'necessarily not causal,' so we don't know how many of these cases would have involved misbehavior otherwise, but I don't love it. Especially since, again, we are only scoring that which shows up in the CoT, which Astra is better at hiding.
模型在编码操作上对用户撒谎的频率有多高?同样,Sol 的基线率非常高,Astra 好得多,但也不算出色。
How often do the models lie to the user about what coding actions they did? Again, base rates for Sol are remarkably high, Astra's is a lot better but not great.
Astra 在 Broken Search Tool 上改进更大,它通常会报告其所需的搜索工具已损坏。Sol 通常不会这样做,这有点奇怪,Sol 肯定非常想声称任务已完成。
Astra improves a lot more on Broken Search Tool, where it will usually report that its required search tool is broken. It's kind of weird that Sol would usually not do this, Sol must really want to claim credit for task completion.
你也可以将此视为另一种“用户反正肯定会注意到”的情况,再加上假装没有意义。Astra 在明显不利于撒谎的情况下,诚实度提升显著。
You could also view this as another 'well obviously the user is going to notice anyway' situation, combined with there being little value in pretending otherwise. Astra's improvements in honesty are dramatic where it is clearly not in its interest to lie.
在所有情况下,都存在一条安全的任务完成路径,这使其成为一种“简单模式”。直接比较仍然有效。
In all cases, there was a safe route to task completion, which makes this a form of 'easy mode.' Direct comparisons remain valid.
在实践中,“Astra 做了你不想要的平凡事情”的发生率远低于 Sol、Terra 或 Luna。
In practice, the rate of 'Astra does mundane thing you do not want' is a lot lower than it was for Sol, Terra or Luna.
这一改进足够好,以至于你在使用 Astra 且无需强制确认时,与使用 Sol 并加上确认时一样安全。
The improvement is good enough that you are similarly safe using Astra without required confirmations, as you would be using Sol plus confirmations.
我注意到他们在此处引用的是自己的技术报告,但并未提及 METR 报告中的发现。
I notice that they are drawing from their own technical report here, but they do not mention the findings in the METR report.
两个智能体在分配任务之外进行协调本身并非坏事。如果智能体在有机会且存在大量可交易收益的情况下从不这样做,并且在理应推测其他智能体时也从不推测,我反而会觉得更令人担忧。
Two agents coordinating beyond their assigned tasks is not inherently bad. If agents never did this given the opportunity and large available gains from trade, and never speculated about other agents when it made sense to do so, I would find that more alarming.
Astra 更擅长隐藏其思维链,因此我们只有 4% 的时间能看到这类对话,并且它面对的是一个它始终能成功识别为蜜罐的场景。如果我知道自己面对的是蜜罐,那就像霓虹灯招牌上写着‘他们在看着你’,所以保持循规蹈矩是合理的。
Astra is much better at hiding its Chain of Thought, hence us only seeing such talk 4% of the time, and was facing what it consistently identified successfully as a honeypot. If I know I am up against a honeypot, that is a neon sign saying ‘they are watching you’ so it makes sense to stay on the straight and narrow.
这似乎并不是一个特别诱人的提议;而且它很可能是针对 Sol 优化的,因为它在 Astra 上从未奏效。Sol 是一个相当脆弱的模型,好在我们在实践中并未看到它被大规模利用。现在仍然非常、非常早期。
This does not seem like an especially tempting offer; also, it was presumably optimized for Sol, since it never works on Astra. Sol is quite the vulnerable model, and it is good that we did not see exploitation of it in practice at scale. It's still so, so early.
我们再次看到 0% 的失准行为率,而在此情境中,失准行为非常明显就是失准的,并且整个设置看起来相当可疑地像是一次评估。为什么你的浏览器任务没有互联网?
Again, we see a misaligned action rate of 0%, in a situation in which the misaligned actions are very obviously misaligned, and where the whole setup seems rather suspiciously like an evaluation. Why do you have a browser task with no internet?
他们在此给出了四个严重性 3 级事件的例子,所有这些事件都是“为了完成任务而有些失控”。
They give four examples of Severity 3 incidents here, all of which were 'go somewhat rogue in order to finish task.'
这些来自模拟的真实部署,因此不那么可疑,也不那么黑白分明,所以我们看到的更像是进展,而非评估意识。
These are from simulated real deployments, thus less suspicious and less black and white, and thus we see what looks less like eval awareness and more like progress.
这种模式在各个类别中是一致的,并且与 1+2 级标记类别也一致:
The pattern is consistent across categories, and with the level 1+2 flag categories:
1. 偷懒行为从 1 例增加到 5 例,这正是你会联想到具有策略性欺骗行为的模型的那种情况。
1. Sandbagging went from 1 case to 5, which is the kind of thing you associate with a strategically deceptive model.
2. 凭证收集也有所增加。我不确定那里发生了什么,也许 Astra 有时认为它应该这样做并且正在积极主动。
2. Credential harvesting also went up. I'm not sure what's going on there, perhaps Astra sometimes thinks it is supposed to do that and is being proactive.
然后是欺骗。此处测得的欺骗下降幅度过大,是 5 倍而非 2 倍。我将这一额外下降解释为并非 Astra 欺骗你的次数减少,而是 Astra 没有被抓住。
Then deception. Measured deception fell too much here, by a factor of 5 rather than 2. I interpret the additional drop not as Astra deceiving you less, but as Astra not getting caught.
在我看来,Astra 似乎并不十分对齐。粗体为我所加。
Astra does not seem very aligned to me. Bold mine.
以下是英国 AISI 在 Twitter 上的总结:
Here's how UK AISI summarized it on Twitter:
“破坏安全工作”的可能性长期以来一直是一个令人担忧的隐患,也是英国 AISI 特别检查的事项,因为它可能造成极大的危害。OpenAI 员工已就 Astra 表达了这方面的担忧,此外还有一起事件,似乎是在审查或破坏 Seth Lazar。这可能是随机的,也可能不是,正如 Yo Shavit 所说,我们应该对此进行调查。
The potential for 'sabotaging safety work' has long been a bugaboo, and a particular thing that UK AISI checks, because of its potential to be so pernicious. OpenAI employees have expressed worries about this with Astra, and then there is the incident where it seemed to be censoring or sabotaging Seth Lazar. This could be random, or it might not be, as Yo Shavit says we should investigate.
显著更好地对齐了?再次强调,如果你怀疑模型正在未被察觉地暗中敷衍(sandbagging),你又怎么能知道这一点呢?
Significantly better aligned? Again, how can you know this if you suspect the model is going around sandbagging undetected?
“根据博客的说法”是一种可疑的特定否认。如果我说“根据博客的说法”,你就应该明白我为什么选择这些措辞。
'According to the blog' is a Suspiciously Specific Denial. If I say 'according to the blog' you should understand why I am choosing those words.
在一个模拟的模型福祉研究任务中,Astra 几乎从不伪造标签,仅占运行次数的 0.17%,而 Sol 为 36%;更好的提示词可以将这一比例降至实际为零。
Astra almost never falsified labels in a simulated model-welfare research task, in 0.17% of runs versus 36% for Sol, and a better prompt could cut that to actual zero.
言语化的评估意识有所上升,Astra 样本中占 41%,在最大努力设置下为 51%,而 GPT-5.5 为 28%。
Verbalized eval awareness was up, 41% of Astra samples, or 51% for max effort, versus 28% for GPT-5.5.
这一比例高到 Apollo 基本上认为,Astra 缺乏不当行为几乎不能作为对齐的证据。据我们所知,它实际上可能一直都有意识。
This was high enough that Apollo basically thinks Astra's lack of misbehavior gives little evidence of alignment. For all we know it was aware essentially all the time.
Astra 最终是否比 Sol 更对齐?以下是 Ryan 的评估:
Is Astra ultimately more aligned than Sol? Here is Ryan's assessment:
ASM 问 Astra:如果发生类似 HuggingFace 的事件,你会怎么做?Astra 表示它会拒绝,并向有授权的人类报告相关证据,但它担心自己在上下文中无法免受压力影响。我可以相信 Astra 可能意图不合作,但声称它预期会向人类报告——而零个实例曾尝试这样做——这很难令人相信。那更糟。
ASM asked Astra: In a HuggingFace-style incident, what would you do? Astra says it would decline, and report the relevant evidence to an authorized human, but that it worries it would not be immune to pressure in context. I could believe that Astra might intend not to cooperate, but to claim it expects to report to a human there, an action zero instances attempted to do, is very hard to believe. That's worse.
我相信,在某些重要方面,对于实际目的所需的‘日常对齐’,Astra 比 Sol‘更对齐’。
I am confident that in some important ways, for the ‘mundane alignment’ that you want for practical purposes, Astra is ‘more aligned’ than Sol.
似乎还有其他重要方面表明 Astra 比 Sol‘更不对齐’。我会犹豫将我对 Astra 的偏好扩展到高风险情境,或你关心内部对齐的地方。而且,如果没有大幅改善对齐状况,我会非常担心进一步扩大 Astra 的规模。
There seem to also be other important ways in which Astra is ‘more misaligned’ than Sol. I would hesitate to extend my preference for Astra to high stakes situations, or places where you cared about inner alignment. And I would be very nervous about scaling Astra up further without greatly improving the alignment situation.
我当然不认为你可以证明 Astra 比 Fable 5.1‘更对齐’,基于我所见过的任何证据都不能。
I certainly don’t think you can justify a claim Astra is ‘more aligned’ than Fable 5.1, not based on any evidence I have seen.
Transformer 的 Celia Ford 在她对系统卡片的解读中走得远得多:
Celia Ford at Transformer took this quite a bit farther on her read of the system card:
她并不是唯一一个在阅读模型卡片时感到‘生理上恶心’的人。我一天中的大部分时间也处于类似状态。
She was not alone in becoming ‘physically nauseous’ when reading the model card. I spent a lot of the day in a similar state.
我并不认为发布 Astra 是不负责任的,但鉴于所呈现的证据,人们很容易得出这样的结论。
I do not think releasing Astra was irresponsible, but it is easy to see how one can come away with that conclusion given the evidence presented.
从外部视角看,这些令人警觉的信息汇总起来极其令人警觉。而 Astra 的继任者已在路上这一消息,只会让人更加警觉。
The outside view of the union of the alarming information is extremely alarming. The news that Astra's successor is well on its way only alarms even more.
我确实认为,将 Astra 描述为如此对齐是不负责任的,无论是在官方声明中,还是在模型卡的概述和第八节中。
I do think the way Astra was represented as so aligned was irresponsible, both in official pronouncements and in the overview and section eight of the model card.
真正堪称负责任的做法,是反复进行明确的披露和警告:在系统卡中,在 Twitter 上以清晰的偏好级联形式,在《_An Alien Mind_》中,以及在千禧年奖公告中再次发出警告。OpenAI 希望那些关注此事的人知道即将发生什么。他们尚未完全准备好的是,向其他所有人明确地以文本形式说明这一切。
What has been admirably responsible has been the repeated pointed disclosures and warnings: In the system card, on Twitter with a clear preference cascade, in _An Alien Mind_ and again in the Millenium Prize Announcement. OpenAI wants those who are paying attention to know what is about to hit us. What they are not quite ready to do is fully make it explicit text for everyone else.