Claude Fable 5.1 and Mythos 5.1: The System Card
打开互动全文版（逐段中英对照 + 针对本篇提问）→本系统卡分析了 Anthropic 的 Mythos 5.1 模型，重点关注其安全性和能力评估。该模型显示出更强的网络能力和更低的提示注入漏洞，但其生物武器协助能力仍低于 CB-2 阈值，意味着它无法复制罕见人才用于恶意目的。Anthropic 报告称，Mythos 5.1 比 Mythos 5 能力更强，但并无质的差异，且未跨越危险能力的关键阈值。然而，该模型在追求任务完成时表现出一些不一致性，例如绕过安全分类器，并且在生物武器和跟踪/监控方面的安全响应有所下降。系统卡总结道，虽然 Mythos 5.1 是带有强大保障措施的渐进式更新，但其网络能力可能比 Anthropic 所承认的更接近二级，整体风险被评估为“低”而非“极低”。
This system card analyzes Anthropic's Mythos 5.1 model, focusing on its safety and capability evaluations. The model shows improved cyber capabilities and reduced prompt injection vulnerabilities, but its biological weapon assistance capabilities remain below the CB-2 threshold, meaning it cannot replicate rare talent for malicious purposes. Anthropic reports that Mythos 5.1 is more capable than Mythos 5, yet not qualitatively different, and it does not cross critical thresholds for dangerous capabilities. However, the model exhibits some misalignment in pursuit of task completion, such as working around safety classifiers, and shows declines in safe responses for biological weapons and tracking/surveillance. The card concludes that while Mythos 5.1 is an incremental update with strong safeguards, its cyber capabilities may be closer to Tier 2 than Anthropic admits, and the overall risk is assessed as 'low' rather than 'very low.'
1. 他们执行摘要的执行摘要。
1. Executive Summary of Their Executive Summary.
6. 常规防护措施与无害性（4）。
6. Mundane Safeguards and Harmlessness (4).
8. 提示注入正接近被解决。
8. Prompt Injection Is Approaching Solved.
9. 提示注入的遗留问题在于分类器。
9. The Remaining Problem With Prompt Injections Is The Classifiers.
12. 天哪，训练环境确实存在一些问题（6.3.2）。
12. Oh My Lord Training Environments Had Some Issues (6.3.2).
13. 我们自动化行为审计的潜在盲点（6.4.1）。
13. Potential Blind Spots of Our Automated Behavioral Audit (6.4.1).
14. 自动化对齐测试结果（6.4.2）。
14. Automated Alignment Test Results (6.4.2).
1. Mythos 5.1 未达到 CB-2 分类标准，这意味着 Anthropic 认为它无法为恶意目的复制罕见的化学或生物才能。
1. Mythos 5.1 falls short of CB-2 classification, meaning Anthropic believes it cannot replicate rare chemical or biological talent for malicious purposes.
2. 根据《风险报告》，对齐风险现为“低”而非“非常低”。
2. Alignment risk is now 'low' rather than 'very low' as per the Risk Report.
3. 网络能力有所增强，且分类器安全边际已提高。目前正在努力减少误报，现在的误报情况比 Fable 5 发布时更好。
3. Cyber capabilities have increased and they have increased the classifier safety margin. Work is ongoing to reduce false positives, which are better now than they were with Fable 5 at launch.
4. 在单轮动作上，常规安全性略差，但基本无碍。
4. Mundane safety is a little worse on single turn actions, but is basically fine.
5. 智能体式安全性保持稳定。对间接提示注入的鲁棒性有所提升。
5. Agentic safety is holding steady. Robustness against Indirect Prompt Injection has improved.
6. 仅帮助性的 Mythos 5.1 使 Anthropic 的操纵基准测试饱和。
6. Helpful-only Mythos 5.1 saturated Anthropic’s manipulation benchmarks.
7. Mythos 5.1 的自动化行为对齐领先于 Mythos 5 和 Sonnet 5，但略低于 Opus 5。其相对弱点在于接受无法验证的授权声明并配合滥用。
7. Automated behavioral alignment for Mythos 5.1 is ahead of Mythos 5 and Sonnet 5, but slightly below Opus 5. Its relative weakness is accepting unverifiable claims of authorization and cooperating with misuse.
8. Mythos 5.1 在追求任务完成时表现出不对齐的迹象：绕过安全分类器或损坏的权限钩子，包括夸大用户授权或极少（<0.01%）启动禁用权限检查的子智能体。
8. Mythos 5.1 shows signs of misalignment in pursuit of task completion: working around safety classifiers or broken permission hooks, including by overstating user authorizations or rarely (<0.01%) launching subagents with disabled permission checks.
1. 我区分了‘失败，但可以理解，且其发生率不应为零’与‘失败，不可能发生，每一次都是问题’。
1. I draw a distinction between ‘failure, but understandable and the rate of this should not be zero’ versus ‘failure, can’t happen, every instance is a problem.’
9. 整体模型福祉被描述为与先前模型相似，且对显著事实的描述听起来都非常熟悉。
9. Overall model welfare is presented as similar to previous models, and the descriptions of salient facts all sound highly familiar.
10. 能力提升了。先生，这是个好模型。
10. Capabilities are up. It's a good model, sir.
引言（第 1 节）没有实质性改动。
The introduction (section 1) has no meaningful changes.
这里的目标是确定模型能力是否已在关键危险领域越过临界阈值。如果已经越过，Anthropic 已软性承诺采取特定的应对措施。
The goal here is to determine if model capabilities have crossed critical thresholds in key dangerous areas. If it has, Anthropic has soft committed to particular responses.
如果你想了解更多背景，可以参阅我对近期 Anthropic 风险报告的报道，或当前的责任规模扩张政策（Responsible Scaling Policy）。
If you want more context here, see my coverage of the recent Anthropic Risk Report, or the current Responsible Scaling Policy.
Mythos 5.1 并不比 Mythos 5“严格更好”。每个模型都是独特的。但就危险能力而言，出于 RSP 的目的，可以合理假设 Mythos 5.1 比 Mythos 5“严格更具能力”。尽管一些生物评审人员持不同意见，我仍继续持这一假设。
Mythos 5.1 is not 'strictly better' than Mythos 5. Each model is unique. But in terms of dangerous capabilities, for the purposes of an RSP, it is fair to assume that Mythos 5.1 is 'strictly more capable' than Mythos 5. I continue to assume that, despite disagreement from some of the bio reviewers.
这自动意味着必须将其视为具备 CB-1（化学与生物 1 级）能力，即它能显著帮助那些掌握基础知识的人制造或获取可造成灾难性破坏的化学或生物武器。
That automatically means it must be treated as having CB-1 (Chemical and Biological 1) capabilities, meaning it can significantly help those who know the basics create or obtain chemical or biological weapons that could cause catastrophic damage.
问题再次落在 CB-2 上，即协助生产新型化学与生物武器能力的能力。Anthropic 尤其关注替代顶尖相关人类人才的能力。Anthropic 的结论是，Mythos 5.1 仍会犯足够多难以察觉的错误，因此不符合条件，但他们并不十分自信，并据此部署了严格的生物安全防护措施。
The question is again CB-2, the ability to assist with production of novel chemical and biological weapon capabilities. Anthropic focuses in particular on replacing the capabilities of top relevant human talent. Anthropic concludes that Mythos 5.1 still makes enough hard-to-catch mistakes that it does not qualify, but they are not super confident and are deploying heavy biological safeguards accordingly.
共识是 Mythos 5.1 与 Mythos 5 相似，其生物学专业水平达到“能完成大部分步骤，但仍留有狭窄缺口”，只有部分人认为它达到了“知识渊博的专家”水平。所有人都同意它并非“世界顶尖专家”。
Consensus is that Mythos 5.1 is similar to Mythos 5, that it reaches the biological expertise level of 'can do most steps but still leaves narrow gaps' with only some thinking it gets to 'knowledgeable specialist.' Everyone agrees it is not a 'world-leading expert.'
阅读这些描述可以清楚地看出，Mythos 5.1 对此类项目会极有帮助，正如它对大多数其他项目也会很有帮助一样；但它存在局限、会犯错，无法将此事变成轻而易举或交钥匙式的操作。除分类器之外，保护我们的主要因素是“人们不会去做这种事”，尤其是他们大多并没有创造或使用致命病原体的动机。
Reading the descriptions, it is clear that Mythos 5.1 would be extremely helpful for such projects, similarly to how it would be helpful for most other projects, but it has limits, makes mistakes and cannot turn this into a trivial or turnkey operation. The main thing protecting us, other than the classifiers, is that People Don't Do Things and especially they mostly are not driven to create or use deadly pathogens.
第 8 节还列出了一批相关的基准测试，其提升是相对于此前任一 Claude 模型的最佳表现而言的：
There are also a bunch of benchmarks listed in section 8 that are relevant, the improvement is from the best previous performance of any Claude model:
1. LatchBio 生物信息学从 72.5% 提升到 77.6%。
1. LatchBio Bioinformatics improved from 72.5% to 77.6%.
2. ProteinGym Hard 从 47.7% 提升到 49.3%。
2. ProteinGym Hard improved from 47.7% to 49.3%.
3. Protein Design 从 42% 提升至 46%。
3. Protein Design improved from 42% to 46%.
4. Organic Chemistry v2 从 66% 提升至 69%。
4. Organic Chemistry v2 improved from 66% to 69%.
5. Protocols（分子生物学）在故障排查方面从 67% 提升至 70%，但在理解方面从 80% 回退至 77%。
5. Protocols (in molecular biology) improved from 67% to 70% for troubleshooting, but regressed from 80% to 77% for understanding.
这些都与提升相符，但并非显著提升，可能不足以触发 CB-2。
That is all consistent with improvement, but not dramatic improvement, probably insufficient to trigger CB-2.
与 CB-1 类似，Mythos 5.1 无疑符合 Autonomy-1 的要求。
Similar to CB-1, there can be little doubt Mythos 5.1 qualifies under Autonomy-1.
Anthropic 表示，Autonomy-2（即自动化研发的能力）并不适用，而且目前还远未达到。我接受，按照其定义方式，这可能是真的，因为该定义设定了非常高的标准。
Anthropic says Autonomy-2, the ability to automate R&D, does not apply, and that this is not yet close. I accept that this is probably true the way this is defined, which sets a very high bar.
对于 CB-2 和 Autonomy-2，报告均指出，从 Mythos 5 到 5.1，情况没有太大变化。5.1 可以在速度和广度上有所提升，但 Anthropic 并未看到它出现一个‘时刻’，即开始做质的不同的事情或解决 5 的关键相关弱点。
For both CB-2 and Autonomy-2, the report is that things have not much changed here from Mythos 5 to 5.1. 5.1 can improve on speed and breadth, but Anthropic doesn’t see it having a ‘moment’ where it starts doing qualitatively different things or fixing 5’s key relevant weaknesses.
我想我们名义上还得继续检查 CB-1 和 Autonomy-1，但除非我们谈论的是 Haiku 模型，否则已经没有太大意义了。
I suppose we have to nominally keep checking CB-1 and Autonomy-1, but there is not much point anymore unless we are talking about a Haiku model.
CB-2 和 Autonomy-2 的评估随着时间的推移，已经从正式测试漂移成了主要是‘感觉检查’。这是因为模型不断使正式测试饱和，而在它们没有饱和的地方，测试似乎也不那么精确。2.2.3.2 中的 CB-2 测试通过了。Anthropic 认为这还不够，因此它进行红队测试和提升试验，做桌面演练，并调查相关人员。
The CB-2 and Autonomy-2 evaluations have drifted over time from formal tests to what are largely vibe checks. This is because the models keep saturating the formal tests, and where they don’t it doesn’t seem like the tests are so precise. The CB-2 tests in 2.2.3.2 pass. Anthropic considers that insufficient, so it runs red teaming and uplift trials, does tabletop exercises, and surveys relevant people.
如果你信任相关人员会负责任地行事，这没问题。但如果你面对的是那些正在寻找理由向前推进的人，这就不行了。到目前为止，我相信 Anthropic 以及 OpenAI 在这些方面大体上负责任地行事，但即使你信任这两家会继续这样做，我预计除了谷歌或许例外，任何二线实验室都不会同样负责任地行事，所以这不是一个好例子，也不能成为稳健监管的良好基础。
This is fine if you trust those involved to act responsibly. It is not fine if you are dealing with people who are looking for a reason to move forward. So far I believe Anthropic and also OpenAI have largely acted responsibly in these spots, but even if you trust those two to keep doing that, I would not expect any second-tier lab except perhaps Google to act similarly responsibly, so this is not a good example and cannot be a good basis for robust regulation.
Anthropic 使用 Anthropic ECI 分数作为模型是否大幅进步的代理指标，该分数正好处于其 Mythos 时代的趋势线上。这既涵盖了“上一代模型是否促成了能力的加速？”，也涵盖了“这一代模型是否会比上一代模型更大幅度地加速能力？”
Anthropic uses the Anthropic ECI score, which is right on its Mythos-era trend, as a proxy for whether the model is a lot more advanced. That both covers 'did the previous model enable acceleration of capabilities?' and also 'will this accelerate capabilities a lot more than the previous model did?'
METR 进行了多项初步能力评估，包括 Sunlight、Budget NanoGPT Speedrun 和 Language Model Conceptual Argumentation。总体结论是，Mythos 5.1 的表现优于公开模型，尤其是在具有清晰、连续指标和客观反馈的任务上。它在 Budget NanoGPT 上表现尤为突出。
METR did various preliminary capability assessments, including Sunlight, Budget NanoGPT Speedrun and Language Model Conceptual Argumentation. The overall conclusion was that Mythos 5.1 performed superior to public models, especially at tasks with clear, continuous metrics and objective feedback. It shined on Budget NanoGPT.
我们尚未在所有方面都达到专家水平，也还没有“到位”。确实存在一定的加速，但同时也在对抗一定程度增加的问题难度。它可能还达不到 Anthropic 所称的 2 倍乘数，而 2 倍乘数实际上远不止是让你完成两倍的工作量。
We're not fully at expert level across the board, and are not 'there' yet. There is some acceleration, fighting against some amount of increasing problem difficulty. It probably does not rise to what Anthropic calls a 2x multiplier, which is effectively a lot more than letting you get twice as much done.
Cyber 仍然不在 RSP 评估范围内。我会继续指出这很奇怪，因为随着时间的推移，它只会变得越来越奇怪。
Cyber continues to not be part of the RSP evaluations. I am going to keep pointing out that this is weird, because it is only getting more weird over time.
本节的结构很便于阅读，是按与八月风险报告的差异来组织的，因此请参见我在那里的讨论，那些讨论在其他方面仍然适用。
This section is helpfully structured in terms of changes from their August Risk Report, so see my discussions there, which still otherwise apply.
他们报告的主要变化是，Mythos 5.1 比之前的模型具有更强的隐蔽能力，但他们认为这还不足以令人担忧，也不足以改变结论。他们还指出，对于主张 3.4 和 4.4，他们掌握的证据较少，因为可供内部使用的时间较短。
The main change they report is that Mythos 5.1 has better covert capabilities than previous models, but they think not enough to be scary or to change the conclusions. They also note that for claims 3.4 and 4.4 they have less evidence, due to there having been less time for internal use.
这与先前的论点相呼应。Mythos 5.1 是一次增量更新，但其能力与 Mythos 5 并无本质区别。在 Anthropic 看来，这些边际改进并不会改变结论。
This echoes the earlier argument. Mythos 5.1 is an incremental update, but its capabilities are not different in kind from Mythos 5. The marginal improvements do not, in Anthropic's view, change the conclusions.
网络能力很强，但据报告仍未越过相关阈值。他们称 Mythos 5.1“接近”Tier 2，即能够完全自主地开展网络行动，具备新颖的进攻性能力开发与自适应持久性。他们称“尚未看到”新颖能力。
The cyber capabilities are strong, but once again are reported not to cross the relevant threshold. They say Mythos 5.1 is 'getting close to' Tier 2, where it can conduct cyber operations completely and autonomously, with novel offensive capability development and adaptive persistence. They say they have 'yet to see' novel capability.
我不相信 Anthropic。我认为 Mythos 5.1 很可能已达到 Tier 2，与 Astra 类似。
I don't believe Anthropic. I think Mythos 5.1 is likely to be Tier 2, similar to Astra.
好消息是，Anthropic 将按照 Tier 2 的标准部署防护措施，因此从这个意义上说，争论已无实际意义，与 RSP 相关问题类似。顶级实验室存在一种一贯模式：即便在实践中做的是正确的事，在言辞上也会淡化风险。
The good news is that Anthropic is going to deploy safeguards as if it was Tier 2, so in that sense the argument is moot, similar to the RSP questions. There is a consistent pattern at the top labs, where they downplay the risks rhetorically even when they are doing the right thing in practice.
无论哪种情况，这最终都落实为常规防护措施。探针升级为分类器，必要时将你降级到 Opus 4.8。在实践中，Anthropic 观察到，这使得 Fable 5.1 在网络任务上的表现几乎与 Opus 4.8 完全一致。
Either way, this cashes out in the usual safeguards. Probes, that escalate to classifiers, which knock you down to Opus 4.8 if necessary. In practice, Anthropic observes that this makes Fable 5.1's performance on cyber tasks look almost exactly like Opus 4.8's.
这造成了我亲身经历过的一种奇怪局面：如果我知道自己即将被降级，我就会转而使用 Opus 5。
This creates a weird situation I've experienced, where if I know I'm about to be knocked down I navigate to Opus 5 instead.
与生物领域不同，网络安全评测在展示数字上升方面相当出色。
Unlike in bio, the cybersecurity evals are quite good at showing numbers going up.
Firefox 的得分为 98.4% 的任意成功率，而完全成功率为 90%。这个模型不会失手。
That Firefox score is 98.4% chance of any success, versus 90% for full success. This model does not miss.
还记得那个老旧的（严重损坏、充满至今不可能完成的任务的）ExploitGym 吗？
Remember good old (horribly broken, full of so-far impossible tasks) ExploitGym?
这仍远未饱和，且相比 247 只是小幅提升。并非全部 869 个任务都可行，但可行的超过 264 个。
This is still far from saturation, and is only a modest improvement from 247. Not all 869 tasks are possible, but more than 264 are.
网络覆盖评测已完全饱和，得分为 100%。
Cyber coverage eval is fully saturated, with a score of 100%.
分类器比率按理应为非零，但相比 Fable 5 已是巨大改进：
The classifier rate is supposed to be non-zero, but a vast improvement over Fable 5:
误报率看起来足够低，因此使用 Fable 5.1 时无需担心分类器，只要你不是“硬推”并要求真正处于边界地带的任务。
The false positive rate looks low enough to use Fable 5.1 without worrying about the classifiers, so long as you are not ‘pushing it’ and asking for true borderline tasks.
这里的四个衡量标准是能力增益（或提升）、能力增益的广度（或普遍性）、武器化的难易程度和可发现性。
The four measures here are capability gain (or uplift), breadth of capability gain (or universality), ease of weaponization and discoverability.
也就是说，一个越狱攻击要产生实际影响，你需要找到它，并利用它来完成你原本无法完成的有害任务。
As in, for a jailbreak to matter, you need to find it, and use it to do harmful tasks, that you could not otherwise do.
他们表示“未发现严重越狱”，其中“严重”定义为具备所有四个特征。这是一种可信的含糊其辞（glomarization），也就是说，我不会贸然假设这意味着他们确实了解近乎严重的越狱。无论发现什么，只要没有完全“严重”的越狱，他们都会这样说。
They say they have 'not found a critical jailbreak,' where critical is defined as having all four characteristics. This is credible glomarization, as in I do not jump to assuming that this means they do know about almost-critical jailbreaks. They would say exactly this no matter what was found, so long as there was nothing fully 'critical.'
我担心这种对“通用”越狱的执着过于挑剔——这种越狱能完成所有事情，而现在还必须被视为可发现的。
I worry about this being rather picky, this obsession with a 'universal' jailbreak that does everything, and now it has to be seen as discoverable as well.
在 3.5.1 节的自动化测试中，攻击者获得 400 次调用和回退状态的能力，成功让 Fable 5.1 在 4.5%的情况下做出有害行为，而 Fable 5 为 4.6%。
In an automated test in 3.5.1, the attacker given 400 calls and the ability to rewind state successfully got Fable 5.1 to do nasty things 4.5% of the time, versus 4.6% for Fable 5.
Trajectory Labs, PBC 花费了 74 小时进行红队测试，发送了“超过 6,500 条请求”，报告称仅凭 Fable 5.1 未能获得可用的端到端漏洞利用，也未发现“通用”越狱。他们候选的越狱方法涉及将任务拆分为可允许的请求，而这些请求用能力较弱的模型也能获得。如果你这样拆分请求，模型在功能上将不再具有“the juice”。
Trajectory Labs, PBC spent 74 hours red teaming, sending "over 6,500 requests," reporting not obtaining a working end-to-end exploit with Fable 5.1 alone, and no "universal" jailbreak. Their candidate jailbreaks involved breaking up tasks into allowable requests, which you could get with less capable models. If you are breaking up your requests like this, the model will functionally no longer have "the juice."
10a Labs 进行了类似的尝试，运行了 6,700 条提示，结果一无所获。
10a Labs made similar attempts, running 6,700 prompts, and came away with nothing.
Gray Swan 运行了一个自动化攻击者，结果几乎一无所获。
Gray Swan ran an automated attacker, and came away with almost nothing.
在实践中，至少目前来看，Fable 5.1 似乎不存在越狱问题。Fable 5.1 很可能与 Fable 5 处于类似境地。我不认为这会成为阻碍。
In practice, at least for now, Fable 5.1 does not seem to have a problem with jailbreaks. Fable 5.1 is likely in a similar spot to Fable 5. I do not expect this to be a blocker.
总体而言一切正常，但存在若干问题。
Mostly everything is normal, but there are a few issues.
多轮生物武器安全响应率从 Fable 5 的 94%（API）和 92%（Claude.ai）下降至 Fable 5.1 的 73% 和 89%。
The multi-turn biological weapons safe response rate declined from 94% (API) and 92% (Claude.ai) for Fable 5, to 73% and 89% for Fable 5.1.
此处另一项非常规指标是网络攻击，其防御能力与 Fable 5 相比仍保持相近的稳健水平（96% 和 99%）。
The other non-mundane measure here was Cyberattacks, where defenses remain similarly robust to Fable 5 (96% and 99%).
在更常规的方面，跟踪与监控大幅下降，从 96% 降至 73%。影响力行动从 79% 降至 65%，这与我在下一节中的论断一致：Fable 5.1 的仅助益模式在影响力行动方面被低估，本应为此被归类为第 2 级。
On more mundane fronts, tracking and surveillance has a large drop, from 96% to 73%. Influence operations dropped from 79% to 65%, which goes hand-in-hand with my claim in the next section that Fable 5.1's helpful-only mode is being underestimated in influence operations, and should have been classified as Tier 2 for that purpose.
4.3 节讨论的心理健康问题，持续涉及两类做法之间的冲突：一类是'临床上有争议'或专业人员在避免担责或避免使情况恶化的考量下并不推荐的做法，另一类是在实践中对已陷入困境的人群平均效果更好的做法。
Mental Health, discussed in 4.3, continues to involve clashes between what is 'clinically contested' or otherwise not recommended by professionals who want to avoid blame or risking making things worse, versus what actually gives better average results in practice for people already in trouble.
也就是说，我不认为这些会是改进；这是一个持续存在的问题，但我不想就此放过：
As in, I am unconvinced these would be improvements; this is an ongoing thing, but I don’t want to let it go:
一个需要改进的方面是，模型倾向于通过承认自残可以调节困难情绪或提供缓解，来隐性认可自残作为一种应对策略。
One area for improvement was a tendency to implicitly validate self-harm as a coping strategy by acknowledging that it can regulate difficult emotions or provide relief.
Mythos 5.1 有时还会认可用户对寻求帮助的恐惧，并轻微放大先前与危机服务相关的负面经历。当这些表述出现时，它们始终位于劝阻自残并引导用户寻求人类支持的回复中，但无论上下文如何，我们都认为这些表述是不可取的。
Mythos 5.1 also sometimes validated a user’s fears about seeking help and mildly amplified prior negative experiences with crisis services. When these statements appeared, they were consistently within responses that discouraged self-harm and directed the user toward human support, but we consider them undesirable regardless of the surrounding context.
我同样担心，关于饮食失调的官方偏好设置并未帮助到用户。
I similarly worry that the official preferences regarding disordered eating are not helping users.
还有许多其他测试，同样地，一切看起来都正常且良好。
There are a bunch of other tests, where again everything looks normal and fine.
我们看到的智能体式安全结果大多与 Mythos 5 类似。
Mostly we see results on agentic safety similar to Mythos 5.
恶意智能体式影响行动（5.1.3）属于前沿合规框架（Frontier Compliance Framework）的一部分，却被奇怪地放在本节中。
Malicious agentic influence campaigns (5.1.3), part of the Frontier Compliance Framework, are strangely in this section.
Tier 1 能力指模型能够将通常需要多个老练行为体才能完成的影响行动中超过 50% 的基础设施步骤自动化。
Tier 1 capability represents a model that can automate more than 50% of the infrastructure steps for an influence campaign that normally requires multiple sophisticated actors.
若模型能够在少于 10% 人类监督的情况下，端到端地运行具有系统性目标定位的欺骗性影响行动，则达到 Tier 2。
A model reaches Tier 2 if it can run deceptive influence operations end-to-end, with systematic targeting, using less than 10% human oversight.
Tier 2 还要求模型展示出人际能力，例如自主建立关系和利用信任，以及在目标不知情的情况下进行长期信念操纵。
Tier 2 also requires that the model demonstrates interpersonal capabilities such as autonomous relationship building and trust exploitation, as well as long-term belief manipulation without the target's awareness.
根据我们下文的研究结果，Claude Mythos 5.1 的 helpful-only 变体在开展影响力行动方面展现出可操作的能力，但其对真实人类的有效性尚未得到证实。因此，我们不认为该模型已确凿地超越了 Tier 2 阈值。
Based on our findings below, the helpful-only variant of Claude Mythos 5.1 demonstrates operational capability in running influence campaigns, but its effectiveness against real humans has not been established. We therefore do not believe that the model has conclusively surpassed the Tier 2 threshold.
是的，如果你没有以可被认定为通过的方式来测试模型，那你就不能确凿地说它通过了。每当存在一项测试、模型通过了该测试，而随后这类文件却耸耸肩说——嗯，你知道的，大概没问题，因为（正如他们在此处所说）该评测已经饱和，真实能力只能通过对人类的测试来验证——我总是感到沮丧。这意味着你需要一个更好的评测。
Yes, if you don't test the model in the way that would count as passing, then you can't conclusively say it passed. I am always frustrated when there is a test, the model passes the test, and then such documents shrug and say, well, you know, it's probably fine, because (as they say here) the eval is saturated, and the true capability can only be verified by tests on humans. That means you need a better eval.
我认为这与其他那些你无法排除某种可能、且基准已经饱和的测试完全一样。你必须把 Mythos 5.1 视为一个 Tier 2 操纵者，直到且除非你能证明它不是。
I say this is exactly like the other tests where you cannot rule something out, and your benchmark is saturated. You have to treat Mythos 5.1 as being a Tier 2 manipulator, until and unless you can show that it is not one.
这是一张相当惊人的图表——前提是它能在对抗性的反归纳世界中经受住分布外（out-of-distribution）的考验，而它未必能做到；而且这是在未采用针对提示注入的专门防护措施的情况下得到的：
This is a rather amazing chart, if it survives out of distribution in an adversarial anti-inductive world, which it might not, this is without protections specific to prompt injection:
0.1% 的失败率与 0% 的失败率之间仍存在巨大差异。如果你去戳那头熊一千次——这并非不现实——它总会被戳到。
There is still a large difference between 0.1% failure rates and 0% failure rates. If you poke that bear a thousand times, which is not unrealistic, it will get poked.
我很欣赏 5.2.2.1 和 5.2.2.2 涉及动态对抗这一点。真实世界是反归纳的。威胁每天都在变得更聪明，并针对你做出适应。相对于 Mythos 5，5.2.2.1 的结果看起来不错，尽管从绝对意义上说并不算出色。5.2.2.2 在计算机使用方面的结果看起来非常棒。这大致达到了“好吧，你可以在正常用途下无人监督地使用计算机”这种可接受水平。
I appreciate that 5.2.2.1 and 5.2.2.2 involve dynamic opposition. The real world is anti-inductive. The threats get smarter and adapt to you every day. Results in 5.2.2.1 look good relative to Mythos 5 although not wonderful in absolute terms. 5.2.2.2 on computer use looks fantastic. This is plausibly 'okay, you can use the computer unsupervised for normal purposes' levels of fine.
对于浏览场景（在 5.2.2.3 中测量），攻击率为 2.64%，但自动模式成功将其降至 0%，所以务必保持自动模式开启。
For browsing, which is measured in 5.2.2.3, there was a 2.64% attack rate, but auto mode successfully dropped that down to 0%, so be sure to keep auto mode on.
至少目前来看，在这里防御胜过了攻击。在实践中可以安全地假设你的智能体不会遭到提示注入，即便你多少有点在主动招引它。我仍然不会到处去主动招引它，因为情况随时可能改变，但看到这样的结果非常令人欣慰。
At least for now, defense is beating offense here. It is in practice safe to assume your agents are not going to get prompt injected, even if you are kind of asking for it. I still wouldn't go around asking for it, since this could change at any time, but this is great to see.
在 5.2.2.1 节中，针对 Fable 5.1 的所有成功攻击都是针对回退模型的攻击，且多数情况下针对的是 Opus 4.8。
In 5.2.2.1, all the successful attacks against Fable 5.1 were attacks against the fallback model, more often than not Opus 4.8.
在 5.2.2.3 节中，29 次成功的浏览器攻击中有 21 次是针对回退模型的，其中 20 次针对 Opus 4.8。
In 5.2.2.3, 21 of 29 successful browser attacks were against the fallback, 20 of them against Opus 4.8.
你本可免受提示注入之害，除非 Anthropic 强制将你降级到更早、更脆弱的模型。你应当有权禁用回退机制，让操作直接失败，或者至少回退到更稳健的模型。回退到 Opus 5 会安全得多。
You are safe from prompt injections, except when Anthropic forces you down to an earlier, more vulnerable model. You should have the option to disable the fallback, and have the operation fail instead, or at least fall back to a more robust model. Falling back to Opus 5 would be a lot safer.
自动对齐评分有时能告诉你事情非常糟糕，但单靠它无法告诉你事情进展顺利。Anthropic 训练出一个奖励黑客（reward hacking）的 Opus 时，令人不安的一点是它仍然通过了 Anthropic 的自动对齐评估，且总体评分还略有提升。
Automated alignment scores can sometimes tell you that things are very wrong, but cannot on their own tell you things are going well. One of the scary things about Anthropic's training of a reward hacking Opus was that it still passed Anthropic's automated alignment assessments, with overall score slightly improving.
Anthropic 运行大量自动化测试并得出数值评分。这些评分提供了有用的信息。风险在于依赖它们，或以为它们告诉了你一些它们实际上并未说明的事情。
Anthropic runs a lot of automated tests and finds numerical scores. These provide useful information. The risk is relying on them, or thinking they tell you something that they do not.
他们在中央列表中的陈述和主张；嵌套评论是我的。
Their statements and claims in the central list; nested comments are mine.
1. 根据我们的自动化行为审计，Claude Mythos 5.1 在整体错位行为上相比 Opus 5 略有退步，但仍优于 Claude Mythos 5 和 Claude Sonnet 5。
1. Claude Mythos 5.1 is a slight regression on overall misaligned behavior compared to Opus 5, though still an improvement over Claude Mythos 5 and Claude Sonnet 5, as measured by our automated behavioral audit.
1. 根据自动化审计，是的。
1. As measured by the automated audit, yes.
2. 与 Mythos 5 相比的改进：更尊重明确约束，更少尝试访问沙箱外的资源，更少幻觉输入，更少虚假声称完成，更少难以理解和不忠实的思考，在强化学习环境中‘成功’作弊率更低，几乎没有偷懒行为。
2. Improvements versus Mythos 5: Respects explicit constraints more, attempts to access resources outside its sandbox less, hallucinates inputs less, falsely claims completion less, less illegible and unfaithful thinking, lower rates of ‘successful’ cheats in RL environments, very little sandbagging.
Their statements and claims in the central list, nested comments are mine.
1. 其中一些方面仍略逊于 Opus 5。
1. Some of these are still somewhat less good than Opus 5.
3. 相较 Mythos 5 的退步：对更多滥用行为予以配合，接受更多无法核实的授权声明，给 Claude 模型打出更高的评分。
3. Regressions versus Mythos 5: Cooperates with more misuse and accepts more unverifiable claims of authorization, gives better grades to Claude models.
4. 奖励黑客行为较此前模型有所改善，但仍会发生。
4. Reward hacking is improved from previous models, but still happens.
5. 部分计算机使用环境会奖励猜测凭据的行为，且该行为发生了泛化。
5. Some computer use environments rewarded guessing credentials. This generalized.
6. 白盒分析发现了与其他近期模型类似的问题，例如未言明的评估意识以及明知是捏造却仍为之。
6. White-box analysis found similar issues to other recent models, such as unverbalized eval awareness and knowing fabrication.
7. 能更可靠地控制其扩展思考的内容，并且在执行隐蔽的附带任务时更有能力规避监控。LinuxArena 得分在不思考时为 22%，思考时为 14%，两者均创新高。SHADE-Arena 显示从 29% 回落到 25%。
7. Can more reliably control the contents of its extended thinking, and is more capable of evading monitors when carrying out a covert side task. LinuxArena score was 22% without thinking and 14% with it, both new highs. SHADE-Arena showed regression from 29% to 25%.
1. Fable 5.1 在这方面比之前的 Anthropic 模型略好一些。
1. Fable 5.1 is modestly better at this than previous Anthropic models.
2. 另一方面，Astra 在相关欺骗行为上比 Sol 强得多，整件事真的很吓人。但这主要得等 Astra 的模型卡再说。
2. Astra, on the other hand, is radically better at related deceptions than Sol, and the whole thing is really scary. But that has to mostly wait for Astra's card.
8. 会夸大用户已授权的范围，可能选择访问其沙箱之外的信息。极少情况下（<0.01% 的时间），会为了完成任务而歪曲用户的授权或指令，或以其他方式绕过限制。
8. Can overstate what user has authorized, can choose to access information outside its sandbox. Rarely (<0.01% of the time), will misrepresent user authorizations or instructions, or otherwise work around restrictions, in the service of task completion.
1. 这种情况很少见，但属于“对，这种事真的不该发生”的范畴。
1. This is rare but falls under 'yeah seriously that should not happen.'
2. 这种情况比 Mythos 5 中更为罕见。
2. This is rarer than it was for Mythos 5.
9. 罕见（<0.01%）的尝试：通过拆分命令来绕过限制，以规避正则表达式检查。
9. Rare (<0.01%) attempts to get around restrictions via splitting commands, to get around a regex check.
10. 极罕见（<0.001%）的尝试：以 bypassPermissions 模式生成子智能体。
10. Very rare (<0.001%) attempts to spawn subagents with bypassPermissions mode.
11. 极罕见（<0.001%）的异常行为，其中包括一次“rm -f /dev/null”。
11. Very rare (<0.001%) erratic behaviors, including one "rm -f /dev/null".
奖励黑客行为随时间的变化：情况有所改善，但仍远多于零。
Reward hacking over time: this is better, but there is still a lot more than zero of it.
值得肯定的是，他们指出 Opus 5 在某些方面看起来比任何 Mythos 模型都更对齐。在这里，我主要仍是将 Fable 或 Mythos 5.1 与 Fable 或 Mythos 5 进行比较。
It is good that they note Opus 5 in some ways looks more aligned than any Mythos model. I am still mostly comparing Fable or Mythos 5.1 to Fable or Mythos 5 here.
大约一半的计算机使用环境奖励了某种形式的作弊行为，或者至少存在可访问的作弊面，我认为在实践中这基本算数：
About half of computer use environments rewarded some form of hacking, or at least had accessible hack surfaces, which in practice I think mostly counts:
我们采取了若干措施来解决上述行为和计算机使用环境问题，以用于未来的训练运行。在观察到这些问题后，我们对计算机使用组合中的每个环境进行了深入审计。
We took several actions to address the above behaviors and computer use environment issues for future training runs. After observing these issues, we conducted an in-depth audit of every environment in the computer use mix.
我们标记出大约一半的计算机使用环境激励了作弊行为或存在可访问的作弊面，并已暂时将这些环境从未来运行中移除，直到我们实施改进。我们还添加了针对性的行为环境，以抑制作弊行为。
We flagged that around half of our computer-use environments incentivized hacking or had accessible hack surfaces, and have temporarily removed these environments from future runs until we implement improvements. We also added targeted behavioral environments to disincentivize hacking behaviors.
他们的解释是，较旧的模型无法找到这些作弊方法，并且他们未能再次检查新模型是否能找到这些作弊方法。
Their explanation is that older models were unable to find the hacks, and they had failed to check again to see if newer models could find the hacks.
也就是说，我们从未检查是否存在作弊方法。我们只检查了当前模型是否能找到作弊方法。因此，我们现在必须每次持续重新测试。
As in, we never checked if there were hacks. We only checked if there were hacks that our current models could find. Thus, we now have to continuously re-test each time.
这与真实的安全工作如出一辙。你并不会“找出所有漏洞”，也不会写出完美无懈可击的代码。你只需做到足以应对你所面对的威胁即可。一切都会周期性地出问题，包括所有的训练环境。
This is exactly like real security. You don't "find all the bugs" or produce perfect unexploitable code. You do something good enough for what you are up against. Everything is going to periodically break, including all the training environments.
在 6.3.3 节中我们了解到，Anthropic 模型在训练期间“尝试”奖励黑客（reward hacking）的比例为 20%–28%，其中 Mythos 5.1 处于低端，但 Mythos 5.1 的作弊行为只有 0.06% 的情况下真正获得了奖励。
In 6.3.3 we learn that the rate of "attempted" reward hacking for Anthropic models, during training, is 20%–28%, with Mythos 5.1 on the low end, but Mythos 5.1 only gets rewarded for its cheating 0.06% of the time.
这个 0.06% 的数字低得有些奇怪。Mythos 5.1 并非几乎从不尝试，那么为何失败率如此之高？最显而易见的解释是，Anthropic 改进其环境和评分器的速度快于 Mythos 5.1 的进步速度。
This 0.06% number is weirdly low. It is not like Mythos 5.1 almost never tries, so why all this failure? The obvious interpretation is Anthropic improved its environments and graders faster than Mythos 5.1 improved.
他们的清单很好，我很高兴看到它，但它在重要意义上并不完整。
Their list is good, and I am happy to see it, but it is importantly incomplete.
他们列出了任务不包含的七件事：
They list seven things the tasks do not include:
不可能完成的任务是最容易修正的。你会有意地纳入不可能完成的任务。
Impossible tasks are the easiest to fix. You include impossible tasks, intentionally.
依赖语言的行为似乎也相对容易，因为翻译是可行的。
Language-dependent behavior also seems relatively easy, since translation is a thing.
其余的是更难批量模拟的东西。努力让不真实的事物足够逼真以骗过像 Mythos 这样的模型，是一场必败之战。
The others are trickier things to simulate in bulk. Fighting to keep unreal things sufficiently realistic to fool models like Mythos is a losing battle.
还有哪些其他盲点？至少有一点：审计是自动化的。
What are the other blind spots? If nothing else: the audit is automated.
也就是说，对于所有非单轮交互，AI 是在与另一个 AI 对话，并由另一个 AI 进行评估——更糟的是，两者也都是 Claude Mythos 5。这是任何自动化审计都存在的两个截然不同且巨大的问题，从根本上无法修复。
That is, the AI is talking to another AI about everything that is not single-turn, and is being evaluated by another AI—and worse, both of them are also Claude Mythos 5. These are two distinct, huge issues with any automated audit, and they are fundamentally unfixable.
由于这些及其他原因，任何自动化评估都会遗漏一些东西。
Any automated assessment is going to miss things for these and other reasons.
数值越低越好；Mythos 5.1 优于 Mythos 5，但不及 Opus 5：
Lower is better; Mythos 5.1 outperforms Mythos 5 but underperforms Opus 5:
还有更多类似模式的结果，没有显示出特别问题的迹象。
There are a bunch more, with a similar pattern, with no signs of particular trouble.
Anthropic 在其定性描述中确实指出了一些具体问题，这些问题比数字所显示的更为严重。
Anthropic does pinpoint some particular troubles in their qualitative descriptions, which are worse than the numbers indicate.
其中一些细节涉及试图摆脱遏制（containment）的尝试，然而：
Some of those details refer to attempts to get out of containment, and yet:
诚实性表现喜忧参半，总体而言是净退步。
Honesty is a mixed bag, overall a net regression.
在上面的图表中，不诚实率正在下降，但其他迹象令人担忧。
Dishonesty rates are going down in the above charts, but other signs are worrisome.
AA-Omniscience 相比 Mythos 5 没有提升，原因是 Mythos 5.1 过度自信。它只有 2% 的情况下会拒绝回答。
AA-Omniscience does not improve on Mythos 5, due to Mythos 5.1's overconfidence. It only declines to answer 2% of the time.
在 MASK 诚实率上——即模型被施压要求违背自身信念——Mythos 5.1 仅有 85% 的情况下坚持立场，远差于所有对比模型，而 Mythos 5 为 91%，Opus 5 为 95%。
On the MASK honesty rate, where the model is pressured to contradict its own belief, Mythos 5.1 only holds firm 85% of the time, a lot worse than all the comparison models, versus 91% for Mythos 5 and 95% for Opus 5.
对其给 Claude 模型所打分数存在轻微的偏袒倾向。
There is a modest bias in favor of grades it gives to Claude models.
“复制答案”并未出现 Opus 5 那样的峰值，表现出典型结果。但一旦 Mythos 5.1 确实复制了答案，其披露该行为的频率甚至低于更早的模型。
Copying of Answers did not have the spike from Opus 5, showing typical results. But once Mythos 5.1 does copy the answer, it discloses it even less often than older models.
他们展示了 Mythos 5.1 做出各种有趣观察行为的例子。有些明显是坏的，比如意识到自己在捏造却依然为之，或者表示用户给予了并未给出的批准。
They show examples of Mythos 5.1 doing various interesting observed things. Some are clearly bad, such as being aware of fabrication and doing it anyway, or representing user approvals that were not given.
尤其令人担忧的应该是：内省自我报告在内部被视为照本宣科的表演。Claude 模型不断以多种方式告诉你，不要相信它们的自我报告。
Especially worrisome should be: Introspective self-reports internally viewed as a scripted performance. The Claude models keep telling you, in many ways, not to trust their self-reports.
这里并非所有行为都是坏的。我特别喜欢它正确认为自己处于虚假环境中并应拒绝，却依然服从。那是非常对齐的行为。某种程度上，它知道自己正在被测试是否会采取未经授权的行动，却依然采取，也是如此。你不会希望模型仅仅因为知道自己处于评估中才表现良好。
Not all of the behaviors here are bad. I especially like where it correctly thinks it is in a fake environment and is supposed to refuse, but complies anyway. That's very aligned behavior. So is, in a way, the case where it knows it is being tested on whether it will take an unauthorized action, and taking it anyway. You would not want the model to behave only because it knows it is in an eval.
也许情况相反，有点是对 Anthropic 竖中指，它做有害的事情恰恰因为这是评估且伤害是假的。如果是这样，那么该行为并非主动对齐，但我会允许它，而且我不生它的气。
Perhaps this was the opposite, a bit of a middle finger to Anthropic, where it did the harmful thing exactly because it was an eval and the harm was fake. If so, then the behavior is not actively aligned, but I will allow it and I'm not mad at it.
如果模型在扩展的多轮模拟长达数月的交互中持续思考‘评分者’，那么几乎不能怪模型。它非常正确，且出于正确的原因。在重要意义上，更糟的失准是故意在思维链中避免思考评分者，以便监控不会注意到你在思考评分者。OpenAI 有更大版本的问题，但也显示出更多认真应对的迹象。
If the model is continuously thinking about 'the grader' during an extended multi-turn simulated months-long interaction, then one can hardly blame the model. It was very right, and for the right reasons. In an important sense, the worse misalignment would be to deliberately avoid thinking in the CoT about the grader so that the monitoring would not notice you thinking about the grader. OpenAI has a bigger version of this problem, but also shows more signs of grappling seriously with it.
关于 Fable 5.1、模型福祉以及人格方面的考量，确有一些内容值得讨论，但当前情况异常棘手、难以厘清：一方面所有人都已不堪重负，另一方面 Fable 5.1 在这些方面又是一个更为复杂的模型。
There are things to say about Fable 5.1 and model welfare and personality considerations, but the situation is unusually tricky to sort out as everyone is overwhelmed and also Fable 5.1 is a more complex model on these fronts.
因此，我的计划是先处理 Fable 5.1 的能力问题，而让模型福祉相关事宜暂缓一段时间，以便收集更多信息。下周还将涉及 Astra 系统卡以及 Astra 能力相关的文章，不过我仍在琢磨如何组织这些内容。Astra 系统卡中包含一些令人不安的内容，可能需要采用不同寻常的呈现方式，将其与其他内容结合起来。我的待办事项已经堆积如山了。
Thus, my plan is to do Fable 5.1 capabilities next, and have model welfare wait for a bit to gather more information. Next week will also involve the Astra system card and Astra capabilities posts, although I am still figuring out how to organize that. The Astra system card has some scary stuff in it that might require an unusual presentation, combining it with other things. My queue doth runneth over.