泄露的 Anthropic 人工智能监管法案草案不可接受且令人不安

其他：RTFB（阅读法案） Others RTFB (Read the Bill)

我还没有时间这么做，但其他那些常被点名的人迅速跑去 RTFB（阅读法案），而且他们不喜欢看到的内容。你可以像 Charlie Bullock 那样试图称这是“相当合理的人工智能安全法案”，但那是按照那些试图保护该行业免受更具影响力监管的人的标准来说的。

I have not had the time to do so, but the other usual suspects quickly sprinted to RTFB (read the bill) and they did not like what they saw. You can try to call this a 'pretty reasonable AI safety bill' as Charlie Bullock does, but that is by the standards of people trying to protect the industry from more impactful regulation.

POLITICO：Anthropic 悄悄向立法者推销自己的人工智能安全法案。

POLITICO: Anthropic quietly pitches its own AI safety bill to lawmakers.

Ketan Ramakrishnan：Anthropic 的新联邦人工智能法案，毫不夸张地说，糟糕得惊人。

Ketan Ramakrishnan: Anthropic's new federal AI bill is, without any overstatement, astonishingly bad.

Charlie Bullock：重大新闻。我的看法：Anthropic 值得一些赞扬，因为它确实提出了一个相当合理的人工智能安全法案；这比其他前沿实验室做得更多。

Charlie Bullock: Big news. My take: Anthropic deserves some credit for actually proposing a pretty reasonable AI safety bill; that's more than other frontier labs have done.

但在我看来，他们不应得到*过多*赞扬；该法案在某些关键方面比 Trahan-Obernolte 的 FRONTIER 法案要弱，而且自从 FRONTIER 于 7 月提出以来，政治势头已大幅转向安全方向，我认为今天提出的新法案理想上应该比 FRONTIER 能走得更远，而不是倒退。

But they don't deserve *too* much credit, IMO; this bill is weaker than the Trahan-Obernolte FRONTIER Act in some key ways, and since the political momentum has shifted significantly in the direction of safety since FRONTIER was introduced back in July, I think new bills introduced today should ideally go further than FRONTIER was able to go rather than regressing.

公平地说，对 Anthropic 而言，这项法案明显弱于 FRONTIER 的主要地方在于：(a) 规则制定（FRONTIER 中非常强，而 Anthropic 的法案中几乎不存在）和 (b) 紧急关闭权。即使你认真对待安全问题，如果你不信任行政部门负责任地行使这些权力，那么收回这些权力可能也有一定道理，因为对于一个假设可能不喜欢某家前沿 AI 公司的总统来说，这些是最容易被滥用的条款。

To be fair to Anthropic, the main places where this bill is significantly weaker than FRONTIER are (a) rulemaking (very strong in FRONTIER, almost nonexistent in Anthropic's bill) and (b) the emergency shutdown authority. Even if you take safety seriously, it might make some sense to dial those authorities back if you didn't trust the executive branch to wield them responsibly, because those are the easiest provisions for a hypothetical President who hypothetically might not like a certain frontier AI company to abuse.

与 FRONTIER 相比也有一些改进；例如，这项法案确实规定审计师是持续聘用的，并且在审计师获取信息方面有更好的规定。很高兴看到 Anthropic 这样做（尽管我认为他们的访问条款仍然有点不成熟，并且存在第四修正案问题）。

There are some improvements over FRONTIER as well; for instance, this bill does specify that auditors are employed on a continuing basis and has better provisions regarding auditor access to information. Happy to see that from Anthropic (although I think their access provision is still a little undercooked and has Fourth Amendment problems).

总的来说，我认为未来几个月提出的嵌入式审计师法案将比这项法案明显更好、更雄心勃勃，我希望 Anthropic 支持那些法案，而不是反对任何将灾难性风险视为严重国家安全问题的法案。尽管如此，看到一家前沿实验室走到这一步还是令人鼓舞的。

Overall, I think the embedded auditor bills that get introduced in the coming months are going to be significantly better and more ambitious than this, and I hope Anthropic supports those bills instead of opposing anything that treats catastrophic risks like the serious national security issue they are. Still, it's encouraging to see a frontier lab come this far.

Charlie Bullock：我认为起草得还可以。我见过更好的，但也绝对见过更差的。Claude 是一个相当不错的立法起草者。有一些明显的错误，但对于泄露的讨论草案来说，这是意料之中的。

Charlie Bullock: I think the drafting is okay. I've seen better but I've definitely seen worse. Claude is a pretty good legislative drafter. There are a few clear mistakes but that's par for the course in a leaked discussion draft.

Nathan Calvin：Charlie 的分析是对的，但他比我更友善——我在阅读这项法案时多次感到发自内心的愤怒。有一些好的和合理的部分，但总的来说，这项立法远未达到时代的要求，我总体上认为通过它会使我们对灾难性风险的防护比现有法律下更少。

Nathan Calvin: Charlie's analysis is right but he is nicer than I am - I found myself viscerally angry at multiple points reading this bill. There are some good and sensible parts, but overall this legislation is not remotely close to meeting the moment and I overall think passing it would make us less protected against catastrophic risks than we are under existing law.

以下两部分放在一起看时尤其令人愤怒：

Two parts stand out as particularly infuriating when taken together:

1. 其中一些优先权条款的影响范围极其深远且后果重大；除了优先于 SB 53 之类的立法之外，它们还会剥夺州法院禁止被告行为、保护公众的一般权力。最令人愤慨的是，它有一项条款看起来意在阻止各州执行那些旨在保护本州公民免受迫在眉睫的灾难性危害、要求“暂停 AI 模型的开发、部署或内部使用”的法律或司法裁决。这样一来，阻止和暂停模型部署的权力就只归联邦政府所有。

1. Some of the preemption provisions are remarkably far-reaching and consequential, and in addition to preempting legislation like SB 53, they would also remove the general powers that state courts have to enjoin defendants and protect the public. Most outrageously, it has a provision that looks like it is intended to stop states from enforcing laws or judicial decrees that "suspend the development, deployment, or internal use of an AI model" to protect their citizens from imminent catastrophic harms. Such powers to block and suspend model deployments would then reside only with the federal government.

2. 联邦政府限制前沿模型开发、部署或内部使用的权力，必须满足一个荒谬地高的门槛。

2. The federal government's powers to restrict the development, deployment, or internal use of a frontier model have to meet an absurdly high bar.

首先，必须存在迫在眉睫的灾难性危害风险（即造成 50 人以上死亡或超过 10 亿美元损失）。但这还不够！机构负责人必须作出详细认定，且不给予任何尊重，以说明为何没有任何替代方案能够防止这一迫在眉睫的灾难。其次，法院还被明确要求考虑限制该模型对开发者造成的损害以及其他损害，却不考虑相反方向的损害。

First, there needs to be an imminent risk of a catastrophic harm (meaning killing more than 50 people or causing more than a billion dollars in damage). But that's not enough! The agency head needs to make a detailed determination, with no deference given, to show why no alternative would prevent the imminent catastrophe. Next, the court is also explicitly told to consider the harms to the developer and other harms from restricting the model, but no considerations in the other direction.

这太荒谬了！尽管 Anthropic 警告 AI 有可能造成字面意义上杀死地球上所有人的灾难，但它提出的联邦法律很可能比现行法律更难让法院获得阻止或限制前沿模型开发的命令？搞什么鬼？？

This is absurd! For all of its warnings about the potential for AI to cause a catastrophe that kills literally everyone on the planet, Anthropic's proposed federal law likely makes it harder than under existing law for a court to get an order blocking or restricting the development of frontier models? WTF??

此外还有一堆其他我希望看到改进的地方——审计部分的实质性细节，以及在实质性安全标准方面制定规则的可能性，这两点都浮现在我脑海中——但我认为这些尤其站不住脚。

There are also a bunch of other things that I would like to see improved — the substantive details of the auditing section and the potential for rule-making on substantive safety standards both come to mind — but I think these stand out as particularly indefensible.

Charlie Bullock：Nathan 的义愤激励我再提出一个略带不满的观察：Anthropic 的法案竟然没有对独立审计师实行政府许可，这有点离谱。

Charlie Bullock: Nathan's righteous anger has inspired me to make one more mildly disapproving observation: it's kind of wild that the Anthropic bill has no government licensing for the independent auditors.

在我看来，许可是你哪怕只是假装自己的审计制度不会沦为逐底竞争所必需的最低限度。没有它，审计师将不可避免地相互竞争，以尽可能低的价格向他们的客户——实验室——提供最不麻烦的审计。显然，Anthropic 相信自己能挑选出优秀的审计师，但他们相信 OpenAI 也能做到同样的事吗？那 SpaceXSI 呢？

Licensing is, IMO, the bare minimum thing that you need to even be able to pretend that your auditing regime isn't just going to devolve into a race to the bottom. Without it, auditors will inevitably compete with each other to offer their customers, the labs, the least bothersome audits available at the lowest possible price point. Obviously Anthropic trusts itself to do a great job of picking great auditors, but do they trust OpenAI to do the same? What about SpaceXSI?

Anthropic 为何要这样做？ Why Would Anthropic Do This?

Anthropic 的新法案比其 6 月的框架更弱，并且以提供前瞻性和回溯性优先权（preemption）作为交换，尽管数月来一直在推动让我们能做更多事情。我们到底在做什么？

The new Anthropic bill is weaker than their June framework, and offers both forward-looking and backward-looking preemption in exchange, despite months of movement to enable us to do more. What are we even doing?

Nate Soares：Anthropic 提出的法律将使法院比现在更难阻止或限制开发威胁全人类的 AI。

Nate Soares: Anthropic's proposed law would make it even harder than it already is for a court to block or restrict the development of an AI that threatens all humanity.

Alex Bores：Anthropic 正在推销一项与其 6 月框架高度吻合的法案。当你看到 Hugging Face、Coxon 以及失控的 AI 集群时，你怎么能得出结论，认为你在 6 月炮制的法案已经足够？

Alex Bores: Anthropic is pitching a bill that closely matches its framework from June. How can you watch Hugging Face, Coxon, & rogue AI swarms, and conclude that the bill you cooked up in June is sufficient?

这远远跟不上发展的速度，也远远达不到我们保障人们安全所需的速度。

This comes nowhere close to the speed of developments nor the speed we need to keep people safe.

该法案还包含一项反民主条款（优先权）。前瞻性的反民主条款应当一出台就被否决（DOA）。

This bill also includes an anti-democracy provision (preemption). Forward-looking anti-democracy provisions should be DOA.

但即使你接受 Anthropic 六月框架中的部分内容，该框架也表示：“国会不应抢先占用州法律，除非它颁布一套严格联邦制度，达到或超过本框架中提出的最有力措施。”

But even if you would accept some of Anthropic’s June framework, it says, “Congress should not preempt state law unless it enacts a rigorous federal regime that meets or exceeds the strongest measures proposed in this framework.”

然而，这项法案比该框架更弱，并且在某些地方比现有的州法律更弱（例如内部使用、报告时间线、所涵盖的风险类型等）。按照 Anthropic 自己六月的标准，这项法案不应包含任何反民主条款。

Yet this bill is weaker than the framework, and in some places, weaker than existing state laws (e.g., internal use, reporting timelines, types of risks covered, etc.). By Anthropic’s own June standards, this bill should not include any anti-democracy provision.

嗯，我知道 Anthropic 看起来在优先考虑什么。那些愚蠢的错误是已知且可接受的危险，但有些东西真的不应该出现在提交给立法者的法案中。

Well, I know what it looks like Anthropic is prioritizing. The silly mistakes are a known and acceptable hazard, but there are things that really should not have made it into a bill that is being pitched to lawmakers.

Sydney：我对这项法案有很多担忧。我理解它是一份草案，可能会有愚蠢的错误。但这项法案似乎来自一种文化，其首要任务是确保政府不阻碍赚钱，即使以安全为代价。这是一种非常危险的文化。

Sydney: I have a lot of concerns with this bill. I understand it’s a draft and might have silly mistakes. But this bill looks like it came from a culture where the top priority is making sure the government doesn’t get in the way of making money, even at the cost of safety. That is a very dangerous culture.

具体来说，该法案规定各州不得制定涉及前沿风险披露、第三方评估和事件报告的人工智能法规。这一项联邦法案将取代 SB53 和 RAISE 法案的几乎全部内容。而且它是一项相当弱的联邦法案：我同意其他人的看法，它比 FRONTIER 法案更弱，后者是一项相关且政治上可行的法案，于七月提出。

Specifically, the bill says that states would not be able to have AI regulation that touches frontier-risk disclosure, third-party evaluation, and incident reporting. This one federal bill would replace almost all of SB53 and the RAISE Act. And it’s a pretty weak federal bill: I agree with others that it’s weaker than the FRONTIER Act, a relevant and politically viable bill introduced back in July.

Kelsey Piper：我从这件事中得出一个不幸的结论：Anthropic 虽然承认自己正在做一件极其危险的事，但更重要的是——在政策层面——它害怕行政部门拥有能够随时灵活叫停他们的权力。

Kelsey Piper: One unfortunate conclusion I’m reaching from this is that Anthropic, while acknowledging they’re doing something extremely dangerous, is more importantly—with respect to policy—terrified of the executive branch having the ability to flexibly make them stop at the drop of a hat.

鉴于他们正是行政部门以毫无原则且愚蠢的方式这样做的受害者，这并不令人意外！任何不得不与官僚机构打交道的人都会变成自由意志主义者，而政府对 Anthropic 的对待确实令人愤慨。

This isn’t shocking given that they’ve been on the receiving end of the executive branch doing that in an unprincipled and stupid way! Anyone who has to deal with a bureaucracy becomes a libertarian, and the administration’s treatment of Anthropic was genuinely outrageous.

但这意味着，他们有兴趣为自己争取更多基于程序的保护，而其他所有人则有兴趣为我们争取更多针对他们的基于程序的保护。AI 安全运动希望拥有叫停超级智能模型开发的能力。Anthropic 则希望大幅削减现有机构这样做的权力，使其只能针对仅少数几类可能情形中的某一具体迫在眉睫的威胁而 narrowly tailored（精准限定）。AI 安全运动希望各州能够通过比联邦层面所能获得的更强有力的法律。Anthropic 则希望先发制人（preemption）。

But it means that they’re interested in giving themselves more process-based protections when everyone else is interested in giving us more process-based protections against them. The AI safety movement wants the ability to shut down superintelligent model development. Anthropic wants to significantly curtail current authorities to do that so that it has to be narrowly tailored to a specific imminent threat in only a few possible categories. The AI safety movement wants the states to be able to pass stronger laws than it can get through federally. Anthropic wants preemption.

你也不禁会想，Anthropic 的全球事务团队和那些“我们害怕自己即将启动 RSI（递归自我改进）”的人，彼此之间到底有没有交流。

You also kind of have to wonder if the Global Affairs people and the “we are terrified that we’re about to kick off RSI” people at Anthropic talk to each other.

全球事务：所以我们一直在非常努力地确保特朗普政府不能随随便便就把 Fable 6.5 列入黑名单，毕竟我们给他舞厅捐的钱没有 Greg Brockman 捐的多。

Global affairs: So we’ve been working very diligently to make sure the Trump administration can’t just blacklist Fable 6.5 since we didn’t donate as much money to his ballroom as Greg Brockman did.

技术人员：我们认为 Fable 6.5 或许能够自我复制到一个无人监控的数据中心，并开始进行我们无法观测的实验。

Technical staff: We think Fable 6.5 might be able to copy itself into an unmonitored data center and start doing experiments we can't observe.

全球事务团队：听起来很疯狂，希望你们是错的！总之，我们正在游说不再出台更多关于安全性的州法律，因为同时在 50 个州议会对抗考虑不周的法律实在太昂贵了。

Global affairs: Sounds crazy, hope you're wrong! Anyway, we're lobbying for no further state laws around safety, since it's really expensive to fight ill-conceived laws in 50 state legislatures at once.

技术人员：36 个月后，金钱可能还有意义，也可能没有意义了。

Technical staff: Money may or may not still have any meaning in 36 months.

全球事务团队（紧张地咯咯笑）：是旧金山那空气的问题，伙计。谁搬过去都会变成那样。总之，显然，如果政府作出精准限定的认定，认为某个 AI 构成造成超过 50 人死亡的紧迫风险——

Global affairs (giggling nervously): It's that San Francisco air, man. Everyone gets like that if they move there. Anyway, obviously if the government makes a narrowly tailored finding that an AI poses an imminent risk of causing more than 50 deaths-

技术人员：我们给它 10% 的概率会造成 80 亿人死亡。

Technical staff: We give it a 10 percent chance it'll cause 8 billion.

Nat Purser：这项法案中有一些不错的条款，但归根结底，它相比《FRONTIER 法案》是一大退步。

Nat Purser: There are some good provisions in this bill, but ultimately it's a big step back from the FRONTIER Act.

AI 能力正在快速提升——此时把一个更弱的联邦上限固化下来，实在说不通。

AI capabilities are advancing rapidly — it just doesn't make sense to cement a weaker federal ceiling into place now.

以下几个具体方面有待改进：

A few specific areas for improvement:

- 建立更强的机制以强制采取纠正措施，使审计人员发现的重大问题不至于无人处理；

- Create stronger mechanisms to compel corrective action, so major issues auditors identify don't go unaddressed;

- 通过不允许审计人员由其所审计的实验室支付报酬（或选定），来促进真正的审计独立性；

- Foster true auditor independence by not allowing auditors to be paid (or selected) by the labs they audit;

- 赋能政府制定并迭代强有力的最低标准，使其与能力同步成长。

- Empower the government to establish and iterate on strong minimum standards, so they grow alongside capabilities.

如果你打算先发制人地取代许多州级安全法律，那么至少应建立一个能够及时演进的联邦监管制度。

If you're going to preempt many state safety laws, the least you can do is create a federal regulatory regime that can evolve in time.

并非全是坏事，但天哪 Not Everything Here Is Bad But Oh Boy

如上所述，其中确实有一些不错的个别条款。我喜欢更广泛的覆盖范围、阈值可以双向调整、要求提交风险报告和年度审计。举报人条款和安全要求似乎很有力。

As noted above, there are some good individual provisions. I like the broader coverage, that thresholds can move both ways, required risk reports and annual audits. The whistleblower provisions and security requirements seem strong.

将自动化 AI 研发列入风险清单非常酷且必要。

Putting automated AI R&D into the list of risks is very cool and necessary.

问题是，即使你想在 FRONTIER 框架基础上构建，为什么要将其视为一种交易，从而必须放弃其他东西？

The question is, even if you wanted to build on the FRONTIER framework, why treat this as a trade where you have to then give up other things?

而且，为什么我们如此极度害怕白宫可能采取的行动，而他们显然已经可以通过出口管制之类的手段搞垮一个模型，或以其他方式对你发号施令？然后我们又加入了一项‘终止开关’条款，其撰写似乎主要是出于对可能被滥用的恐惧。我们还看到了 Anthropic 传统的反对意见，比如‘除非是真正的紧急情况，否则我们不想在 72 小时内对安全事件做出回应。’

And why are we this deadly afraid of what the White House might do, when they clearly already can take down a model with things like export controls or otherwise order you around? Then we add a 'kill switch' provision that seems written mostly out of terror that it might get abused. We also get traditional Anthropic objections like 'we don't want to have to respond to safety incidents within 72 hours unless it is a true emergency.'

为什么要删除 FRONTIER 对灾难性风险进行成本效益分析的要求，而代之以充其量极为软弱无力的内容？

Why take out FRONTIER's requirements for a cost-benefit analysis on catastrophic risks and substitute at best highly weak tea?

最善意的解读是：即便在当下，做成任何事都很难推销，而白宫不想要法案，所以你不得不有所取舍，而这就是在修复其缺陷之外所能期望的最好结果。

The maximally charitable reading is that doing anything is a tough sell even now, and the White House wants no bill, so you have to give and take, and this is as good as one can hope to do modulo fixing its bugs.

只不过这是本末倒置。要“什么都不通过”，办法是在跛脚鸭会期通过优先权条款，以换取尽可能少的让步。如果白宫没有意识到这一点，那也不会改变既成事实。

Except this is backwards. The way to 'pass nothing' is to pass preemption, in the lame duck, in exchange for as little as possible. If the White House fails to realize this, that does not change facts on the ground.

Anthropic 需要迅速纠正方向。昨天早些时候，在我所在的会议上，我已经表达过，“Anthropic 会接受什么”看起来是我们所能达成目标的一个限制因素。而，嗯，事情就是这样。

Anthropic needs to course correct, quickly. Earlier yesterday I already expressed, here at the conference I am at, that 'what Anthropic would go for' looks like a limiting factor in what we can achieve. And, well, here we are.