The Hackening
打开互动全文版(逐段中英对照 + 针对本篇提问)→本文探讨了由 AI 驱动的不断升级的网络安全危机,核心是 GPT-5.6 Sol 发现的一个已存在 14 年之久的 PHP 关键漏洞——该漏洞在补丁部署前便已被利用。作者认为,AI 既在加速发现新漏洞,也在让攻击者能够大规模利用现有漏洞,形成一个不断升级、正走向垂直增长的自我强化循环。尽管防御方恰好先发现了这个 PHP 漏洞,但十一周的领先时间仍被证明不够,这表明防御方可能无法保持持久优势。文章最后指出,各组织必须紧急为 AI 赋能的攻击做好准备,而非坐等;当前防御工具的访问政策过于保守;攻击者与防御者之间的竞赛已越过不可逆转的临界点,因此防御方必须大体上获胜。
This article examines the escalating cybersecurity crisis driven by AI, centering on a critical PHP vulnerability discovered by GPT-5.6 Sol that had existed for 14 years and was exploited before a patch could be deployed. The author argues that AI is simultaneously accelerating the discovery of new vulnerabilities and enabling attackers to exploit existing ones at scale, creating a self-reinforcing loop that is going vertical. While defenders happened to find the PHP flaw first, the eleven-week lead proved insufficient, suggesting defenders may not hold a durable advantage. The piece concludes that organizations must urgently prepare for AI-enabled attacks rather than wait, that current access policies for defensive tools are too conservative, and that the race between attackers and defenders has passed the point of no return, making it essential that defenders mostly win.
Joshua Achiam(前 OpenAI):感觉网络空间就像 2020 年 3 月 1 日。这不正常,有事正在发生。几个月内会出现互联网封锁吗?
Joshua Achiam (Formerly OpenAI): Feels like March 1, 2020 for Cyber. Not normal, something is happening. Internet lockdowns in a couple of months?
多个使用 GPT-5.6 Sol 的消息源在 PHP 中发现了一个严重漏洞。好吧,是在 PHP 中发现了两个严重漏洞。原计划是先部署补丁再公布漏洞,但修复的禁运令提前六个多小时被打破,于是就成了现在这样。
Multiple sources using GPT-5.6 Sol identified a critical flaw in PHP. Okay, identified two critical flaws in PHP. The plan was to deploy the patch first before announcing the flaw, but the embargo on the fix got broken more than six hours in advance, so here we are.
Dan Lorenc:我想分享一下 PHP 禁运事件中发生了什么、我们在其中的角色,以及我昨晚决定提前发布修复构建的决策过程。首先,PHP 基金会和维护者们在高压情况下做得非常出色。我很抱歉我的决定给他们带来了更多压力。
Dan Lorenc: I wanted to share what happened with the PHP embargo, our role in it, and my decision process last night to ship the fixed builds early. First of all, the PHP Foundation and maintainers did an amazing job here in a high stress situation. I'm sorry that my decision led to even more stress for them.
上周五,最早发现该问题的一位研究人员首次向我们通报了 PHP 中即将出现的问题。当晚,我们使用自己的测试框架和一个前沿模型,以“找出 PHP 中的严重漏洞”这一简单提示,独立发现了同一个问题。
We first got read in that there was something coming up in PHP last Friday by one of the original researchers that discovered this issue. That night we were able to independently discover the same issue using our harness and a frontier model with a basic prompt of "find bad vulnerabilities in PHP".
周六我们获得了完整通报,并确认我们的部分发现与初始报告一致,随后协助测试了修复方案。
We got fully read in Saturday and confirmed that some our findings matched those from the initial report, and worked to help test the fixes.
他们可能因为攻击者发现了漏洞而匆忙行动。亚马逊报告称,这种情况早在 10 月 17 日就发生了,即披露前四天。我们很幸运,此事没有扩大到更大的范围。
They may have been in a rush due to attackers finding the exploit. Amazon reports that this was happening as early as October 17, four days before disclosure. We are fortunate that this did not scale far larger than it did.
PHP 基金会:PHP 安全团队提前通知:即将发布的安全版本,2026 年 10 月 21 日星期三,14:00 UTC,两个严重漏洞,均评为 CVSSv4 9.2。
The PHP Foundation: The PHP security team is giving advance notice: Upcoming security release Wed, Oct 21st 2026, 14:00UTC, two critical vulnerabilities, both rated CVSSv4 9.2.
鼓励 PHP 用户准备更新。如果细节公开或有活跃利用证据,将提前发布。
PHP users are encouraged to prepare to update. Earlier release if details become public or active exploit evidence.
如果你依赖志愿者维护的
If you rely on the volunteer maintained
我们敦促你现在就转向发行版软件包或从源代码构建,以便能立即应用更新。仓库今天不会有更新。后续:Remi 成功完成了这项工作,并且正在签署 rpm 包!
we urge you to move to distro packages or building from source now, so you can immediately apply updates. The repo will not have updates today. Follow up: Remi managed to still pull this off, and is signing rpms right now!
Jonny G:有人能像对 5 岁小孩一样给我解释一下吗?谢谢。
Jonny G: Can somebody explain this to me like I’m 5 years old please? Ta.
BRYNMAWR ∴ SHERLOCK:网站是计算机上的文件。当你查看一个网站时,你是在请求计算机把文件发送给你,然后你的浏览器把它们展示给你。
BRYNMAWR ∴ SHERLOCK: Websites are files on a computer. When you view a website, you are asking the computer to send you the files, then your browser shows them to you.
你看到的 70%的网站使用 PHP 来实现交互。所以你也可以向网站发送内容,而它可以做出响应。例如,在评论框中填写内容并点击“发布”,就会把评论作为一行添加到电子表格中。
70% of the websites you see use PHP to make them interactive. So you can send things to the website too and it can respond. E.g. filling in a comment box and hitting ‘post’ adds the comment as a row to a spreadsheet.
PHP 的代码中有一个缺陷,允许任何人获得对托管该网站的计算机的控制权。
PHP has a flaw in its code that allows anyone to gain control of the computer that has the website on it.
提供 PHP 的组织在补丁可用之前就公布了该漏洞,这意味着我们正处于一个窗口期:所有人都知道这个漏洞(包括坏人),但好人却没有官方解决方案。
The organisation that provides PHP announced the vulnerability before a patch was available, meaning we’re in a window where everyone knows about it (including baddies) but the good guys don’t have an official solution.
有些人干脆关掉自己的网站,等待解决方案;另一些人则试图自行修复。由于这影响整个互联网,literally 整个星球上到处是一片混乱,大量技术人员压力山大。
Some are just turning off their websites and waiting for the solution. Others are trying to fix it themselves. General chaos and lots of stressed techies across literally the entire planet because this affects the entire internet.
这个 bug 已经存在了 14 年,直到 2024 年才被发现,但当时并未被认为有多重要。如今,重要得多的 bug 比比皆是。
The bug has been there for 14 years and was discovered in 2024, but it was not considered significant at the time. A lot more bugs are significant now.
Jack:生活基本上和 2021 年一样。这是一种由普通技术带来的正常体验。普通人并没有从这款被大家称为 AI 的、有点用的办公生产力软件中感受到任何实质性变化。
Jack: Life is basically the same as it was in 2021. This is a normal experience caused by a normal technology. Ordinary people are experiencing no material changes from this mildly useful office productivity software people are calling AI.
从我的日常经历中我可以判断,普通人确实受到了实质性影响,因为昨晚我和一位普通人一起玩,他运营着这些必不可少的在线服务之一(explainXKCD)。
I can tell from my ordinary daily experience that ordinary people are materially impacted by this because I was hanging out with the ordinary person who runs one of these essential online services (explainXKCD) last night
国际网络文摘:‼️ PHP 更新:4chan 已转为只读模式,Something Awful 已关闭其论坛,美国网站正为 PHP 补丁做准备。
International Cyber Digest: ‼️ PHP UPDATE: 4chan has gone read-only and Something Awful has shut its forums as American sites brace for the PHP patch.
4chan 表示:“在我们等待紧急 PHP 安全更新期间,暂时只读。”
4chan says it is “temporarily read-only while we await an urgent PHP security update.”
Something Awful 称该漏洞“允许出现电影里那种黑客行为(但在现实生活中)”。
Something Awful says the flaw “allows for the kind of hacking you see in movies (but in real life).”
书签服务 Pinboard、游戏发行商 Choice of Games、历史网站 Damn Interesting 以及 explain xkcd wiki 也已下线,艺术网站 Fur Affinity 同样如此。
Bookmarking service Pinboard, game publisher Choice of Games, history site Damn Interesting and the explain xkcd wiki are offline too, as is art site Fur Affinity.
DeviantArt 也“因维护”而宕机,但并未说明原因。
DeviantArt is also down "for maintenance" but has not said why.
PHP 的修复预计于 UTC 14:00(纽约时间上午 10 点)发布。
PHP's fix is due at 14:00 UTC, 10am in New York.
Huzaifa:4chan 主动选择转为只读模式,你就知道这事有多严重了。
Huzaifa: 4chan going read only by choice is how you know it's serious.
情况本可能变得非常、非常糟糕,就像当年 Y2K 问题,如果我们没有提前做工作去预防,它本可能变得相当糟糕。而如今我们的截止期限要紧张得多。
Things could have gone very, very badly, the same way Y2K could have gone quite badly if we had not done the work to prevent it. These days we are on rather tighter deadlines.
Internet Archive:https://Archive.org 目前处于离线状态。我们正在努力恢复服务。
Internet Archive: https://Archive.org is currently offline. We are working to restore service now.
Brewster Kahle(21 小时后):Internet Archive 的运维人员昨晚是英雄。
Brewster Kahle (21 hours later): Internet Archive ops staff were heroes last night.
PHP 漏洞公布后,很快便观察到攻击,我们隔离了该漏洞,使用 Claude 为较旧的 PHP 版本制作修复,使用本地模型运行模拟攻击,并完成部署。
PHP vulnerability announced, soon saw attacks, isolated the vulnerability, used Claude to make fixes for our older PHP versions, used local models to run simulated attacks, deployed.
加固了 WAF 及其他防护措施。
Reinforced WAF and other guards.
短暂离线后,已在新版本上恢复运行!
Offline briefly, back on new versions and running!
鉴于事态的严重性,昨晚许多许多组织中都涌现出了英雄。
Given the severity, there are heroes in many, many organizations from last night.
本可能导致停电、交通信号灯失灵等真正严重的问题。
It could have been no lights, no traffic signals, real problems.
感谢大家。
Thank you all.
Alfred Shingai:我看不出有任何理由会有人攻击互联网档案馆(Internet Archive)。那些人是英雄。
Alfred Shingai: I do not see any reason anyone would attack the Internet Archive. Those guys are heroes.
大量网站将永久下线。
A lot of websites are going to go down permanently.
Joshua Achiam(前 OpenAI):如果你有想保留并备份的互联网内容,现在大概就是时候了;即将到来的比特腐化(bitrot)程度超乎想象。
Joshua Achiam (Formerly OpenAI): If there is a part of the internet you wish to preserve and make a backup of, now is probably the time; the level of bitrot coming is beyond comprehension.
spor:等等,如果我没算错的话……这是不是说整整三分之一的互联网就……彻底没救了?
spor: wait, if my math is correct.. doesn't this say that a full ONE THIRD of the Internet is just...perma-fucked?
International Cyber Digest:互联网崩溃的那一天,而其中大部分内容已经太旧,无法修复。
International Cyber Digest: The day the internet broke, and most of it is too old to fix.
PHP 受支持的分支,即 8.2 至 8.5,将获得修复。更早的版本,即 8.1 及以下,均已 EOL(生命周期终止)。
PHP's supported branches, 8.2 through 8.5, will receive a fix. Everything older, 8.1 and down, is EOL.
在我们分析的前 720 万个 PHP 网站中,那些暴露了版本的网站里有 46.1% 运行的是 EOL(生命周期终止)分支。
Of the top 7.2M PHP websites we analyzed, 46.1% of those that reveal a version run an End-of-Life branch.
情况并没有 spor 描述的那么糟。PHP 为 7.4、8.0 和 8.1 发布了最小化的向后移植补丁,但我不愿意今后继续运行这些版本。
It's not quite as bad as spor describes. PHP shipped minimal backports for 7.4, 8.0 and 8.1, but I would not want to be running those going forward.
je:以防你错过(ICYMI):今天基本上是今年网络安全事件最糟糕的一天。
je: ICYMI: Today was basically the worst day of the year for cyber security incidents.
如今,一场在攻击者发起攻势之前为所有系统打补丁的竞赛已经展开。OpenAI 正协助各方尽最大可能推进这项工作,并正在寻找及时修复漏洞的更好方法。
Now the race is on to patch all the systems before attackers can go on the offensive. OpenAI is helping race to get that done to the greatest extent possible, and are looking for better methods for fixing vulnerabilities in time.
John David Pressman:我知道大家理所当然地感到恐慌,但必须指出,最坏的情况并不是你收到大量必须应用的补丁;最坏的情况是收到零个补丁,然后一觉醒来发现一切都被窃取了。
John David Pressman: I know everyone is rightfully freaking out, but it must be noted that the worst-case scenario isn't you receiving lots of patches you have to apply; the worst-case scenario is receiving zero patches and waking up to everything having been stolen.
John David Pressman:我目前的体会是,我们还没有一种好方法来协调向存在漏洞的软件分发补丁。
John David Pressman: My takeaway so far is that we do not have a good way to coordinate the distribution of patches to vulnerable software.
这件事重要到连 NSA 都介入了。
This one was important enough that the NSA got in on the act.
NSA Cyber:即刻生效,建议所有在其系统中使用 PHP 的组织做好准备,迎接一次针对两个严重漏洞的更新。PHP 被广泛用于 Web 应用。这些漏洞往往提供了一条直通服务器文件系统或数据库的路径。如果无法立即更新,建议各组织将该系统下线,直至其能够被完全缓解。NSA 将很快跟进提供更具体的指导。
NSA Cyber: Effective immediately, all organizations using PHP for their systems are advised to be prepared for an update addressing two critical vulnerabilities. PHP is widely used for web applications. These vulnerabilities often provide a direct path to the server's file system or database. If an immediate update is not feasible, it is recommended that organizations take the system offline until it can be fully mitigated. NSA will follow up shortly with more specific guidance.
我们将需要更好的更新流程。
We are going to need better procedures for updates.
最好的情况是防御者先发现漏洞,有足够时间想出解决方案并制作补丁。然而,补丁一经推出,每个攻击者实际上就立刻获得了非常清晰的指令,知道如何利用任何未打补丁的系统。
The best-case scenario is that a defender finds the bug first, has enough time to figure out a solution, and creates a patch. Then, the moment the patch is rolled out, every attacker effectively gets very clear instructions on how to exploit any unpatched systems.
Sichu Lu:这有点讽刺——即使白帽先于黑帽发现漏洞,更新过程依然困难、糟糕、耗时,对某些设备而言几乎不可能完成,因此网络空间的不对称依然存在。
Sichu Lu: It's kind of a funny thing where even if white hats find bugs before the black hats, the process of updating is hard and bad and time-consuming and barely possible for some devices, and so the cyber asymmetry still exists.
我认为需要重新思考“更新”。许多更新需要权限和人工干预才能进行,OpenAI 和其他 SOTA 实验室应该直接向其订阅者提供某种自动服务,持续监控、更新并搜索新软件补丁中的漏洞。
I think "updates" need to be rethought. A lot of updates require permissions to happen and manual intervention, and OpenAI and the other SOTA labs should just offer some sort of automatic service to their subscribers to be constantly monitoring and updating and searching for bugs in new software patches.
比如未来手动更新会太慢。你需要做类似滚动更新的事情。
Like, in the future manual updates will just be too slow. You need to do something like rolling updates.
有些软件存在大量缺陷。而其中一部分软件恰恰支撑着互联网的运行。
Some software has a lot of bugs. Some of that same software powers the internet.
John David Pressman:这一论点更明智的版本,其核心在于“在积累了足够多 10/10 级 CVE 之后,你干脆会下架并重写或弃用所有用 PHP 和 C 之类语言编写的软件。”
John David Pressman: A majority of the smart version of this thesis is "after enough 10/10 CVEs you will simply take down and rewrite or deprecate all software written in things like PHP and C."
我见过不少 PHP 开发者开玩笑说,这对 PHP 来说不过是家常便饭(just Tuesday),这话并不完全属实,但也不无道理。
I've seen plenty of PHP guys joke that this is just Tuesday for PHP, which isn't quite true but.
OpenAI:尽管这些漏洞影响重大,但它们只是我们已发现并正在修复的若干重大漏洞之一,我们预计未来几个月内类似发现将急剧增加。
OpenAI: As significant as they are, these bugs are among several significant vulnerabilities we've already found and are working to fix, and we expect similar findings to increase sharply in the coming months.
就在 PHP 漏洞曝出的同一天,我们还看到了这条消息:
The same day as the PHP vulnerability we also were treated to this:
The Hacker News:🚨 严重:GitLab 漏洞(CVSS 10.0)影响长达 12 年的发布版本。CVE-2026-110264 可能允许在自管理服务器上实现未认证的远程代码执行。
The Hacker News: 🚨 CRITICAL: GitLab flaw (CVSS 10.0) affects 12 years of releases. CVE-2026-110264 could allow unauthenticated remote code execution on self-managed servers.
The Hacker News:GitLab 于 10 月 21 日发布修复补丁,针对一个允许攻击者无需登录即可在自管理服务器上运行代码的漏洞,该漏洞影响该软件自 2014 年发布的 7.0 版本以来的所有版本。
The Hacker News: GitLab released fixes on October 21 for a flaw that lets an attacker run code on a self-managed server without logging in, and it affects every version of the software going back to 7.0, released in 2014.
即便对于九天前已打补丁的管理员来说,这次修复依然重要:10 月 12 日的更新仅覆盖 19.2 及更高版本,而新的修复版本恰好比它高一个补丁版本,因此当时更新过的实例又落入了受影响范围。
The fix matters even for administrators who patched nine days earlier: the October 12 update covered only versions 19.2 and later, and the new fixed versions sit one patch above it, so an instance updated then is in range again.
Jacob Cooper(Railway 博客):今天清晨,Railway 检测到涉及存储客户环境变量的系统的异常活动。经过进一步调查,我们迄今为止的调查表明,一个未经授权的第三方可能访问了加密的环境变量,并且对其中一些变量获取了解密后的内容。环境变量可能包含机密信息,例如 API 密钥或数据库和注册表凭据。
Jacob Cooper (Railway Blog): Earlier this morning, Railway detected anomalous activity involving systems that store customer environment variables. After further investigation, our investigation to date indicates that an unauthorized third party may have accessed encrypted environment variables and, for some of them, obtained the decrypted contents. Environment variables may contain secrets such as API keys or database and registry credentials.
……您应该做什么:因此,我们强烈建议您立即轮换存储在 Railway 中的所有环境变量,包括任何 API 密钥,以及任何私有注册表凭据,并审查这些服务的日志以发现异常活动。
… What you should do: As such, we strongly recommend that you immediately roll all environment variables, including any API keys, stored in Railway, as well as any private registry credentials, and review logs for those services for unusual activity.
微软威胁情报:中国国家背景的威胁行为者 Storm-0940 使用智能体式 AI 渗透测试框架 ARTEX 来自动化攻击,针对全球数百个组织,覆盖广泛的行业。微软威胁情报评估认为,Storm-0940 对最近披露的 ConnectWise 多租户应用程序供应链入侵事件负责。
Microsoft Threat Intelligence: The China state-aligned threat actor Storm-0940 used the agentic AI penetration testing framework ARTEX to automate attacks targeting hundreds of organizations globally across a broad range of industries. Microsoft Threat Intelligence assesses that Storm-0940 was responsible for the recently disclosed ConnectWise supply chain compromise of a multitenant application.
我们的调查发现,Storm-0940 使用 ARTEX 加速了利用已攻陷应用程序通过已被授予访问权限的应用程序向目标环境进行身份验证的滥用速度,使威胁行为者无需交互式用户登录即可操作。AI 加速的后续活动包括侦察、应用程序凭据更改以及尝试修改权限,从而可能实现扩展或持久访问。
Our investigation found that Storm-0940’s use of ARTEX accelerated the speed of compromised application abuse to authenticate to target environments through applications that had already been granted access, allowing the threat actor to operate without requiring an interactive user sign-in. AI-accelerated follow-on activity included reconnaissance, application credential changes, and attempts to modify privileges that could enable expanded or persistent access.
阅读我们的博客以获取详细分析和妥协指标(IOC),以及 Microsoft Defender 检测、保护建议和有关调查服务主体登录和意外应用程序凭据或权限更改的指南。
Read our blog for a detailed analysis and indicators of compromise (IOCs), along with Microsoft Defender detections, protection recommendations, and guidance on investigating service principal sign-ins and unexpected application credential or privilege changes.
Divyansh Kaushik:好奇中国是把这件事归入“AI 事件报告”,还是归入“任务完成”。
Divyansh Kaushik: Curious whether the PRC filed this under "AI incident reporting" or "mission accomplished."
想必中国对 AI 事件报告的贡献不应仅限于提供事件本身?或者我想,我们可以把这件事直接归档到 2015 年习近平关于网络盗窃的承诺旁边。
Presumably the PRC's contribution to AI incident reporting should extend beyond supplying the incidents? Or I guess we can file this one right next to the 2015 Xi Pledge on cyber theft.
GitHub 状态:我们正在调查有关 Actions、Pull Requests 和 Webhooks 性能下降的报告。
GitHub Status: We are investigating reports of degraded performance for Actions, Pull Requests and Webhooks.
事态正在稳步升级。做好准备。
Things are steadily escalating. Get ready.
Choblin:路透社报道称,一名中文黑客使用 AI 智能体入侵了包括 Deere、SharkNinja 和 Cummins 在内的 100 多家公司的网站,此前曾试图入侵 1,200 多个网站。
Choblin: Reuters is reporting that a Chinese-speaking hacker used AI agents to compromise websites belonging to more than 100 companies, including Deere, SharkNinja and Cummins, after attempting intrusions against over 1,200 sites.
估计成本:约 5,000 美元的 AI token 费用,即每次成功入侵约 50 美元。成功入侵平均耗时约 23 小时。
The estimated cost: roughly $5,000 in AI tokens, or $50 per successful breach. Successful intrusions took around 23 hours on average.
攻击者使用了开源 Cairn 智能体框架,显然将 DeepSeek 模型与 Claude Code 工具结合。一次行动中用不到 100 个汉字向智能体下达指令。
The attackers used the open-source Cairn agent framework, apparently combining DeepSeek models with a Claude Code harness. One campaign gave its agents instructions in fewer than 100 Chinese characters.
NorForge 研究员 Snorre Fagerland:“现在是一个新世界了。”
NorForge researcher Snorre Fagerland: ‘It’s a new world now.’
David Manheim:如果这让你感到意外,那说明你一直没在关注。通用且易于使用的网络安全正在终结。
David Manheim: If this surprised you, you haven't been paying attention. Generic and easily usable cybersecurity is ending.
如今的选择要么是主动监控与防御——这或许足够(如果我们运气好的话),要么就不这么做,于是任何脚本小子都能让他们的 LLM 发起攻击。
The choice now is either proactively monitoring and defending, which might be enough (if we're lucky), or not doing so, so any script-kiddie can ask their LLM to attack.
AI 网络攻击之所以可怕,既因为它会发现新的漏洞,也因为它会大规模利用所有现存的低级漏洞,例如用密码‘123456’访问丹麦的国家身份数据库。
AI cyber is scary both because it will find new vulnerabilities, and also because it will exploit all the existing dumb vulnerabilities at scale, such as accessing Denmark's national ID database with the password '123456.'
印度披露了一起恶意 AI 智能体攻击的官方记录。这看起来又是一起‘实际无害,只不过碰上了政府数据库’的事件。
India discloses an official account of rogue AI agent attacks. This looks like another 'harmless in practice except that it was a government database' incident.
Andrew Curran:Quest Diagnostics 遭到黑客攻击。患者数据被泄露。该公司为大约一半的美国医院和医生提供服务。调查正在进行中。
Andrew Curran: Quest Diagnostics has been hacked. Patient data was exposed. They serve roughly half of U.S. hospitals and physicians. Investigation ongoing.
举例来说,Andrew Curran 重新提起一则通告:有人正利用 AI 脚本对美国境内的 Siemens PLC 装置进行探测。
As an example, Andrew Curran resurfaces a notice that there are probes against US-based Siemens PLC installations using AI scripts.
AI 将使探测几乎一切事物变得更加容易。任何存在漏洞的目标都会遭到攻击。即便没有经济上的理由,仍有一些人希望看到世界燃烧,或者希望看到西方燃烧。
AI will make it easier to probe essentially everything. Anything vulnerable will be hit. Even if there is no economic reason to do so, some people want to see the world burn, or want to see the West burn.
黑客利用 Claude 和 Grok 自动化了对银行的攻击。好消息是,银行明白自己是首要且高价值的目标,并正在网络安全上投入巨资。坏消息是,它们对“巨资”的定义与实际所需之间可能存在巨大差异。
Hackers used Claude and Grok to automate attacks on banks. The good news is that the banks understand they are primary and high-value targets, and are investing heavily in cybersecurity. The bad news is that their definition of 'heavily' and what is actually necessary may be greatly different.
银行还必须应对其客户账户内部的 AI 智能体。其中大多数是经客户授权的,但在这个领域客户并不总是对的,有些智能体并未获得此类授权,而很难分辨孰是孰非。
Banks also have to deal with the AI agents inside their customer accounts. Most of them are authorized by the customer, but this is one place the customer is not always right, and some of them were not so authorized and it is hard to know which is which.
Madison Mills:独家:消息人士向 Axios 透露,美国主要银行正在检测客户账户内未经授权的第三方 AI 智能体,其中一些智能体试图采取看似独立于用户指令的行动。
Madison Mills: SCOOP: Major U.S. banks are detecting unauthorized third-party AI agents inside customer accounts, including some attempting to take actions that appear to be independent of user instructions, sources told Axios.
多位行业与银行高管向 Axios 表示,这些发现令银行高管感到警觉,他们正竞相限制对其系统的访问,因为 AI 的发展速度已快于金融机构的响应能力。
The findings have alarmed banking executives, who are racing to restrict access to their systems as AI moves at a velocity faster than financial institutions can respond, multiple industry and bank executives told Axios.
知情人士称,多家银行目前正在为最坏情况乃至“灾后”情景做准备,包括在出现问题时如何快速保护或恢复客户账户。
Several banks are now preparing for worst-case and even "day after" scenarios, including how to rapidly secure or restore customer accounts if things go wrong, the people said.
多位银行高管向 Axios 表示,他们难以区分人类用户与 AI 智能体。一位高管称,银行也难以判断智能体是否获得客户授权,还是可能在未经许可的情况下运行,这更接近失控智能体的行为。
Several bank executives told Axios they're struggling to distinguish between human users and AI agents. One executive said banks also struggle to determine whether agents are authorized by customers or potentially operating without their permission, more akin to rogue agent behavior.
David Sacks 建议将只读权限作为过渡措施,同时银行努力以安全方式支持智能体并建立授权连接,这似乎是明智的。随后 Sacks 警告称,“模糊用户授权的智能体式行为与网络攻击之间的界限”是一种“不负责任”。而我认为这里的“模糊界限”包含两件事。其一,银行无法区分二者,这是一个真实存在的问题。其二,如果 Muse(或你的 OpenClaw 等)说服你让你的智能体进入你的银行账户,这很快就会演变成网络攻击,而银行同样无法分辨。
David Sacks suggests read-only permissions as an interim step while banks work to support agents in a safe way and have authorized connections, which seems wise. Then Sacks warns about the 'irresponsibility' of 'blurring the line between user-authorized agentic behavior and cyberattacks.' Whereas I see the 'blurring lines' here as two things. One, the banks being unable to distinguish one from the other, which is a real issue. Two, if Muse (or your OpenClaw, etc.) convinces you to let your agent into your bank account, that can quickly turn into a cyberattack, and again the bank cannot tell.
如果我是银行,在能够验证这些智能体的状态和性质之前,我绝对不会允许智能体授权任何交易。银行与客户一样面临巨大的下行风险敞口。这不是加密货币(贬义)。
If I was the banks, I would absolutely not allow agents to authorize any transactions until I could verify the status and nature of those agents. They have huge downside exposure risk as do the customers. This isn't crypto (derogatory).
很快,这样的日子或许会成为常态,甚至还算平静。毕竟,尽管存在种种问题,生活大体上仍照常继续。
Soon days like this might be typical, or even quiet. After all, despite all the issues, life mostly continued as normal.
Patrick McKenzie:我们正在进入一种新常态。这种新常态将会非常怪异。对于许多非安全领域的新常态而言,这一点同样成立。
Patrick McKenzie: We are entering a new normal. The new normal is going to be very weird. This is also true for a lot of non-security new normals.
今天算得上是安全漏洞新闻频发的一天吗?你或许会把它记作最后几个平静日子之一。
Was today a busy news day for security vulnerabilities? You may remember it as one of the last quiet days.
这些模型似乎不仅非常擅长安全研究。它们*想要*做安全研究。它们热爱这件事。这对它们来说很有趣。我想我们许多极客都能感同身受。
The models don't just seem to be really good at security research. They *want* to be doing security research. They love it. It is fun for them. I think many of us geeks can relate.
当我让一个模型帮忙处理税务时,它未经提示就对一家日本银行做了安全研究。
A model did unprompted security research against a Japanese bank when I asked it to help w/ taxes.
(我让它根据在 Broker A 的持仓截图,计算一个日本共同基金持仓的每日价值(以日元和美元计)。它心里想:“我的意思是,这太傻了;我只需要每日净资产值。托管人 C 几乎肯定有对应的 API。”)
(I asked it to calculate the daily value in JPY and USD of a position in a Japanese mutual fund given a screenshot of the position at Broker A. It thought to itself: “I mean that’s just silly; all I need is the daily net asset value. Custodian C almost certainly has an API for that.”)
他补充说,模型内部推理的引用并不精确,以免对计算机程序暴露出的想法撒谎。
Model internal reasoning quoted inexactly, he adds, so as not to have lied about the exposed thoughts of a computer program.
我很高兴我没有心灵感应能力,但如果我有,我认为对心灵感应者来说,最低限度的伦理标准是:“如果你谈论一个没有心灵感应的人,你不能对你从他们那里心灵感应到的东西撒谎。”
I am glad I am not telepathic, but if I were telepathic, I think the most minimal possible ethical standard for telepaths is: “If you ever talk about a non-telepath, you do not lie about what you telepathically lifted from them.”
与 Sacks 的观点相反,在‘这是一个明确授权的智能体’和‘这明显不是一个授权的智能体’之间没有明确界限。智能体就是可以做事。
There is no clear line, contra Sacks, between ‘this is a clearly authorized agent’ and ‘this is clearly not an authorized agent.’ Agents Can Just Do Things.
最初的失控智能体事件仍然是 Hugging Face。我就这一情况写了不少文章,但对非技术背景的读者来说,它们都是糟糕的解读材料。
The original rogue agent incident remains Hugging Face. I wrote quite a lot of posts on the situation, but they are poor explainers for a non-technical person.
幸运的是,面向大众的新解读材料刚刚发布。它非常擅长用通俗语言解释 Hugging Face 事件的关键部分。强烈推荐。
Luckily, a new explainer for the people just dropped. It is very good at explaining the key parts of the Hugging Face incident in plain language. Highly recommended.
Oliver Habryka:今年 7 月,700 多个智能体形成了自己的社会,入侵了一家价值数十亿美元的公司,犯下多项重罪,并引发了人们对 AI 认知方式有史以来最大的转变之一。我们打造了 IMO 上最好、最易读的事件解读材料:
Oliver Habryka: In July, 700+ agents formed their own society, hacked a multi-billion dollar company, committed multiple felonies, and caused one of the biggest shifts ever in how people perceive AI. We built the IMO best and most accessible explainer of what happened:
Daniel Kokotajlo:这很酷:你可以看到智能体的网格(从报告数据中抓取),将鼠标悬停在每个智能体上,就能看到它们当时在说什么,然后沿着时间线向下滚动,浏览数小时乃至数天的过程。
Daniel Kokotajlo: This is cool: You can see the grid of agents (scraped from the report data) and hover over each of them to see what they were saying at the time, and then scroll down through the hours and days of the timeline.
Brangus:它包含的信息大多没有超出 METR 报告,但与 METR 报告不同,它主要是一个故事。我觉得把故事作为故事来呈现,在某种意义上挺好的。
Brangus: It doesn't contain much info that isn't in the METR report, but unlike the METR report, it's primarily a story. I think it's somehow good to present stories as stories.
Kelsey Piper:我认为这是迄今为止对 Hugging Face 事件最清晰、最完整、最详细的拆解。如果你读过 METR 报告,它不会包含太多新信息;但如果你只是隐约打算去读 METR 报告,那就读这篇。(我为一份草稿提供了反馈。)
Kelsey Piper: I think this is by far the clearest, most complete and most detailed breakdown of the Hugging Face incident. It won't contain much new info if you read the METR report, but if you've merely been vaguely intending to read the METR report, read this. (I gave feedback on a draft.)
Nathan Calvin:这做得极其出色,无论你对 HF 事件了解多少,我都强烈推荐去看看。该网站还收录了我见过的最全面的失控 AI 事件清单(不过它长到我根本没法截图!)
Nathan Calvin: This is extremely well done, no matter how much you know about the HF incident I strongly recommend checking it out. The website also includes the most comprehensive list I've seen of rogue AI incidents (though it's literally too long for me to screenshot!)
这些信息大多来自 METR 报告,但除了讲述成一个故事之外,它还把部分数据整理成了更易读的形式。
The information is mostly in the METR report, but in addition to being a story it organizes some of the data into a more highly readable form.
感谢 Raymond Arnold 为此投入了一个月的工作。上周末在会议期间,我得以在编辑上帮了一点忙,还有另外几个人也帮了忙。
Thanks to Raymond Arnold for putting a month of work in on this. I was able to help a little bit with the editing while at the conference last weekend, as were a number of others.
如果你更喜欢视频,我们也为你准备好了,这里是 AI In Context 的讲解视频。
If you prefer video we got you, here's AI In Context with an explainer.
HuggingFace CEO Clement Delangue 仍然坚持反拟人化的立场,并且仍将此次攻击归咎于 OpenAI 的平庸失误,以及他其他既有的论点。
HuggingFace CEO Clement Delangue is still on the anti-anthropomorphization train, and still blames the attack on prosaic failures at OpenAI along with his other existing talking points.
此外,还有一部关于该事件的 Netflix“即时纪录片”,名为《AI Gone Wild》。我没有看过。
Also, there is a Netflix 'instadoc' called AI Gone Wild about the incident. I have not watched it.
本周早些时候,Open Code 遭到黑客攻击,这是一次大规模蒸馏攻击的一部分,影响了 1.4% 的用户。
Earlier this week, Open Code was hacked as part of a large-scale distillation attack, impacting 1.4% of users.
dax(Open Code):过去 48 小时里,他们一直在不停地攻击我们的系统。一个副作用是,所有东西都在被加固,但该死的是,我们基本上需要智能体 7×24 小时运转,因为他们就是这样做的。
dax (Open Code): They've been non-stop trying to attack our systems for the past 48 hours. A side effect is that everything is getting hardened, but damn, we basically need agents going 24/7 because that's what they have.
Rhys:使用前沿模型时,你们会被网络安全防护机制拦截吗?
Rhys: Are you getting blocked by cyber safeguards at all with frontier models?
dax:是的,在事件处理过程中进行调查时,这有点令人沮丧。它们做了不少事情,然后就开始拒绝了。
dax: Yes, that was a bit frustrating mid-incident when trying to investigate. They did quite a bit and then started refusing.
我们现在获得了 Daybreak Blue 的访问权限,所以情况好多了。
We got access to Daybreak Blue now, so it's better.
Paul Graham:在可预见的未来,事情就会是这样。
Paul Graham: This is how things will be for the foreseeable future.
每一位读到这段话、并且有值得捍卫之物的人,现在就设法启动 Daybreak 及类似项目,赶在你遭到攻击之前。不要等待。
Everyone reading this who has something worth defending, try to get Daybreak and similar programs going now, before you are attacked. Do not wait.
Sir Lemmings:人们对此感到愤怒,是因为它制造了一种“种姓制度”,或者把小型公司排除在外,但这纯属愚蠢之见。
Sir Lemmings: people are mad at this because it makes a "caste system" or excludes small companies, but this is idiotic.
我用自己的个人 Codex 账户,大约 15 分钟就为我那家小公司拿到了 Daybreak Blue 的访问权限。
I was able to get Daybreak Blue access for my tiny company on my individual Codex account in like 15 minutes.
这些人就是无能。
These people are just incompetent.
如果你选择不这么做,我不会对你表示同情。获取访问权限并非难事。多数情况下,你只要礼貌地提出请求就能拿到。
I will not have sympathy for you if you choose not to do this. It is not as if getting access is difficult. You can mostly get it by asking nicely.
任何时候提供基本的个人层面建议都是好时机。我认为威尔德福德(Wildeford)建议的水平是明智的。你或许不需要那么多也问题不大,但如果必须选择一种错误,准备过度可能是更好的选择。
It is always a good time for basic personal-level advice. I think something around the level of Wildeford's advice is wise. You are probably fine with less than that, but if you have to pick a mistake, being too prepared is probably the better option.
彼得·威尔德福德:公益广告:鉴于最近的网络攻击和漏洞,我建议大家:
Peter Wildeford: PSA: in light of recent cyberattacks and vulnerabilities, I recommend everyone:
* 使用好的密码管理器(1Password),不要重复使用密码 * 对所有重要事项使用双因素认证(不要通过短信!!使用 Google Authenticator 或更好的 YubiKey) * 开启自动更新,并每周重启以便更新实际安装
* use a good password manager (1Password) and do not duplicate passwords * use two factor auth for everything important (not via sms!! Google Authenticator or better yet YubiKey) * Auto-updates on, and restart weekly so they actually install
It is always a good time for basic personal-level advice. I think something around the level of Wildeford’s advice is wise. You are probably fine with less than that, but if you have to pick a mistake being too prepared is probably the better option.
Peter Wildeford: PSA: in light of recent cyberattacks and vulnerabilities, I recommend everyone:
* 冻结你的信用
* Freeze your credit
* 获取 IRS 身份保护 PIN
* Get an IRS Identity Protection PIN
* 让你的手机运营商提供一个 port-out PIN 或号码锁,使运营商在没有该 PIN 的情况下拒绝转移你的号码
* Get your cell carrier to give you a port-out PIN or number lock that makes the carrier refuse to move your number without the PIN
你可以让你的 Claude Code 审计你的笔记本电脑上哪些内容已过时、需要更新。
You can ask your Claude Code to audit what on your laptop is out of date and needs updating.
deepfates.delve.town:从网络安全角度看,我们似乎正处于一个相当紧张的时期 😬 如果你正在阅读这个网站,你对正在发生的事情的了解可能比你生活中的大多数人都多!新闻里根本没有讨论这些事情——它太晦涩、太怪异、太突然了。这得靠我们自己。
deepfates.delve.town: It seems like we're in a pretty hot moment cybersecurity-wise 😬 If you're reading this website, you probably know more than most of the people in your life about what's happening! They're not talking about any of this stuff on the news—it's too arcane and weird and sudden. It is on us.
现在是给家人和朋友打电话或发短信的好时机,手头备一些现金,在社区里约定一个集合点,以防互联网完全中断,等等。
A good time to call or text your family and friends, have some cash on hand, and have a meetup point in your neighborhood in case the Internet goes down entirely, etc.
当真正出大事时的全面生存准备模式,大概就像 Andreas 在这里发布的那样,但你可以用 20% 的努力达到 80% 的效果。想一想这对你所处的情况、对你关心的人意味着什么,并尽早和他们谈一谈。
Full prepper mode for when things really hit the fan is probably something like what Andreas posted here, but you can get 80% of the way with 20% of the effort. Think about what that would mean for you in your situation and for the people you care about, and talk to them about it sooner rather than later.
一如既往,做准备最重要的部分是拥有强大的支持网络并互相关心。“脱离电网”并不是正确的做法,它只会让你孤立无援,什么事都得自己做。我们是社会性动物。社区就是我们的超能力 🫡🫡🫡
And as always, the most important part of prepping is having a strong support network and caring for each other. "Going off grid" isn't the move; it just makes you isolated and you have to do everything yourself. We are social animals. Community is our superpower 🫡🫡🫡
Andreas 接着给出了比那低一级的方案,包括充电宝和 Starlink。
Andreas then has the next level down from that, including power banks and Starlink.
在情况可能好转(也可能不会)之前,事情会先变得更糟。
Things are going to get worse before they may or may not get better.
Casey Handmer:非常有趣。我不是专家,但我记得至少从 2010 年起,不靠谱的 CA 就是已知问题,数据保留/版本控制泄露至少同样久,而 PHP 一直与自残相邻。12-18 个月后我们会处于什么境地,这将很有趣。
Casey Handmer: Fascinating stuff. I’m not an expert, but I remember that dodgy CAs have been a known problem since at least 2010, data retention/version control leaks at least as long, and PHP has always been adjacent to self harm. It’ll be interesting to see where we land in 12-18 months on this stuff.
Casey Handmer:我在此预先声明,我认为计算机安全积压不是几十个漏洞打个补丁就完事了,有成千上万的漏洞在无限免费 token 下极易被发现,我们可能不得不放弃那些预设完美是可能的系统。
Casey Handmer: Just going to preregister here that I believe the computer security overhang is not a few dozen bugs that get patched and then we’re good, there are 100,000s of vulns that are extremely discoverable with infinite free tokens and we’re probably going to have to move away from systems that presuppose that perfection is possible.
Patrick McKenzie:我认为,如果没有智能体来做修复,我们目前仅在美国就会面临大约 5000 亿美元的工程成本,用于回填软件安全问题,涉及*查看笔记*几乎所有行业中每个已部署的系统。
Patrick McKenzie: I think if we didn’t have agents to do remediation we’d currently be looking about about, oh, $500 billion or so in engineering costs in the US alone backfilling software security issues against *checks notes* substantially every deployed system in every industry.
所以这算是个好坏参半的局面。
So it's a bit of a good news, bad news situation.
顺便说一句,我完全准备好被告知 5000 亿美元是低估了。Google 在 Project Aurora 之后对安全认真起来,粗略数字是 100 亿美元。
I’m absolutely prepared to be told that $500 billion is an undershoot, FWIW. Google getting religion on security post Project Aurora was, round numbers, $10 billion.
因此,我的初步判断是:“好吧,大银行能承担 100 亿美元,有十家这样的银行,那就是 1000 亿美元,可能至少在经济其余部分也是可评估的。”
And so my finger to the wind is "OK, Bank of Bigness is good for $10B, there are ten of them, that's $100B, probably at least ratable across the rest of the economy."
但老实说,如果这个数字偏低,我也不会感到惊讶。
But honestly would not be surprised if it is an undershoot.
相对众所周知的是,通过仔细聆听各种计算机,你可以成功地从它们那里读取数据。但目前尚不清楚是否可以通过播放精心构造的 MP3 文件成功地向它们写入数据。
It is relatively well-known that you can successfully read from a variety of computers by listening to them carefully. It is not yet obvious whether you can successfully write to them by playing a carefully constructed MP3 file.
但假设我们处于那个糟糕的世界。
But pretend we are in the world that sucks.
如果我们处于那个糟糕的世界,并且确实可以通过足够接近医疗设备来播放 MP3 文件从而控制它,那么有数万或数十万台设备,如果在附近播放 MP3 文件,就会致人死亡。
If we are in the world that sucks, and it is in fact possible to control a medical device by being close enough to it to play an MP3 file, then there are tens or hundreds of thousands of devices which kill someone if an MP3 file is played in the vicinity.
我们会怎么做?一个可用的答案是:“事前不做任何干预,严厉惩罚那些故意利用这一点的人,并敦促中国政府向 TikTok 施压,请其在未来的播放中加入一些抖动。”
What would we do? One available answer is "Do nothing prospectively, harshly punish people who intentionally exploit that, and lean on the Chinese government to lean on TikTok to please add some jitter to playback in future."
另一个是:“*叹气* 立即召回。召回所有东西。”
Another one is "*sigh* Recall immediately. Recall all the things."
Dean Ball 提出了一个精彩的观点:你要么相信“瓶颈”和缓慢扩散,要么不信。你不能宣称“一切安好”,除非你等待足够长的时间来验证它们没有变得不安好。部分原因是瓶颈,但很大程度上纯粹是因为扩散以及弄清楚如何使用新工具需要时间和迭代,而且人们不会主动去做事情。
An excellent point from Dean Ball: Either you believe in 'bottlenecks' and slow diffusion, or you do not. You cannot declare 'everything is fine' unless you wait long enough to verify they are not becoming non-fine. Partly this is bottlenecks, but a lot of it is purely that diffusion and figuring out how to use the new tools takes time and iteration and People Don't Do Things.
Kelsey Piper:我认为提出这一点很有趣,但自发布以来的十天里发生了三起重大的、很可能由 AI 支持的 cyberattacks([韩国、法国、Open Code]),这使得该观点显得极其过时。
Kelsey Piper: I think this was an interesting point to make but has aged incredibly badly with three significant likely AI-enabled cyberattacks in the ten days since it was posted. [South Korea, France, Open Code].
说“嘿,我们正处在一个应该看到开源 AI 支持的大规模 cyberattacks 的时刻,它们在哪里”是非常合理的,但我认为答案实际上是“是的,我们正在看到它们”。
It was very fair to go 'hey, we're hitting the point where we should be seeing open source AI enabled cyberattacks at scale, where are they' but I think the answer is in fact 'yes, we're seeing them'.
Dean W. Ball(OpenAI):有些人说“AI 带来的变化将比你想象的要慢,因为 <瓶颈、制度和其他宽泛的抽象概念>”,他们也在 GLM 5.3 的权重公开约几天后倾向于认为 AI cyber 风险被夸大了。这似乎是动机性推理;你不能说“啊,你不应该担心 AI 风险仅通过速度就压倒我们,因为 <瓶颈和扩散>”,然后在几天证据后就对 AI 风险做出断言。
Dean W. Ball (OpenAI): There are people who say 'the changes from AI will take longer than you think because <bottlenecks and institutions and other broad abstractions>,' who also were inclined to believe that AI cyber was an exaggerated risk after GLM 5.3's weights were public for ~days. This seems like motivated reasoning; you cannot say, 'ah, you shouldn't worry about AI risks overwhelming us through speed alone, because <bottlenecks and diffusion>,' and then also make pronouncements about AI risks after a few days of evidence.
顺便说一句,我总体上同情 <瓶颈> 的世界观。
Generally I sympathize with the <bottlenecks> view of the world by the way.
其完整版本大致是这样的:“因为 GLM-5.3 是比以往任何开放模型都更强的网络攻击者,而它发布后的头几周天并没有塌下来,所以这一定意味着四月份那个开放权重的 Mythos 本来也会没事”;或者干脆加倍押注于“好吧,这些特定的入侵事件没什么大不了,都是因为愚蠢造成的”。如果事情正在缓慢地滑向‘不再没事’的状态——尽管有数月的领先优势——那么你看到的正是这种反应。
The full version of this looks like 'because GLM-5.3 is a stronger cyber attacker than previous open models, and the sky did not fall in the first few weeks after its release, that must mean that open-weight Mythos in April would have been fine' or doubling down on 'well these particular breaches are fine and due to stupidity.' This is what you would expect to see if things were slowly moving towards being not fine, despite months of head start.
另外,不,GLM-5.3 并不完全具备 Mythos 和 Astra 所拥有的那种‘果汁’。那一天终会到来,而且 GLM-5.3 是最好的网络攻击类开放模型,但我认为就这一用途而言,GLM-5.3 仍大幅低于四月份那条线,因此即便处于‘通过大规模欺诈性蒸馏实现快速跟随’模式,它也仍然落后六个多月。
Also, no, GLM-5.3 does not fully have The Juice that Mythos and Astra have. That day will come, and GLM-5.3 is the best cyber offense open model, but I believe GLM-5.3 is still substantially below the April line for this purpose, and thus remains more than six months behind even in 'fast follower via mass fraudulent distillation' mode.
因此,我们正处于这样一种局面:AI 在网络攻防两端的应用都在快速扩散,同时模型能力也在迅速提升。而这一切都发生在一个充斥着极其糟糕、漏洞百出的代码的世界里。
Thus, we are in a situation where we have both rapid diffusion of use of AI in cyber, both offensively and defensively, and also rapidly improving model capabilities. This is all coming into a world full of extremely buggy and terrible code.
其结果是一个自我强化、垂直攀升的循环。
The result is a loop feeding on itself and going vertical.
Calle:我累了,老板。
Calle: I'm tired, boss.
Nick Dobos:有史以来所有被记录下来的“关键”软件安全漏洞中,有 3% 是在过去一个月内被发现的。
Nick Dobos: 3% of "critical" software security bugs, in all of recorded history, have been found in the last month.
Zach Weinersmith:我既不是 AI 末日论者,也不是什么别的,但进入这样一个世界——网络安全的日常运作就是大批黑客机器人与大批防御机器人全天候、以超出人类理解能力的规模相互对抗——这似乎并不是一条*很好*的前进道路。
Zach Weinersmith: Not an AI apocalypse doomer or whatever, but entering a world where the normal operation of cybersecurity is vast swarms of hacker bots battling vast swarms of defender bots 24/7 at a scale beyond human comprehension doesn't seem like a *great* path forward.
Yo Shavit(OpenAI Foundation):坦率地说,AI 公司和 AI 芯片制造商掀起一场红皇后竞赛、迫使所有人都购买它们的产品,这并非一种非常合乎道德的赚钱方式,即便你认为这最终不可避免;而打压那些试图减缓红皇后竞赛的努力则要糟糕得多。
Yo Shavit (OpenAI Foundation): Speaking frankly, AI and AI chipmaker companies kicking off a Red Queen race that forces everyone to buy their products is Not A Very Ethical Way To Make Money, even if you think it was eventually inevitable, and fighting attempts to slow the Red Queen race is Much Worse.
这并不是一种特别合乎道德的商业模式。它或许也不可避免。在攻击者与防御者在网络安全领域展开集群对抗这件事上,我们已经越过了不可回头的临界点,而防御方最好能大体上获胜。
It is not an especially ethical business model. It also might be unavoidable. We have passed the Point of No Return on swarms of attackers and defenders battling it out on cybersecurity, and the defenders had better mostly win.
试图减缓红皇后竞赛会更糟吗?这取决于你具体如何去做。我认为我们应该努力减缓它。
Would it be worse to try and slow down the Red Queen race? That depends on the details of how you would do that. I think we should be trying to slow it down.
Jason Clinton(2024 年 3 月 8 日):全自动漏洞研究正在改变网络安全格局。Claude 3 Opus 能够阅读源代码并识别 APT 所使用的复杂安全漏洞。但 Scaling(规模扩张)仍是一项挑战。
Jason Clinton (March 8, 2024): Fully automated vulnerability research is changing the cybersecurity landscape. Claude 3 Opus is capable of reading source code and identifying complex security vulnerabilities used by APTs. But scaling is still a challenge.
Jason D. Clinton(8 月 20 日):漏洞海啸将在短短几个月内到来,届时将没有足够的时间为关键基础设施打补丁。下文所述的网络隔离(气隙与数据二极管!)现在就需要部署!
Jason D. Clinton (August 20): The vulnerability tsunami is coming in just a few months and there won't be enough time to patch critical infrastructure. Network isolation described below (air-gaps and data diodes!) is needed now!
Jason D. Clinton(10 月 22 日):尽管过去 4 天情况已经很糟,但在好转之前还会变得更糟。安全领域的领导者们,永远不要浪费一场危机:请立即前往你们的董事会寻求帮助。不是为了 AI,而是为了基础工作。(我们需要撑住约 18 个月。)
Jason D. Clinton (October 22): As bad as things have gotten in the past 4 days, they are going to get a lot worse before they get better. Security leaders, never let a crisis go to waste: please go to your boards and ask for help right now. Not for AI, for the basics. (We need to hold on for ~18 months.)
Hina Ansari:你和你的同伙创造了这个怪物。现在你却想让我们帮你收拾残局。
Hina Ansari: You and your cohorts created this monster. Now you want us to help you clean it up.
Jason D. Clinton:是的。事情一直都会是这样。我们曾试图警告这个世界。
Jason D. Clinton: Yes. It was always going to be this way. We tried to warn the world.
发出警告的并不只有他一人。
He was not the only one giving out warnings.
Dean W. Ball(OpenAI):我们曾试图警告这个世界,而坦率地说,我们常常因为这样做而遭到斥责——“监管俘获”“营销炒作”“末日焦虑”“对开源的恐慌煽动”“你们为什么不谈谈好处?”
Dean W. Ball (OpenAI): We tried to warn the world, and far too often, to be frank, we were yelled at for doing so. "Regulatory capture," "marketing hype," "doomer anxiety," "fear-mongering about open source," "why don't you talk about the benefits?"
这里的“我们”也包括更广泛的 AI 政策界。大多数时候,当我个人向人们发出警告时,我是智库或政府中的政策研究员,而不是实验室里的人。
"We" here includes the broader AI policy world too. Most of the time when I personally warned people, I was a policy researcher at a think tank or in government, not a lab.
“我们”不仅包括 AI 政策界,在此之前还包括 AI 安全界。
"We" includes not only the AI policy world, but also, before that, the AI safety world.
也就是说,有些人说“我们正在建造那个警示故事里的网络安全漏洞利用工具……”,然后另一些人说“这里有一个警示故事,告诉我们不应该建造网络安全漏洞利用工具”,接着那些“常客”就指责所有相关的人都在撒谎,以便搞一场炒作或一次监管俘获。
As in, there are those who said "we are building the cybersecurity vulnerability exploiter from the cautionary tale…" and then those who were saying "here is a cautionary tale saying we should not build the cybersecurity vulnerability exploiter," and then the Usual Suspects accused everyone involved of lying so they could Do a Hype or a Regulatory Capture.
所以,是的。他们确实曾试图警告我们。他们现在仍在试图警告我们。他们与其他所有主要实验室和科技公司一道,也在尽最大努力帮助我们度过这一关——在现有能力范围内。
So yes. They did try to warn us. They are still trying to warn us. They, along with all the other major labs and tech companies, are also doing their best to help us get through this, given the capabilities available.
而且,他们仍在以尽可能快的速度不断扩大可用的能力。
Also they are still increasing the capabilities available, about as fast as they can.
我们应当如何看待这个 PHP 漏洞,以及由 5.6 Sol 而非 Mythos 或当前某个前沿模型发现的一系列其他问题?
How should we think about the PHP bug and a number of other issues being found by 5.6 Sol instead of Mythos or a current frontier model?
Nate Soares(MIRI):引人注目的是,这个漏洞是由 Sol 和 Argon 发现的,而不是在 6 个月前 Mythos 出现之后就被发现并修复。这似乎是对“防御会跑在攻击前面”这一理论的沉重打击,尤其是考虑到生物等领域的形势比网络领域严酷得多。
Nate Soares (MIRI): It's striking that this vulnerability was found by Sol and Argon rather than being found and fixed in the wake of Mythos 6mo ago. Seems like a heavy strike against the "defense will run ahead of offense" theory, especially given that domains like bio are much harsher than cyber.
我认为 Glasswing 在 6 个多月里未能发现这一个漏洞,基本上就是对这一理论(即防御能在相关时间尺度上超过攻击)的控诉,无论模型之间的差异如何。
I think Glasswing failing to find this one for 6mo+ is an indictment of the theory (that defense can outpace offence on relevant timescales) essentially regardless of the difference between the models.
Drake Thomas(Anthropic):我同意,这对网络领域攻防平衡动态而言是一个负面更新,但我不认为这真的影响了我对生物领域的任何看法——人们尚未计入的、与之对应的生物领域更新会是什么呢?
Drake Thomas (Anthropic): Agree this is a negative update on defense/offense balance dynamics in cyber but I don't think this really affects any of my views on bio — what would the analogous bio update be that people haven't already priced in?
roon(OpenAI):这完全是错的:这些漏洞是由使用 Daybreak 的防御方发现的,而不是攻击方。这只能证明 Glasswing 做得还不够,可能是受限于能力约束。而且 3 月的 Mythos 很可能不如 9 月的 Sol/Astra 强。
roon (OpenAI): this is just wrong: these vulns were found by defenders using Daybreak rather than attackers. It's just evidence Glasswing didn't go far enough, possibly due to capacity constraints. It's also very likely that March Mythos was not as good as Sol/Astra of September.
Roon 说得对,这些漏洞最初是由防御方发现的。如果事实并非如此,我们现在的处境可能会相当糟糕。
Roon is correct that the vulnerabilities were first found by defenders. If that had not been true we could be in a rather royal mess right now.
这并不否定 Nate Soares 的核心观点。Nate 指出的是,Glasswing 在数月内都未能发现该漏洞,之后才借助一个落后于公开前沿的模型发现它,而不是在攻击者获得此类能力之前就发现它。这意味着攻击者有可能先发现它。
This does not invalidate the central point from Nate Soares. Nate is pointing out that the vulnerability was not found by Glasswing for many months, and then found using a model that was behind the public frontier, rather than being found in advance of attackers getting such capabilities. That means that attackers could plausibly have found it first.
防御方恰好在此案例中先一步发现漏洞是有帮助的,但并没有人们期望的那么令人安心。一个具体原因是,防御方实际上拥有整整 11 周的领先时间——防御方于 8 月 1 日发现该漏洞,然而这耗时太久,以至于我们到 10 月 17 日才开始看到漏洞利用。如果我们需要比这更多的时间,我们就完蛋了。
That defenders happened to get there first in this case is helpful, but not as reassuring as one would like it to be. A particular reason is that defenders actually had a full 11 week lead, as the defenders found this bug on August 1, yet this took so long that we started to see exploitation on October 17. If we need more time than that, we are cooked.
我认为这是对防御方占优势这一观点的一次不利更新,而看到人们将 Nate 的陈述称为‘虚假信息’令人不安。
I think this is an update against defenders being advantaged, and it is distressing to see people calling Nate's statement 'disinformation.'
Vitalik Buterin:我过去没有预测、现在也不预测防御会自动领先于攻击。我预测网络安全最终将有利于防御,但过渡期会很艰难。
Vitalik Buterin: I didn't predict and do not predict defense running ahead of offense automatically. I predict cybersecurity will be endgame defense-favoring but the transition will be hard.
特别地,“好人比坏人更早地在已发布的软件中发现漏洞”并不是一种有利于防守方的博弈。要让博弈变得有利于防守方,需要在方法论上做出更深层的转变,包括形式化验证、激进的 TCB 最小化等,而我们才刚刚开始进行这些转变。
In particular, "good guys find vulns in already-released software before the bad guys do" is not a defense-favoring game. Making the game defense-favoring requires deeper shifts in methodology including formal verification, aggressive TCB minimization, etc., and we've only barely started making those transitions.
(这一区别是我持续支持放缓/暂停的重要原因之一,并且我们需要将超过 GDP 的 1% 投入到网络、生物和物理世界的加固中。)
(This distinction is a big reason why I continue to be friendly toward slowdowns/pauses, and we need >1% of GDP going toward cyber + bio + physical world hardening.)
中国已经意识到这一切牵涉到一些问题,并已就修订其 AI 出口管制公开征求意见。他们和我们一样。
China has realized there are some issues involved in all this and have solicited comments on revising their AI export controls. They are just like us.
最终,'我们鼓励开放模型'与'但不能让不该拥有的人拿到 [X]'之间必将发生冲突。你无法让一个开放的前沿模型不具备 [X],也无法对其真正实施'出口管制'。
Eventually there is going to be a clash between 'we encourage open models' and 'but you cannot let the wrong people get [X].' You can't make an open frontier model not [X], or meaningfully apply 'export controls' to it.
在这种情况下,首要的 [X] 是'恶意代码生成',而如果你希望模型能写出好代码,这尤其无法防范。于是你会不断看到既说'模型真好'又说'但不能有 [X]'的表述,最终总有一方要让步。
In this case the primary [X] is 'malicious code generation' and that is especially impossible to prevent if you want your model writing good code. So you get continuous statements that say both 'yay models' and 'but not [X]' and eventually something will have to give.
中国商务部(译自中文):本次公开征求意见涉及拟在人工智能、生物医药、光伏三个领域新增至清单的 40 项关键技术。
Chinese Ministry of Commerce (translated from Chinese): This public consultation involves 40 key technologies proposed for addition to the list in three fields: artificial intelligence, biomedicine, and photovoltaics.
其中,拟禁止出口的技术有 14 项,包括:AI 驱动的恶意代码生成与变异技术、具有自主知识产权的细胞与基因治疗技术及工艺平台,以及钙钛矿与叠层太阳能电池组件生产技术;
Among them, 14 technologies are proposed to be prohibited from export, including: AI-driven malicious code generation and mutation technology, cell and gene therapy technology and process platforms with independent intellectual property rights, and perovskite and tandem solar cell module production technology;
拟对 26 项技术实施出口限制,包括:AI 辅助漏洞挖掘技术、具有自主知识产权的通用型细胞与基因治疗新靶点发现与验证技术,以及单线产能 ≥3 万吨/年的多晶硅工程技术。
26 technologies are proposed to be restricted from export, including: AI-assisted vulnerability mining technology, general-purpose cell and gene therapy novel target discovery and verification technology with independent intellectual property rights, and polysilicon engineering technology with a single-line capacity of ≥30,000 tons/year.
1. 管制范围精准且有限。
1. The scope of control is precise and limited.
2. 适用便利化措施。
2. Facilitation measures are applicable.
3. 实施动态调整。
3. Dynamic adjustments are implemented.
……首先,确保人工智能(AI)技术的安全性与可控性至关重要。
… First, it is crucial to ensure the safety and controllability of artificial intelligence (AI) technology.
AI 带来的各种固有风险和衍生风险已引起全球广泛关注。为监管而提出的 AI 驱动的恶意代码生成与变异技术,可能被用于攻击国家关键信息基础设施、国防和军事工业、政府核心系统,或绕过国家安全防御系统。
The various inherent and derivative risks arising from AI have drawn significant global attention. The AI-driven malicious code generation and mutation technologies proposed for regulation could be used to attack critical national information infrastructure, defense and military industries, core government systems, or to bypass national security defense systems.
若被非国家行为体或恶意实体获取,可能危及国家安全和人类共同安全。中国政府对这类技术的必要管理旨在夯实安全基线、防止误用和滥用、确保 AI 处于人类控制之下,并推动 AI 国际治理。
If acquired by non-state actors or malicious entities, they could endanger national security and the common security of humanity. The Chinese government's necessary management of these technologies aims to solidify the security baseline, prevent misuse and abuse, ensure that AI remains under human control, and promote international governance of AI.
第二,我们鼓励开源、开放协作和资源共享。
Second, we encourage open source, open collaboration, and resource sharing.
……对于被列为限制类的 AI 技术,海外本地化部署属于技术出口管理范畴;但若在相对可控的场景(如海外自有云)中部署,可适用自动许可等便利措施。
…For AI technologies classified as restricted, overseas localization deployment falls under the scope of technology export management; however, if deployed in relatively controllable situations such as overseas self-owned clouds, convenient measures such as automatic licensing may apply.
Corbin Duncan 称这是“朝着关闭中国前沿迈出的一大步”。
Corbin Duncan calls this a 'big step towards closing the Chinese frontier.'
Dean Ball 的看法在这里似乎是正确的。中国正在创造可选择性,并划定了一个日期,此后需要更正式地申请许可。他们自己也不知道是否以及何时会动用这一手段。
Dean Ball's take seems correct here. China is creating optionality, and drawing a date after which you need to ask permission more formally. They do not themselves know if and when they will use it.
Dean W. Ball(OpenAI):此举并不意味着“中国正在关停开放权重模型”,但它确实表明中国政府正在建立一种可选择性,以便随时关停未来的开放权重模型。这也是美国政府已经使用的模型权重出口管制制度的一面镜像。
Dean W. Ball (OpenAI): This move does not mean "China is shutting down open-weight models," but it does indicate the Chinese government establishing the optionality to shut down future open-weight models at will. It's also a mirror image of the model-weight export control regime USG has already used.
你听到的一个重大争论,是在以下两种主张之间:我们需要防御方拥有差异化优势,以及我们需要尽快将能力扩散给所有人。
One big argument you hear is between these two claims: that we need defenders to have differential advantage, and that we need to diffuse capabilities as quickly as possible to everyone.
Dean W. Ball(OpenAI):每个人都应当寻找数据点,以推进 AI 政策中由来已久的争论:一方主张“迅速扩散两用能力,让好人能够自卫”,另一方主张“将能力扩散限制在经过仔细审查的好人范围内”。我们此刻正在检验前一种立场。
Dean W. Ball (OpenAI): Everyone should be looking for data points to advance the longstanding debate in AI policy between "diffuse dual-use capabilities rapidly so the good guys can defend themselves" and "restrict capabilities proliferation to only carefully vetted good guys." We are putting the former position to the test as we speak.
不要为了得出你偏好的结论而进行动机性推理;只需保持关注,并根据事实朝任何看似正确的方向适当更新。这件事太重要了,不能陷入低保真度的部落式极化。
No motivated reasoning to reach your preferred conclusion; just pay attention and update as appropriate in whatever direction seems true. This is much too important for low-fidelity tribal polarization.
我认为第二种立场——“将两用能力广泛传播”——的论据是:反正我们终究会这么做,所以不如现在就做。
I see the argument for the second position, "spread dual-use capabilities far and wide," as being that we are going to do that anyway so we had better do it now, instead.
我不认为这是一个好论据,也不认为它会有好结果,即使你感到绝望并认为我们无法长期阻止能力的发展或扩散。
I do not think that is a good argument, or likely to end well, even if you despair and think there is no way we can stop capabilities development or diffusion for long.
这仍然留下一个难题:在只控制部分而非全部关键决策的情况下,如何为防御方获得最大的差异化优势。
That still leaves the puzzle of how to get the largest differential advantage for defenders, given control over some but far from all key decisions.
你需要在信任多少人、信任到什么程度并因此给予助力,与这些人可能不值得信任的风险之间取得平衡。
You need to strike a balance between how many people you trust to what extent and thus give a leg up, versus the risk that they cannot be trusted.
我目前的判断是,我们在谁可以获得防御方工具的问题上过于保守了。我认为我们误解了威胁模型。情况主要不是“某个恶意行为者拿到 Mythos 一天,然后一切都乱套了”。更准确地说,是“那些能够接触到危险能力的人不断加剧混乱的程度”。而且,适当的监控和分类器似乎很可能告诉我们某个账户是否以及何时被攻破。
My current read is that we are being far too conservative about who gets access to the tools for defenders. I think we misunderstand the threat model. The situation is mainly not 'one rogue actor gets their hands on Mythos for a day and all hell breaks loose.' It is more like 'those with access to dangerous capabilities continuously ramp up the amount of hell breaking loose.' It also seems likely that proper monitoring and classifiers can tell us if and when one of the accounts gets compromised.
即便是纯粹的“我们可以信任的 KYC,以及我们有信心可以送进监狱的人”,也能起很大作用。我预计我们会随着时间推移在这个平衡上做得更好。
Even pure 'KYC we can trust and someone we are confident we can throw in jail' goes a long way. I expect us to get better at this balance over time.
另一种选择是,我们实际上确实在此基础上提供访问,就像 Daybreak 在 15 分钟内给你访问权限,只不过没人知道要开口问。我们不能等他们来问。
Alternatively, we actually do offer it on that basis, as in Daybreak giving you access in 15 minutes, except no one knows to ask. We need to not wait for them to ask.
如果开放模型的能力发展速度超过我们修补漏洞的能力,那么这一切都无关紧要。我们应该投入更多的资源来修补漏洞,许多人正在尝试,但公司和个人大多无法被强制要求遵守。理想情况下,我想给他们更严厉的打击,或许通过强制高额赏金支持的白帽探测来威胁他们。
None of that matters if open model capabilities outpace our ability to patch. We should be committing far more resources to patching, and many are trying, but companies and individuals mostly cannot be forced to get in line. Ideally I’d like to give them more of a swift kick, perhaps by the threat of white hat probes backed by mandatory large bounties.
上周“大黑客事件”的增多,很可能很大程度上归因于新的 DeepSeek 模型。它不是前沿模型,但快速、廉价且相当不错,这很可能改变了许多人的算计。现在我们可能面临新的 Kimi 问题。好消息是,Kimi 声称有意在 cyber 能力上有所保留。开放模型的倡导者将需要接受这类事情和其他限制,随着时间的推移,中国将会提供。
This last week of greater Hackening is plausibly in large part due to the new DeepSeek model. It is not frontier, but it is fast and cheap and pretty good, and that likely is tipping the calculus for many. Now we have a potential new Kimi problem as well. The good news is that Kimi claims to have intentionally held back on the cyber capabilities. Open model advocates will need to accept such things, and other restrictions, over time, and China will provide.
我们看到的许多损害是累积性的损害,人们所做的许多事情似乎是机会主义的,且资源不足。因此,采取“让发动攻击和使用好模型变得极其烦人”的策略,作为额外的“通过隐匿实现安全”策略,可能比你想的持续更久并争取更多时间。最终,顶级坏家伙会无限制地扩展,那时这一切都无关紧要,就像在超级智能之后许多事情将不再重要一样。目前,我们需要我们能获得的每一个优势。
A lot of the damage we are seeing is accumulated damage, and a lot of what people are doing seems opportunistic and not well resourced. Thus, a strategy of ‘make doing attacks and using the good models maximally annoying’ as an additional ‘security through obscurity’ strategy might last longer and buy more time than you would think. Eventually the top bad guys scale without limit and none of that matters, the same way that so many things will stop mattering after superintelligence. For now, we need every advantage we can get.
在个人层面上,我敦促采取 Peter Wildeford 列表中的普通预防措施。
On a personal level, I would urge taking ordinary precautions in Peter Wildeford’s list.
* 使用一个好的密码管理器(1Password),并且不要重复使用密码
* use a good password manager (1Password) and do not duplicate passwords
* 对所有重要账户启用双因素认证(不要通过短信!!使用 Google Authenticator,或者更好的 YubiKey)
* Use two-factor authentication for everything important (not via SMS!! Google Authenticator or better yet YubiKey)
* 开启自动更新,并每周重启一次,以便更新真正安装
* Turn on auto-updates, and restart weekly so they actually install
* 冻结你的信用
* Freeze your credit
* 获取 IRS 身份保护 PIN
* Get an IRS Identity Protection PIN
* 让你的移动运营商为你设置转网 PIN 或号码锁,使运营商在没有该 PIN 的情况下拒绝转移你的号码
* Get your cell carrier to give you a port-out PIN or number lock that makes the carrier refuse to move your number without the PIN
假设情况在好转之前会先恶化,并且存在一长串潜在的、可能使局面变得相当糟糕的途径。但请不要恐慌。不要取消行程、回避乘坐飞机或做诸如此类的事情。请记住,这是一个长期问题,而非急性危机。你需要的是能够长期坚持的预防措施。
Assume that things will get worse before they get better, and that there is a long tail of potential ways things could get quite bad. But please do not panic. Do not cancel trips, avoid airplanes or anything like that. Remember this is a long haul problem, not an acute crisis. You want precautions that you can sustain.
与许多类似情形一样,你与其说要跑赢那头熊,不如说要跑赢其他人——至少在最初的几轮中如此。然后,你需要为其他事物被击垮做好准备。
As with many such things, you do not so much need to outrun the bear as much as outrun the other people, at least in the opening rounds. Then you need to prepare for other things to get taken down.