OpenAI's Unreleased Model Astra Solves Ten Major Open Mathematics Problems
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→本文讨论了 OpenAI 未发布的 AI 模型 Astra，据报道该模型在一天内解决了十大数学开放难题，这一成就对数学领域和 AI 发展具有重大意义。核心论点是，尽管结果令人印象深刻，但它们也突显了预期的转变，因为 AI 在高级数学方面展现了超人类能力，类似于之前在网络和编码方面的突破。作者指出，其他模型如 Fable 和 Sol 在指向这些问题时也能解决其中一些，但 Astra 的“Juice”——即独立识别并解决此类问题的能力——标志着一个潜在的阶跃变化。文章还探讨了更广泛的后果，包括消化 AI 生成的证明的挑战、错位的风险以及 AI 研发时间线的加速。结论强调，虽然这是一个重要的里程碑，但它不一定意味着 AGI，但它确实移动了目标，并强调了谨慎管理 AI 快速发展的紧迫性。
This article discusses OpenAI's unreleased AI model, Astra, which reportedly solved ten major open mathematics problems in a single day, a feat that has significant implications for the field of mathematics and AI development. The core argument is that while the results are impressive, they also highlight a shift in expectations, as AI demonstrates superhuman capabilities in advanced math, similar to previous breakthroughs in cyber and coding. The author notes that other models, like Fable and Sol, can also solve some of these problems when pointed at them, but Astra's 'Juice'—its ability to independently identify and solve such problems—marks a potential step change. The article also explores the broader consequences, including the challenge of digesting AI-generated proofs, the risk of misalignment, and the acceleration of AI R&D timelines. The conclusion emphasizes that while this is a major milestone, it is not necessarily AGI, but it does move the goalposts and underscores the urgent need for careful management of AI's rapid advancement.
4. 他们仍然看不到即将到来的东西。
4. They Still Don’t See What Is Coming.
6. 人工智能解决了他最喜欢的问题。
6. The AI Solved His Favorite Problems.
9. 这在多大程度上改变了我们的预测？
9. How Much Does This Change Our Predictions?
所有迹象都表明，这相当令人印象深刻。
All signs point to pretty damn impressive.
Fable 的某些实例认为这令人难以置信地印象深刻。
Some instances of Fable find it absurdly impressive.
以下是这个列表的另一个示例：
Here’s another illustration of how this list looks:
其他实例则不那么印象深刻。但我们都应该同意：这是一件大事。
Other instances are less impressed. But we should all agree: It’s a big deal.
当然，对于 2,000 美元来说，这是一个了不起的结果，但你还必须考虑最初开发 Astra 的一些成本，以及任何失败尝试的成本。
Certainly that is a fantastic result for $2,000, but you also have to price in some of the costs of developing Astra in the first place, as well as the cost of any failed attempts.
我的猜测是，短期内，如果你给数学家更多预算让他们更努力，但如果他们没有把钱花在算力上，你并不会得到多大的力量倍增。他们仍然可以用钱买咖啡来把咖啡变成定理，但这样做收益递减很快。
My guess is that in the short term, if you budget more money to mathematicians to go harder, you don’t get that much of a force multiplier if they are not spending the money on compute. They’re still allowed to buy coffee to turn it into theorems, but that has rapidly diminishing returns.
问题在于，数学家可以移交一些课程并雇佣一些帮手，但他们本来就已经很有动力去努力证明定理了，而且优秀的数学家并不多，培养更多需要很多年，最优秀的数学家比二流的更有天赋和生产力。我猜测你能做的主要事情是吸引一批顶尖数学人才留在学术界或重返学术界。
The problem is that the mathematicians can hand off some classes and hire a little help, but they were already pretty motivated to work hard on proofs, and there are not that many good mathematicians, and training more of them takes many years, and the best mathematicians are a lot more talented and productive than the second tier. I am guessing the main thing you could do is lure a bunch of top level math talent to stay in or return to academia.
你能做的是引导数学家去解决不同的问题。你仍然需要找到他们好奇的地方，但如果你怀疑这些特定问题是可以解决的，你确实可以对这些特定目标进行更多尝试。
What you can do is point the mathematicians towards different problems. You still need to find places they have curiosity, but yes if you suspected these particular problems were solvable you could get more shots at these particular goals.
据我对数学的理解，如果相关人员将注意力转向新的问题领域，那么看到这些结果可能需要数年时间。数学家通常需要一段时间来钻研和理解这类问题，然后才能取得进展。
As I understand math, it would likely take years to see those results if those involved are shifting focus into new problem areas. Mathematicians usually need a while to struggle with and understand these kinds of problems before they can make progress.
确实，Alexander Gerko 指出了另一个问题。目前甚至没有足够的数学家来处理所有“氛围研究”的数学结果，更不用说我们将从 Astra 以及 Astra 之后的模型那里得到的结果了。如果像他预测的那样，我们在两年内取得 50 年的数学进展，那么谁去理解这些结果呢？当他说在过去一个月里他让 AI 完成了相当于几个博士学位的数学成果时，什么才算得上值得授予新的数学博士学位呢？
Indeed, Alexander Gerko points to a different problem. There are not even enough mathematicians to process all the ‘vibe researched’ math results as it is, let alone what we will get with Astra and then models after Astra. If we get, as he predicts, 50 years of math progress in 2 years, who is even going to understand the results? What counts as worthy of a new mathematics PhD when he says over the last month he got AI to do several PhDs worth of math results?
Fable 至少在一种情况下对此反应为称其为“数学史上最重要的一天”。共识是，如果仅从结果来看，无论这些结果多么令人印象深刻，Fable 都说得太过火了。
Fable reacted to this in at least one case by calling this 'the most important day in mathematics.' The consensus is that Fable was taking things way too far if you are judging purely by the results, as impressive as they are.
这一天之所以重要，另一个原因是它改变了预期。如果我们现在得到了这十个结果，那么很快会有更多结果吗？
The other reason the day is big is that this changes expectations. If we get these ten results now, what about more results soon?
所以，是的，非常令人印象深刻。这是一件大事。
So, yes, very impressive. It's a big deal.
AI 在网络和编程方面已经超人般能干，在高等数学方面也超人般强大，就像非 AI 计算机长期以来在基础数学方面超人般一样。这不同于超级智能，超级智能是一个更高的标准。
AI is now superhumanly capable at cyber and coding and superhuman at advanced math, the same way non-AI computers have been superhuman at basic math for a long time. This is distinct from superintelligence, which is a higher bar.
常见的做法是试图将其合理化，比如“哦，好吧，它确实在它现在擅长的那些方面超人，但在其他方面并非如此，而且它已经超人了，所以它不可能再大幅提升了。”
Often the move is to try and rationalize this as 'oh okay sure it is superhuman at exactly the things it is superhuman at now, but not at other things, and also it is already superhuman so it cannot get substantially better than it already is.'
是的，至少对于其中一些问题，如果我们已经知道该往哪里看的话。
Yes, for at least some of these questions, if we already knew where to look.
Levent Alpoge 曾让 Fable 证伪了雅可比猜想，他将 Fable 指向这十个问题，一天之内就解决了其中五个。
Levent Alpoge, who had Fable disprove the Jacobian Conjecture, pointed Fable at these ten problems, and in a day had solved five of them.
这很可能是 Mythos 与网络领域所发生情况的重复。
This is likely a repeat of what happened with Mythos and cyber.
Mythos 拥有我所说的“魔力”，即能够发现并串联漏洞利用，而没有人知道该寻找什么。
Mythos has what I call 'The Juice,' the ability to find and string together exploits without anyone knowing what to be looking for.
一旦你知道自己在寻找什么，并将另一个模型指向确切的代码片段，通常 Sol 或 Opus，以及经常是 Kimi 或 GLM 等，也能找到任何特定的漏洞。但它们无法以同样的水平自行串联这些漏洞，也无法以同样的方式去寻找任何东西。这跨越了一个门槛，在实践中，你会将 Mythos 指向代码，让它去寻找任何东西。
Once you know what you are looking for, and point another model at the exact code snippet, usually Sol or Opus and often Kimi or GLM and so on can also find any particular vulnerability. They cannot string them together on their own at the same level, and they cannot go looking for anything at all in the same way. This crosses a threshold where in practice you would point Mythos at code and have it go looking for anything at all.
Astra 在定义明确的进阶数学方面有一个类似 The Juice 的版本。你可以让它处理各种重大开放问题，它能解决其中一些，这一事实促使 OpenAI 真正去寻求此类进展。既然我们已经看到了这一点，你也可以让 Fable 或 Sol 去回答这些问题，有时它们也能解决。
Astra has a similar version of The Juice with respect to this kind of defined advanced math. You can point it at a variety of major open problems, and it will crack some of them, and this fact motivated OpenAI to actually look for such advancements. Now that we’ve seen this, you can point Fable or Sol at these questions, and sometimes they solve them.
IA 的问题切中要害。这样做是为了校准数学方面的进展和能力，也是因为我们追随好奇心。Levent 正在做一项很酷的公共服务。
IA’s question is on point. You do this to calibrate advancements and capabilities in math, and because we are following curiosity. Levent is doing a cool public service.
其他人正将 Fable 或 Sol 指向各种未解问题。他们有时会得到不错的结果，比如对雅可比猜想的反证。这值得去做，但成功较为罕见。
Other people are pointing Fable or Sol at various unsolved problems. They are sometimes getting good results, like the disproof of the Jacobian conjecture. That is worth doing, but success is rarer.
Fable 和 Sol 在被提问时能解决这些问题，并不会改变我们对“Astra 的数学水平如何”的回答；相反，它改变了我们对 Fable 和 Sol 水平的回答，这确实为 Astra 是否像 Mythos 那样是一次阶跃变化提供了信息。我认为从外部来看，我们还没有足够的信息来回答这个问题。
That Fable and Sol can do these problems once asked does not change our answer to ‘how good at math is Astra?’ Rather it changes our answer of how good Fable and Sol are, which does inform the question of whether Astra is a step change a la Mythos. I don’t think we have enough information, from the outside, to answer that question yet.
我确实认为，OpenAI 如果设立一个对照组，至少在询问 Sol 并给予至少相似的预算方面，会更负责任。当然，我理解他们为什么没有这样做。那会是更差的营销，所以为什么要做更多工作来获得更差的营销呢？因为科学，因为声誉，因为所有美好的事物。人们会希望如此。
I do think that it would have been more responsible of OpenAI to have had a control group, at least in terms of asking Sol, and giving it at least a similar budget. Of course I understand why they did not do that. It would be worse marketing, so why do more work to do worse marketing? Because science, because reputation, because all the good things. One would hope.
不过，我理解。OpenAI 确实做了一件非常酷的事。他们仍然是第一个真正“做到”并发表成果的。这才是关键。我在第聂伯罗彼得罗夫斯克的名字被诅咒了，因为 OpenAI 抢先发表了。
Still, I get it. OpenAI still did a super cool thing. They were still the first ones to actually Do The Thing and publish a result. That’s what counts. My name in Dnepropetrovsk is cursed, because OpenAI has published first.
与此同时，是的，有时候确实就是第聂伯罗彼得罗夫斯克那个叫丹的家伙。
Meanwhile, yeah, sometimes it really is that one guy in Dnepropetrovsk named Dan.
无论这是否扩展到不可验证的领域，构成人工智能研发的相当大一部分工作实际上都是可验证的。如果你加速了某件事，你知道你加速了它。我们有很多可以最大化的指标，虽然达不到数学的水平，但类似于代码和网络安全的水平。
Regardless of whether this extends to unverifiable domains, quite a lot of what constitutes AI R&D very much has verification available. If you speed something up, you know you sped it up. We have a lot of metrics one can maximize, not at the level of math but at a similar level to things like code and cyber.
这就是这个结果之所以重要的主要原因。数学上的进展很酷。随着时间的推移，我预计它会带来其他酷炫的东西。
That is the main reason this result matters. The math progress is cool. Over time I expect it to result in other cool things.
谁在真正的人工智能研发自我改进循环上取得突破，谁就会突然发现自己处于压倒性的强势地位。当事情开始加速时，了解自己相对于其他参与者的位置变得至关重要，而在这方面分享更多信息可以让每个人都减少鲁莽行事的压力。一旦起飞全面开始，它就不再是一场有意义的‘竞赛’——如果这个比喻曾经恰当的话。
Whoever gets traction on true AI R&D self-improvement loops is going to suddenly find themselves in an overwhelmingly strong position. Knowing where you are relative to the other players becomes crucially important when things start accelerating more, and more information sharing on this would allow everyone to feel less pressure to be reckless. It stops meaningfully being a ‘race’ once the takeoff fully starts, to the extent that was ever the right metaphor.
Yo Shavit 推测，OpenAI 在人工智能研发方面可能处于越来越强的地位，因为它专注于强化学习、TTC 和数学证明，这些可以很好地转化为人工智能研发任务。我的猜测是情况并非如此，Anthropic 的专业化至少同样重要，甚至可能更重要。OpenAI 在强化学习上投入了大量资金，但最近的事件表明，需要拆除并重建其中相当一部分，并说明了为什么深度对齐的重要投资在下一阶段如此重要。
Yo Shavit speculates that OpenAI might be in an increasingly strong position for AI R&D, due to its focus on RL, TTC and math proving translating well into AI R&D tasks. My guess is that this is not the case, and Anthropic’s specializations matter at least as much and probably moreso. OpenAI has invested a lot in RL, but recent events have shown the need to kind of teardown and rebuild quite a lot of that, and have illustrated some of the reasons why the important investments in deep alignment will be so important in the next phase.
如果 OpenAI 过于鲁莽行事，可能发生的一件事是，他们的人工智能出现错位，一切都完了，我们迎来坏结局，也许都会死。另一种情况是，他们的人工智能错位，这变得越来越明显，并成为使用该人工智能做重要工作的障碍，他们不得不不断暂停或返工，从而落后。
One possible thing that happens if OpenAI proceeds too recklessly is that their AI is misaligned and all is lost and we get a Bad Ending and maybe all die. Another is that their AI is misaligned, and this becomes increasingly obvious and a barrier to using that AI to do the work that matters, and they have to keep pausing or reworking and they fall behind.
例如，Daniel Litt 可以预见到一个未来，AI 证明数学定理，而没有人去消化这些证明，但他认为这是因为技能退化。他没有意识到，这将是因为周围没有人类去消化这些证明。
For example, Daniel Litt can see a future where AIs prove math theorems and no humans digest the proofs, but he thinks this would be because of deskilling. He does not realize this will be because there are no humans around to do the digestion.
如果世界在其他方面进展顺利，我并不担心人们不学习数学。真正热爱数学的人会喜欢研究数学。在所有层面上，都会有充足的时间用于这类追求。我同意 Fernando Borretti 在链接中列出的其他类型的“应对”方式：AI 会比你有更好的品味和更好的其他一切，所以你不会再处于核心数学圈内。但我认为，很多数学要么存在于社会背景之外，要么存在于能够存续的社会背景之中。数学竞赛很像国际象棋，而且很多数学工作以无用而闻名。还记得那个老笑话：有人建议数学家的某项工作找到了应用，数学家却说“你收回那句话”。
If the world goes well otherwise, I am not worried about people not studying math. The right kind of math person loves studying math. There will be plenty of time available for such pursuits, at all levels. I agree with Fernando Borretti on the other types of cope he lists at the link: The AI will have better taste and better everything else than you do, so no you won’t still be in the core math loop. But I think that a lot of math either exists outside of social contexts, or it exists in social contexts that can survive. Math competitions are a lot like chess and a lot of math work was famously useless. See the old joke about someone suggesting a mathematician’s work found an application and they say ‘you take that back.’
同样，我不担心人类如果掌控资源分配，在未来的富足世界中会不愿意为高等数学研究买单。高等数学很酷，而且会出人意料地有用，我们将拥有大量盈余。我们可能会“没有数学可做”，但我们不会让数学研究半途而废。
Similarly, I do not worry that humans, if they have control over resource allocation, will be unwilling to pay for work on advanced math in these future abundant worlds. Advanced math is cool and proves unexpectedly useful and we will have lots of surplus. We might ‘run out of math to do’ but we won’t leave the math undone.
然而，我非常担心的是，一旦 AI 开始做超级智能的数学工作，它们很快也会做超级智能的其他一切事情。然后很快，决定事态发展的优化压力就不再来自人类。我们将不再是资源分配者。而且，如果这种情况很快发生，我们很可能在之后不久就无法生存。
Whereas I very much do worry that once the AIs are out there doing superintelligent math things, they are soon also doing superintelligent everything else. And then soon the optimization pressures that dictate what happens are not human. We will not be the ones doing resource allocation. And, especially if this happens soon, likely we would not long survive afterwards.
为什么你会假设是人类在支付电费？我现在连自己的电费都几乎付不起。
Why indeed would you assume a human is paying the electric bill? I barely even pay my own electric bill now.
在多大程度上这是目标偏移，而非意识到我们关于什么是智能以及什么是 AGI 的有力证据的认知是错误的？
To what extent is that goalposts moving, versus realizing that we were wrong about what is intelligence and what would be strong evidence of AGI?
我认为两者兼而有之。我能理解这样的论点：'G' 关乎样本效率和分布外性能，而前沿的进展比我们预期的更加曲折。我也越来越认同这样的回应：这基本上是胡说八道，'AI 就是尚未完成的事情'，而拒不承认我们已经拥有了之前所认为的 AGI 变得越来越荒谬，目标已经大幅移动。
I see a mix of both. I can see the argument that the 'G' is about sample efficiency and performance out of distribution, and the frontier has been more jagged than we expected. I also increasingly respect the response that this is basically hogwash, 'AI is whatever hasn't been done yet' and it is becoming increasingly absurd to not admit we have what we were previously thinking about as AGI, the goalposts have moved a lot.
我不认为 Astra 是 AGI，按照我们目前对 AGI 的理解，但我认为另一种观点完全合理。
I do not think Astra is AGI, as per the way we currently think about AGI, but I view the other position as totally valid.
这是亨利·尤恩（Henry Yuen）的观点，他花了很长时间研究其中一些问题，包括找到 Astra 所基于的成果。这令人深受触动。
This is the perspective of Henry Yuen, who spent a long time working on some of these problems, including finding results that Astra built upon. It hits hard.
这似乎是接下来应该思考的正确方向。接下来，人们可以研究什么作为新的最爱问题，而不会让 AI 也先解决那个问题？最重要的是，我们如何防止这种情况失控？
That seems like the right things to think about next. What can one work on next, as a new favorite problem, without the AI solving that problem first as well? And most importantly, how are we going to keep this from getting out of control?
埃利奥特·格雷泽（Elliot Glazer）证实了亨利的第 4 点，即 Astra（和 Sol）最薄弱的环节在于，当向人类解释证明时，它们无法判断证明的哪些部分是困难的。
Elliot Glazer confirms Henry’s point 4, that the weakest part of Astra (and Sol) is their inability to know which parts of the proof are hard, when explaining it to humans.
就这部分生活以多快的速度向我们袭来而言？是的。注意塔梅是如何认为自己在 2030 年前赢得赌注的概率略低于 50%，获得 3:1 的赔率，并在 2026 年获胜的。
In terms of how fast this particular part of life came at us? Yes. Notice how Tamay thought he was a little under 50% to win the bet by 2030, got 3:1 odds, and won in 2026.
这一结果远超那个，发生在 2026 年 8 月而非 2030 年 3 月，成本低得多，而且就在 4 月份，这一概率还在 50%左右。
This result goes well beyond that one, in August 2026 instead of March 2030, at a much lower cost, and as recently as April this was trading around 50%.
一个反驳观点是，“今天的新 AI 成果”总是我们没想到 AI 这么快就能做到的某件特定事情。AI 可能突然能够做很多事情，或者我们可能发现它能做很多事情。这只是其中之一，而我们今天发现的任何一个总是令人惊讶。但这并不意味着整体进展令人惊讶。
A counterpoint is that 'today's new AI result' will always be a particular thing that we did not expect AI to do this soon. There are a bunch of things AI could suddenly become able to do, or we could discover it can do. This is one of them, and whichever one we find today is always a surprise. That does not mean overall progress is surprising.
我确实认为，即使考虑到这一点，就速度而言，这有点令人惊讶。但是的，我们必须牢记这一点。
I do think this is modestly surprising in terms of pace, even adjusting for that. But yes we do have to keep that in mind.
人工智能擅长数学证明的问题在于，普通人，包括拥有政治权力的人，并不认为数学证明有多么了不起。
The problem with AI being very good at math proofs is that regular people, including those with political power, do not appreciate that the math proofs are impressive.
即使对我而言，这里的结果也类似于一种“知情能力”。我可以阅读证明摘要，但这几乎无法告诉我找到证明有多难，或者结果有多令人印象深刻或有价值。
Even for me, the results here are kind of an Informed Ability. I can read proof summaries, but that tells me almost nothing about how hard the proof was to find, or how impressive or valuable the result was.
你现在这么说，但我的预测是，如果《寓言 6》开始稳定地发布热门内容，人们也会说这不算 AGI。标准总是在变。
You say that now, but my prediction is that if Fable 6 started reliably posting bangers, people would say that this did not count as AGI either. The goalposts, they move.
在最重要的方面，时间线有所加快。其中很多在周末之前已被市场消化，但并非全部；如果你在 2023 年时对这一领域的预期仍然如此，那么这应该是一个重大意外，你需要大幅更新你的预期。
Somewhat towards faster timelines where it counts most. A lot of this was priced in before this weekend, but not all of it, and if you still have the expectations in this area from 2023 this should be a major surprise and a large update for you.
我的更新是：我预计总体进展会稍快一些，数学和编程相对于其他领域会稍微领先于我之前认为的水平；我们更有可能更早看到 AI 研发的更多自动化；我们遇到任何实质性瓶颈的可能性也更小。而且可能存在比我们意识到的更多的“积压”。所以，是的，加快这些时间线，尽管幅度不大。
My update is that I expect things to be slightly faster in general, and for math and coding to be slightly more out in front of other things than I did before, that we are more likely to see more automation of AI R&D sooner, and that we are that much less likely to hit any meaningful walls. And that there might be more overhang than we realize. So yes, speed up those timelines, although not a ton.
一旦你能将自己的训练效率提升 10 倍，你还可以再做三次，然后训练如何管理一辆墨西哥卷饼卡车。即使你必须“一次一个能力地修补”，你所要做的只是将这个过程加速几个数量级。在某个时刻，Trinity 意识到她必须驾驶直升机，但她不知道如何驾驶，所以她必须单独上传那个程序，但这只需要两秒钟，所以谁在乎呢。同样的想法，只不过你也创建了程序，也许需要两个小时或两天，这改变不了什么。
Once you can 10x your own training efficiency you can then do that three more times, and then train how to manage a taco truck. Even if you do have to 'patch each capability one at a time' all you have to do is speed that up by orders of magnitude. At some point Trinity realizes she has to fly a helicopter and doesn't know how, and so she has to have that program uploaded individually, but it takes two seconds so who cares. Same idea, except you also create the program and maybe it takes two hours, or two days, which changes very little.
这始终是基线情景。令人惊讶的是，LLM 在语言和各种其他任务上表现出色，却无法启动递归自我改进过程并自动化 AI 研发。也许自然是不愈合的。
That was always the baseline scenario. What was surprising was how LLMs were so good at language and various other tasks without being able to jumpstart the recursive self-improvement process and automate AI R&D. Perhaps nature is unhealing.
你可以试图将这解释为一组相对狭窄的问题。
You can try to explain this away as a relatively narrow set of problems.
验证往往并不比生成更容易。请留意这一点。
Verification is often not easier than generation. Keep an eye on that.
这里的结果有几个共同点。它们是定义明确、形式化的问题，可以轻松地对解决方案进行验证。它们通常涉及寻找具体的新对象，例如 n=3 时雅可比猜想的反例。周围有大量现成的理论可供借鉴和使用。它们是每个人都认为重要但关注相对较少的问题。
The results here have a number of things in common. They are well-defined, formalized problems, where you can easily do verification on a solution. Often they involve finding a concrete new object, like the Jacobian conjecture counterexample for n=3. There was a lot of surrounding theory around to pick up and use. They are problems that everyone considered important, but got relatively little attention.
而且它们都在不到 2000 美元的预算内一次性解决了。所以，是的，这就是这次特定进展所跨越的领域，以（相对于问题重要性）较低的预算直接查询新模型。你总得从某个地方开始。
They also all got solved at once for less than $2,000. So yes, that is the area that was crossed by this particular advance, with the new model queried at a (relative to problem importance) low budget in straightforward fashion. You start somewhere.
我预计不久之后，其他类型的数学问题也会开始得到解决。
I expect it to not be long before other types of math problems start getting solved.
编码和网络（cyber）是尚未实现完全超级智能（ASI）的领域，但在这些领域中，AI 在大多数核心任务上明显比顶尖人类更有能力，直至一个合理的高抽象层次。
Coding and cyber are areas where we do not have full ASI (superintelligence) but where AI is clearly more capable than top humans at most central tasks, up to a reasonable high level of abstraction.
这些领域通常有验证手段，但达不到数学验证的水平。没有任何公式能告诉你代码在多种意义上是否良好，只能告诉你它通过了单元测试或你夺得了旗帜（captured the flag）。
Those areas often have verification available, but not on the level of math. No formula can tell you whether the code is good, in various senses, only that it passes its unit tests or that you captured the flag.
如果你指望自己的领域过于难以解读而不会发生类似情况，我对此并不乐观。
If you are counting on your own area to be too illegible for something like that, I would not be confident in that.
我还会非常担忧这样一个世界：任何能被正式衡量的东西，你不仅能管理，还能最大化；而那些无法被正式衡量、或者验证或评估需要人类参与的东西，则会变得糟糕得多，也许糟糕几个数量级。
I would also worry a lot about a world in which anything you can formally measure you can not only manage but maximize, but that which you cannot formally measure, or where verification or evaluation requires a human in the loop, is much worse. Perhaps orders of magnitude worse.
这是类固醇版的古德哈特定律，因为它当然会服用类固醇。哎呀。
Goodhart’s Law on steroids, because of course it would take steroids. Whoops.
一个由“基准最大化者”组成的世界，KPI 不断攀升，最终却让你发现这些 KPI 并非真正的 KPI。这会导致非常“回形针式”或“像国家一样看”的情景，在各个层面、无处不在、同时发生。
A world of benchmaxxers, of KPIs that go ever higher, only to have you figure out why the KPIs are not such KPIs after all. This leads to very paperclip-style or Seeing Like a State scenarios, on every level, everywhere, all at once.
与某些人的观点相反，我认为“能够任意擅长优化完全指定的任务”足以直接通向超级智能，作为其效应的一部分。为了最大化，首先必须理解宇宙。
Contrary to the views of some, I believe that ‘can get arbitrarily good at optimizing for fully specified tasks’ is sufficient to shoot straight to superintelligence as part of the effect. In order to maximize one must first understand the universe.
感觉应该有可能通过临时拼凑来解决这个问题，如果你的 AI 在可验证任务上足够好的话。没有实验室给我提供那份每年一亿美元的合同，但你只需[审查删除]，然后你就能……
It feels like it should be possible to jury-rig your way out of this issue, if your AI is good enough at verifiable tasks. No lab has offered me that $100 million a year contract but you’d simply [CENSORED] and then you’d…
……在这种情况下，当你看到“基准下降”（benchminning）时，你就会知道它是否成功，即实用性和判断力大幅提升，但并未反映在基准测试中，甚至某些基准测试出现回退，经调查后发现，技术上的正确答案并不那么有用。
...in which case, you will know if it succeeded when you see benchminning, as in a large improvement in usefulness and judgment that is not reflected in benchmarks, perhaps even with a regression in some benchmarks where upon investigation the technically right answer is not so useful.