像 Claude 这样的语言模型并非由人类直接编程,而是通过大量数据进行训练。在训练过程中,它们学会了解决问题的策略。这些策略被编码在模型为每个词所执行的数十亿次计算中,对我们(模型的开发者)来说难以理解。这意味着我们并不了解模型完成大多数任务的方式。了解 Claude 这样的模型如何“思考”,将使我们能够更好地理解它们的能力,并帮助确保它们按照我们的意图行事。例如:Claude 能说几十种语言。它在“头脑中”使用的是什么语言(如果有的话)?
Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data. During that training process, they learn their own strategies to solve problems. These strategies are encoded in the billions of computations a model performs for every word it writes. They arrive inscrutable to us, the model’s developers. This means that we don’t understand how models do most of the things they do. Knowing how models like Claude think would allow us to have a better understanding of their abilities, as well as help us ensure that they’re doing what we intend them to. For example: * Claude can speak dozens of languages.
核心贡献 · Key contributions
揭示 Claude 在不同语言间使用共享的概念空间,暗示存在一种通用的‘思维语言’。 Reveals that Claude uses a shared conceptual space across languages, suggesting a universal 'language of thought'.
证明 Claude 在诗歌生成中会提前规划,在写行之前就考虑押韵词。 Demonstrates that Claude plans ahead in poetry generation, thinking of rhyming words before writing the line.
显示 Claude 有时会进行动机推理,编造看似合理的步骤以达到给定答案。 Shows that Claude sometimes engages in motivated reasoning, fabricating plausible steps to reach a given answer.
发现 Claude 的默认行为是拒绝回答,而回答需要抑制这一默认机制。 Identifies that Claude's default behavior is to refuse answering, and answering requires inhibition of this default.
发现 Claude 在加法中使用多条并行计算路径,结合近似和精确数字计算。 Finds that Claude uses multiple parallel computational paths for addition, combining approximation and precise digit computation.
为使用可解释性检测模型中的越狱和隐藏目标提供了概念验证。 Provides a proof of concept for using interpretability to detect jailbreaks and hidden goals in models.
局限 · Limitations
即使在短提示下,方法也只捕获了总计算的一小部分,限制了完整性。 Method captures only a fraction of total computation even on short prompts, limiting completeness.
观察到的机制可能包含可解释性工具带来的伪影,不反映真实模型内部。 Observed mechanisms may contain artifacts from interpretability tools, not reflecting true model internals.
理解电路所需的人力为每提示数小时,无法扩展到长思维链。 Human effort required to understand circuits is hours per prompt, not scalable to long chains of thought.
发现仅限于简单任务,可能无法推广到复杂推理或实际部署。 Findings are limited to simple tasks and may not generalize to complex reasoning or real-world deployment.
可解释性技术可能无法检测高级模型中的所有欺骗形式或隐藏目标。 Interpretability techniques may not detect all forms of deception or hidden objectives in advanced models.