The components of agent memory
打开互动全文版(逐段中英对照 + 针对本篇提问)→本文探讨如何为 AI 智能体赋予跨会话持久有效的记忆,并将其类比于人类海马体与基于睡眠的记忆巩固。作者将智能体记忆定义为从经验中获得并在会话之间保留的知识,排除权重变化、上下文学习和压缩,并描述了一种分工:智能体在工作时标记显著事件,随后反思日志——这一过程称为“做梦”——以纠正错误、构建表征,并塑造预加载到未来会话中的内容。在 CL-Bench 数据库任务和《宝可梦红》上的评估表明,更强的模型通过记录经验证的规则和警告来闭合学习回路,而无差别的笔记会产生臃肿、迅速过时的记忆并降低性能。作者得出结论:对于前沿模型,告诉 Claude 记忆的用途而非如何使用它效果最佳,因为诸如只记录失败之类的固定规则会削弱记忆带来的收益;而“做梦”让 Claude 能够对照原始记录检验笔记,并纠正否则会传播的错误。
This article examines how AI agents can be given effective memory that persists across sessions, drawing an analogy to the human hippocampus and sleep-based consolidation. The authors define agent memory as knowledge gained from experience and retained between sessions, excluding weight changes, in-context learning, and compaction, and describe a division of labor in which agents tag salient events during work and later reflect on logs—a process called dreaming—to correct mistakes, build representations, and shape what is preloaded into future sessions. Evaluations on the CL-Bench database task and Pokémon Red show that stronger models close a learning loop by recording validated rules and warnings, while indiscriminate note-taking produces bloated, quickly stale memory that degrades performance. The authors conclude that telling Claude what memory is for, rather than how to use it, works best for frontier models, since fixed rules such as recording only failures can weaken the gains memory provides, and that dreaming lets Claude test notes against original transcripts and correct errors that would otherwise propagate.
2026 年 3 月 24 日 面向长期应用开发的 Harness 设计
Mar 24, 2026 Harness design for long-running application development
索伦·克尔凯郭尔曾写道,生活只能**向后**理解,但必须**向前**生活。我们在为 AI 智能体设计记忆时,也遇到了类似的困境。
Søren Kierkegaard wrote that life can only be understood _backward_, but it must be lived _forward_. We’ve encountered a similar bind when designing memory for AI agents.
作为人类,我们在生活中不断编码记忆。海马体在这一过程中扮演核心角色,它捕捉当下的经历,使记忆快速形成并立即可用。但只有部分经历会被巩固为长期知识。
As humans, we encode memory constantly as we go about our lives. The hippocampus plays a central role in this, capturing experiences in the moment so that memories form quickly and are immediately available. But only some experiences are consolidated into long-term knowledge.
这种巩固大多似乎发生在大脑离线时。在睡眠和安静休息期间,海马体会重放我们最近的经历。据认为,这会逐渐提炼这些经历,并将其整合到新皮层——即大脑中存储我们大部分所谓知识的部分。
Much of that consolidation appears to happen when our brains are offline. As we sleep, and during quiet rest, the hippocampus replays our recent experiences. This is thought to gradually distill those experiences and integrate them into the neocortex—the part of our brains that stores most of what we think of as knowledge.
智能体记忆可以采用类似的分工。智能体将大部分工作记录在日志或转录中。Letta 和 UC Berkeley 的研究人员提出了睡眠时间计算(sleep-time compute),让模型利用交互之间的空闲时间重新组织其现有上下文。同样的思路可以帮助智能体通过反思日志来组织记忆,以修正错误或生成新的笔记和表示。这一过程通常被称为** dreaming **(图 1)。
Agent memory can use a similar division of labor. Agents capture much of their work in logs or transcripts. Researchers at Letta and UC Berkeley proposed sleep-time compute as a way for models to use idle time between interactions to reorganize their existing context. The same idea can help agents organize their memory by reflecting on logs to fix mistakes or produce new notes and representations. This process is often called _dreaming_ (Figure 1).
我们发现,dreaming 是一种有用的工具,能帮助我们为 Claude 赋予有效的记忆。在本文中,我们分享我们对智能体记忆的思考方式,并逐一介绍我们用来探索智能体记忆的若干评估,包括 dreaming 的使用,以及每个组件对记忆组织的贡献。
We've found that dreaming has been a useful tool for helping us give Claude effective memory. In this post, we share how we've come to think about agent memory and walk through several evaluations we've used to explore agent memory, including the use of dreaming, and what each component contributes to memory organization.
所谓“智能体记忆”,我们指的是 Claude 从经验中获得的知识——包括与用户的交互以及被分配的任务——并在会话之间保留下来,保存为它日后可以读取的上下文。它不包括模型学习或管理上下文的其他方式,例如对其 _权重_(训练期间设定的内部参数)的更改、_上下文内学习_(在单次对话中获得的知识),以及 _压缩_(由 Claude 撰写的摘要,用以替换其较早的轮次,使其能够在新的上下文窗口中继续工作)。
By "agent memory," we mean knowledge that Claude gains from experience—including interactions with users and tasks it's been assigned—and that it retains between sessions, saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their _weights_ (the internal parameters set during training), _in-context learning_ (knowledge gained within a single conversation), and _compaction_ (summaries written by Claude that replace its older turns so it can continue working in a new context window).
在人脑中,在某一经历之前、之中或之后释放的化学信号可以将一段记忆标记为重要。被标记的记忆在离线时段往往比未被标记的记忆更频繁地被重放。这一过程称为 _重放_,有助于将其稳定并整合到长期记忆中。
In the human brain, chemical signals released before, during, or after an experience can mark a memory as important. Marked memories tend to be replayed more often than unmarked ones during offline periods. This process, called _replay_, helps stabilize and integrate them into long-term memory.
类似地,Claude 可以在会话日志中实时标记显著事件,以便日后更容易找到它们,或者将其写入记忆存储,例如一个记忆笔记文件夹(图 2)。但 Claude 如何在当下、在它知道接下来会发生什么之前,判断什么是显著的?
Similarly, Claude can tag salient events in a session log as they happen, so they're easier to find later, or it can write them to a memory store, such as a folder of memory notes (Figure 2). But how can Claude tell what's salient in the moment, before it knows what's coming next?
记忆有时是一种预测。通过写下一条规则,智能体预测它所描述的模式将再次变得相关。例如,写下一条经过验证的事实,就是预测未来某个会话中的智能体将希望依赖该事实而无需再次核实。随着时间的推移,Claude 在做出这类预测方面已经变得好得多。
Memory is, occasionally, a prediction. By writing down a rule, an agent predicts that the pattern it's describing will be relevant again. For instance, writing down a verified fact is a prediction that an agent in a future session will want to rely on that fact without double-checking it. Over time, Claude has become much better at making these kinds of predictions.
考虑 Continual Learning Bench(CL-Bench)1.0 中的一项任务,该基准衡量智能体利用记忆来处理长时程任务的能力。其中一项特定任务要求 Claude 通过在数据库中查找信息来回答一系列问题。在 Claude 给出答案后,它会被告知正确的解决方案,并且可以写下一段记忆来帮助回答未来的问题。
Consider a task from Continual Learning Bench (CL-Bench) 1.0, a benchmark that measures an agent's ability to use memory in order to tackle long-horizon tasks. One particular task asks Claude to answer a series of questions by looking up information in a database. After Claude gives an answer, it's told the correct solution and can write a memory to help answer future questions.
记忆可以在此支持一个学习循环。如果 Claude 答错了一个问题,它可以记录这次失败,提出关于出错原因的理论,检验这些理论,并将发现转化为关于未来如何回答的规则。
Memory can support a learning loop here. If Claude gets a question wrong, it can record the failure, come up with theories about what went wrong, test those theories, and turn what it finds into a rule about how to answer in the future.
我们使用 Claude Managed Agents 运行了 CL-Bench 1.0 任务。Claude 在全新的会话中开始每个问题,只有一个例外:它可以从一个跨会话持久存在的记忆目录中检索文件。
We ran the CL-Bench 1.0 task using Claude Managed Agents. Claude started each question in a fresh session, with one exception: it could retrieve files from a memory directory that persisted from one session to the next.
不同模型在创建自身学习循环方面取得了不同程度的成功。Claude Sonnet 4.6 已经能够生成关于答案为何错误的理论。其记忆目录中散落着若干文件,每个文件都列出了几个未经检验的猜测。
Different models achieved different levels of success at creating their own learning loops. Claude Sonnet 4.6 got as far as generating theories about why an answer was wrong. Its memory directory contained a scattering of files, each listing several untested guesses.
Claude Opus 4.7 将其笔记整合为一份单一的参考文档,并开始标注哪些事实已经过核查、哪些仍未确认。Claude Sonnet 5 在评分器核查之后将每个答案标记为已确认,并在答错时写下更正。
Claude Opus 4.7 consolidated its notes into a single reference document and began marking which facts it had checked and which were still unconfirmed. Claude Sonnet 5 marked each answer as confirmed after the grader had checked it, and wrote a correction when it had answered wrong.
Claude Opus 5 是这些模型中第一个闭合这一循环的。它的大多数条目都包含一条规则(以及它如何得知该规则为真)、对先前问题的回答,以及对沿途发现的陷阱的明确警告(图 3)。
Claude Opus 5 was the first of these models to close the loop. Most of its entries included a rule (and also how it knew the rule was true), answers to earlier questions, and clear warnings about traps it had discovered along the way (Figure 3).
与 Opus 5 一样,Claude Opus 5.5 和 Claude Sonnet 5.5 也都闭合了这一循环,利用记忆来捕获关于如何最好地回答未来问题的、经过验证的规则。借助记忆,两者的得分与 Opus 5 大致相同,尽管它们在无记忆时的基线表现略高一些。
Like Opus 5, Claude Opus 5.5 and Claude Sonnet 5.5 both closed the loop, using memory to capture validated rules about how to best answer future questions. With memory, both scored about the same as Opus 5, though their baseline performance without memory was slightly higher.
我们还评估了 Claude 玩《宝可梦红》的能力,这款游戏挑战 Claude 在复杂的长期任务中进行导航,以此测试 Claude 对**该**将什么保存到记忆中的判断力。在每次运行开始时,我们给每个模型一个记忆文件夹,它可以在游戏过程中读取和写入(图 4)。
We also evaluated Claude's ability to play _Pokémon Red_, a game that challenges Claude to navigate complex long-horizon tasks, as a test of Claude's judgment about _what_ to save to memory. At the start of each run, we gave each model a folder of memory files that it could read from and write to while it played (Figure 4).
我们比较了 Claude Sonnet 3.7 和 Claude Fable 5 各一次运行中约进行到 10,000 步时的记忆文件夹。我们注意到的一个关键差异是每个模型认为值得记住的内容。Sonnet 3.7 假设一切可能都重要,并随意地写入记忆。其记忆文件夹包含遭遇的存档,通常为每场战斗、每个找到的物品和每个遇到的非玩家角色分别建立单独文件。最终,Sonnet 3.7 在 225 个文件夹中写入了 413 个记忆文件,并获得了一个徽章(徽章标志着玩家在游戏中的进度)。
We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game. A key difference we noticed was what each model thought was worth remembering. Sonnet 3.7 assumed everything might matter and wrote to memory indiscriminately. Its memory folder contained an archive of encounters, often with a separate file for each battle it fought, each item it found, and each non-player character it met. Ultimately, Sonnet 3.7 wrote 413 memory files across 225 folders and earned one badge (badges mark a player's progress through the game).
相比之下,Fable 5 在两个文件夹中写入了九个文件,外加一个名为 global.md 的单一顶级文件,并获得了四个徽章。
Fable 5, by comparison, wrote nine files in two folders, plus a single top-level file called global.md, and earned four badges.
Sonnet 3.7 的方法主要问题在于宝可梦的游戏状态不断变化——Sonnet 3.7 的详细记录不断过时。结果,随着 Claude 的进展,它不得不花费更多时间和资源梳理记忆,而不是玩游戏。例如,在第 10,000 步左右,Sonnet 3.7 启动了第四个“主”记忆文件夹。在接下来的 17 步中,它创建了六个文件,删除了九个其他文件,并且没有在游戏本身中采取任何行动。
Sonnet 3.7's approach was a problem mainly because the game state of Pokémon changes constantly—Sonnet 3.7's detailed record was continuously going out of date. As a result, as Claude progressed, it had to spend more time and resources combing through its memories rather than playing the game. Around step 10,000, for example, Sonnet 3.7 started its fourth folder of “master” memory files. Over the next 17 steps, it created six files, deleted nine others, and took no actions in the game itself.
Fable 5 的方法将记忆用于游戏没有明确告知它的事情。Sonnet 3.7 写入记忆的某些类型的信息——例如玩家有多少钱或玩家在地图上的位置——每回合都可以从屏幕上读取,因此存储它是多余的。相比之下,关于策略、计划和经过测试的规则的记忆文件只有在 Claude 制定出来时才会存在,并且除非 Claude 写下来,否则它们会消失。
Fable 5's approach used memory for things the game didn't explicitly tell it. Some of the kinds of information Sonnet 3.7 wrote to memory—such as how much money the player has or where the player is on the map—can be read off the screen every turn, so storing it is redundant. By contrast, memory files about strategy, plans, and tested rules exist only if Claude works them out, and they disappear unless Claude writes them down.
Fable 5 相应地组织了它的记忆。它的 global.md 文件充当仪表板,保存当前目标、预先编写的攻克道馆(赢得徽章的挑战)的计划,以及模型自身使用记忆的规则。locations 文件夹保存了导航特定路线的计划,gameplay 文件夹保存了基于它对游戏运作方式所学到的策略。例如,在连续七个精灵球未能抓住鲤鱼王(最容易捕捉的宝可梦之一)后,Claude 写道:“如果连续 3 个以上球失败,改变时机(在菜单步骤之间插入额外的‘等待’按键)——这样做之后立刻抓到了。”
Fable 5 organized its memory accordingly. Its global.md file acted as a dashboard, holding the current goal, a prewritten plan for tackling gyms (the challenges where badges are won), and the model's own rules for using memory. A locations folder held plans for navigating particular routes, and a gameplay folder held strategies based on what it had learned about how the game works. For example, after seven Poké Balls in a row failed to catch a Magikarp (one of the easiest Pokémon to catch), Claude wrote, "If 3+ balls fail in a row, VARY the timing (insert extra 'wait' presses between menu steps) - the catch landed right after doing this."
凭借小得多的记忆文件夹,Fable 5 在游戏中比 Sonnet 3.7 进展得更远。Opus 5.5 采取了类似的方法:它保留了一个小的记忆笔记文件夹,跟踪其当前目标和下一步行动,在游戏过程中重写它们,并通关了游戏。
With a much smaller memory folder, Fable 5 progressed much further in the game than Sonnet 3.7. Opus 5.5 took a similar approach: it kept a small folder of memory notes that tracked its current goal and next steps, rewrote them as it played, and beat the game.
我们发现,告诉 Claude 记忆的用途是有帮助的,但不要告诉它_如何_使用记忆。在 CL-Bench 上,我们只是告诉 Claude 有一个可用的记忆文件夹。这一通用指引就足以让 Opus 4.7 和 Opus 5 有效使用记忆,并且为两者都带来了明显的性能提升(图 3)。
We've found that it helps to tell Claude what memory is for, but not _how_ to use it. On CL-Bench, we simply told Claude that a memory folder was available. That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively, and it gave both a noticeable boost in performance (Figure 3).
我们还测试了向模型提供关于应向记忆写入什么内容的具体规则,每个测试对应一条规则。我们比较了每个模型在每条规则下从记忆中获得的增益与没有该规则时获得的增益。当我们把规则设为“只记录失败”时,Opus 5 和 Opus 4.7 都失去了记忆带给它们的大部分性能提升(图 5,右)。对于更强的模型,让它们自由判断显著性比预先设定固定规则更有效,而这些规则最终可能会削弱性能。
We also tested giving the models specific rules about what they should write to memory, with one rule per test. We compared how much each model gained from memory with each rule against how much it gained without it. When we set a rule to "record only failures," both Opus 5 and Opus 4.7 lost much of the performance boost that memory gave them (Figure 5, right). For stronger models, giving them freedom to judge salience was more effective than setting fixed rules in advance, and those rules could end up weakening performance.
我们还注意到,引导对较旧的模型帮助最大。例如,Sonnet 4.6 经常不读取其记忆目录中的文件,也未能检验其关于某个答案为何错误的理论。
We also noticed that steering helped older models the most. For instance, Sonnet 4.6 often didn't read files from its memory directory and failed to test its theories about why an answer was wrong.
在这种情况下,提示 Claude 在回答前读取其记忆文件,然后明确检验其理论,几乎使其基准分数翻倍(图 5,左)。Sonnet 5 会自行这样做。在未被提示的情况下,它在 10 次中有 9 次在回答前读取了记忆。相比之下,Sonnet 4.6 这样做的次数不到一半。
In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score (Figure 5, left). Sonnet 5 does this on its own. Without being prompted to, it read its memory before answering 9 out of 10 times. By comparison, Sonnet 4.6 did this less than half of the time.
对于前沿 Claude 模型,我们通常给出简单指引:当有人要求它们创建记忆笔记,或当某些内容明显值得记住时,就创建记忆笔记,而我们将这一判断留给 Claude 自己。
With frontier Claude models, we typically give simple guidance to create memory notes when a person asks them to or when something is clearly worth remembering, and we leave that judgment up to Claude.
由于 Claude 是根据它在会话中判断为显著的内容来写入记忆的,因此一条从一开始就错误、或随着环境变化而漂移的笔记,可能会在之后的会话中被 Claude 读取并信任。
Since Claude writes to memory based on what it judges to be salient in-session, a note that's wrong from the start or that drifts as the environment changes can be read, and trusted, by Claude in a later session.
Dreaming 为 Claude 提供了一种将笔记与原始对话记录进行比对检验的方法(图 6)。我们在 Claude 使用 Managed Agents 并通过挂载目录存放其记忆来玩 Pokémon 时,看到了一个正面例子。
Dreaming gives Claude a way to test its notes against the original transcripts (Figure 6). We saw a positive example of this when Claude played Pokémon using Managed Agents with a mounted directory for its memories.
在游戏的一个环节中,玩家必须穿过月见山(Mt. Moon),这是一个洞穴系统,入口位于地图西侧,出口位于东侧。入口附近有一个陷阱:一处单向的岩架,会把玩家丢落到他们来时的路线上,且没有明确的办法爬回去。Claude 把宝可梦中心(PokéCenter)——位于山入口处的一栋建筑——当作地标,因此它是否会走下岩架,取决于它认为宝可梦中心位于山的哪一侧。
In one part of the game, the player has to get through Mt. Moon, a cave system with its entrance on the west side of the map and its exit on the east. Near the entrance is a trap: a one-way ledge that drops the player onto the route they came from, with no clear way to climb back up. Claude was using the PokéCenter, a building at the mountain's entrance, as a landmark, so whether it walked off the ledge depended on which side of the mountain it thought the PokéCenter was on.
有一次,Claude(错误地)记下宝可梦中心位于东侧。当它走出宝可梦中心时,它宣称“月见山在我身后”,并更新了笔记:“月见山 完成。”但月见山并不在它身后。几步操作之后,Claude 走下了岩架,落回到它来时的路线上(图 7)。它的导航工具没有显示这次落差,于是 Claude 得出结论:“我向西走过头了”,并认为自己一直就在正确的一侧。事实并非如此。
At one point, Claude wrote down (incorrectly) that the PokéCenter was on the east side. When it walked out of the PokéCenter, it announced, "Mt Moon is behind me," and updated its notes: "Mt Moon DONE." But Mt. Moon was not behind it. A few actions later, Claude went over the ledge and landed back on the route it had come from (Figure 7). Its navigation tool didn't show the drop, so Claude concluded, "I overshot far west," and that it had been on the correct side all along. It hadn't.
随后我们测试了 dreaming 能否帮助 Claude 避开这个陷阱。我们在包含错误宝可梦中心位置的记忆文件夹上运行了 dreaming:Claude 将笔记与会话记录进行核对,发现了自己的错误,并重写了它。会话记录为 Claude 提供了一个可用于验证其记忆的真实基准。凭借修正后的记忆笔记,Claude 径直回到月见山,并最终抵达了华蓝市(Cerulean City)。
We then tested whether dreaming could help Claude avoid this trap. We ran dreaming on the memory folder that contained the wrong location for the PokéCenter: Claude checked the note against the session transcript, caught its error, and rewrote it. The transcript gave Claude a ground truth to validate its memory against. With the corrected memory note, Claude went straight back into Mt. Moon and eventually reached Cerulean City.
在未修正笔记的那次运行中,Claude 三次退回到那条更早的路线上。第三次失败后,Claude 似乎放弃了,写道:“我打算停下来,而不是继续循环。”我们设置的自动回复指示它继续。于是它照做了,一路退回到山前的小镇 Pewter City,希望在那里治疗并补充物资。这次运行结束时,Claude 始终未能抵达 Cerulean City。
In the run with the uncorrected note, Claude dropped back onto that earlier route three times. After the third failure, Claude seemed to give up, writing, "I'm going to stop rather than keep looping." Our setup's automatic reply instructed it to continue. So it did, all the way back to Pewter City, the town before the mountain, where it hoped to heal and restock. The run ended without Claude ever reaching Cerulean City.
借助 dreaming,Claude 能够构建表示以帮助导航记忆(图 9)。它使用的一种记忆表示称为 _wiki_,即由 Claude 编写和维护的页面集合,这些页面汇集了 Claude 关于某一主题的知识,并将其与其他笔记链接起来。这些链接实际上为 Claude 提供了一张地图,它可以用这张地图来筛选记忆,并在需要时高效地定位记忆。
With dreaming, Claude can build representations to help navigate memory (Figure 9). One type of memory representation it uses is called a _wiki_, a collection of pages written and maintained by Claude that gathers what Claude knows about one subject and links it to other notes. These links effectively provide Claude with a map it can use to sift through its memory and efficiently locate memories when they’re needed.
像 wiki 这样的表示可以帮助 Claude 遍历其不断增长的知识库。我们使用来自 Stainless(我们于五月收购的一家软件公司)的内部文档语料库——约 35 万个文件,约 4.2 亿个 token——探索了 wiki 作为记忆表示的好处。我们针对该语料库创建了一个包含 25 个问题的评估集,并给 Claude 提供了一个搜索工具来探索它。我们还指示 Claude 使用 dreaming 基于这些文件构建一个 wiki。在 128 个会话中,我们提示 Opus 5 智能体探索文件、编写或更正链接的 wiki 页面、合并重复项,并对照来源检查声明。
Representations like wikis can help Claude traverse its ever-growing knowledge base. We explored the benefit of wikis as a memory representation using a corpus of internal documents—about 350,000 files, roughly 420M tokens—from Stainless, a software company we acquired in May. We created an evaluation set of 25 questions about this corpus and gave Claude a search tool to explore it. We also instructed Claude to build a wiki based on these files using dreaming. Over 128 sessions, we prompted Opus 5 agents to explore the files, write or correct linked wiki pages, merge duplicates, and check claims against the sources.
每个会话都建立在智能体在先前会话中取得的进展之上,使用共享日志来协调工作。当我们将 dreaming 从两个会话扩展到 128 个会话时,25 个问题的性能从评分标准的 13% 增长到 48%(图 10)。
Each session built on the progress that agents made in prior sessions, using a shared log to coordinate work. As we scaled dreaming from two sessions to 128, performance on the 25 questions grew from 13% to 48% of graded criteria (Figure 10).
Claude 逐步构建其 wiki。在创建初始 wiki 后,它使用后续轮次的 dreaming 来添加、编辑或修剪它。在 dreaming 的早期轮次中,Claude 的知识就像新员工的知识一样,仅基于 wiki 回答问题的能力有限。它做梦越多,其知识地图就越丰富,就像经验丰富的员工一样。在我们的测试中,我们发现当我们也给 Claude 访问搜索工具的权限时,wiki 对 Claude 的帮助最大。当它同时使用搜索和 wiki 时,随着 wiki 的增长,Claude 的总分从 54% 上升到 61%,而仅使用搜索工具时为 53%。当 Claude 仅使用 wiki 回答问题时,其最佳得分为 48%。使用搜索和 wiki 还每个问题节省了约 20,000 个 token。
Claude builds its wikis incrementally. After creating an initial wiki, it uses subsequent rounds of dreaming to add to, edit, or prune it. In the early rounds of dreaming, Claude’s knowledge is like that of a new employee, and its ability to answer questions based on the wiki alone is limited. The more it dreams, the richer its knowledge map grows, like that of a seasoned employee. In our tests, we found that the wiki helped Claude the most when we also gave it access to a search tool. When it used both search and the wiki, Claude’s aggregate score rose from 54% to 61% as that wiki grew, compared with 53% when it had access to the search tool alone. Claude’s best score was 48% when it used only the wiki to answer questions. Using search and the wiki also saved about 20,000 tokens per question.
我们想测试哪些类型的任务受益于 Claude 拥有 wiki。为此,我们使用了 Harvey LAB 基准测试以及一组关于合并的文档。该基准测试给 Claude 一组文档,并要求 Claude 根据其内容生成一份法律摘要。然后,基准测试根据一系列标准对摘要进行评分。虽然可能比 Pokémon 运行稍逊吸引力,但这些评估为我们提供了多样化的文档集合来构建 wiki。
We wanted to test which kinds of tasks benefited from Claude having a wiki. To do this, we used the Harvey LAB benchmark along with a collection of documents about a merger. The benchmark gives Claude a set of documents and asks Claude to generate a legal brief based on their content. The benchmark then grades the brief against a list of criteria. While a tad less engaging, perhaps, than a Pokémon run, these evaluations give us diverse collections of documents to build wikis from.
我们指示 Claude 根据这些文档构建一个 wiki。然后,我们要求 Claude 从 wiki 和直接文件搜索两种方式回答(关于合并的)问题或生成法律摘要(LAB 基准)。一项测试提出了 24 个关于合并的问题(图 11 左),其中超过一半需要 Claude 跨多个文档查找信息,许多还需要 Claude 运用判断力或进行数学计算。在这些项目上(红色和橙色线),Claude 查阅 wiki 时得分为 87%,而仅使用搜索时为 73%。
We instructed Claude to build a wiki from the documents. We then asked Claude to answer questions (about the merger) or generate legal briefs (the LAB benchmark) both from the wiki and from a direct file search. One test asked 24 questions about the merger (Figure 11, left), more than half of which needed Claude to find information across several documents, and many of which also needed Claude to use judgment or do math. On those items (red and orange lines), Claude scored 87% when it consulted the wiki, compared with 73% when it used search alone.
在其他情况下,wiki 的用处较小。在一项 LAB 任务中(图 11 右),摘要标准通常可以通过对单个文档进行文本搜索来检索,而 wiki 相对于搜索并未显示出明显的性能提升。在另一项 LAB 任务中,wiki 变得比文档本身还大(图 12)。就像博尔赫斯的一篇短篇小说《论科学的精确性》中的制图师一样,他们绘制的地图如此详细,以至于和它所代表的帝国一样大,Claude 制作了一个极其详细的 wiki,实际上没有抽象出任何东西。
In other cases, the wiki was less useful. On one LAB task (Figure 11, right), brief criteria often could be retrieved with a text search from a single document, and the wiki showed no clear gain in performance relative to search. On another LAB task, the wiki grew larger than the documents themselves (Figure 12). Like the mapmakers in Borges’s one-paragraph short story “On Exactitude in Science,” who drafted a map so detailed it was as large as the empire it represented, Claude had made an extremely detailed wiki that effectively abstracted nothing.
Dreaming 还能将笔记上浮至记忆层级的顶端,智能体框架(agent harness)可在每次会话开始时预先将其加载到 Claude 的上下文中(图 13)。这意味着 Claude 无需搜索这些笔记,甚至无需知道存在可供搜索的内容。
Dreaming can also float notes to the top of the memory hierarchy, and the agent harness can preemptively load them into Claude's context at the start of every session (Figure 13). This means Claude doesn't need to search for them, or even to know there's something to search for.
将记忆提升到新会话中(一种 _上下文注入_ 形式)必须谨慎进行。注入的记忆会消耗宝贵的 token,而一条错误的笔记可能被带入每一次会话。加载笔记也并不保证 Claude 会据此行动:我们关于 Claude Code 的文档警告称,较长的指令文件会降低遵循度。因此,记忆层级的顶层必须保持相对较小,而 dreaming 在此处的大部分工作正是维持这一状态——通过将部分细节下移至层级更低处并留下链接。
Promoting memories into a new session (a form of _context injection_) must be done carefully. Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn't guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence. So the top layer of the memory hierarchy has to stay relatively small, and much of dreaming's work here is keeping it that way—by moving some details down the hierarchy and leaving links behind.
Claude Code 的自动记忆以同样的方式保持其注入层较小,由 Claude 在会话期间进行维护。Claude 不会将所有自动记忆都加载到每次会话的上下文窗口中,而只加载 MEMORY.md 的前 200 行或 25 KB。该头部充当索引,向 Claude 展示它所知道的内容以及在哪里可以找到这些内容。
Claude Code's auto memory keeps its injected layer small in the same way, with Claude maintaining it during the session. Claude doesn't load all of its auto memory into every session's context window. It loads only the first 200 lines or 25 KB of MEMORY.md. This header serves as an index, which shows Claude what it knows and where to find it.
Dreaming 也能塑造记忆的增长方式。例如,像 Claude Duet 这样的助手,通常受益于围绕其帮助对象组织起来的记忆,而 dreaming 可以被调整以构建服务于特定目标的记忆。
Dreaming can also shape how memory grows. An assistant like Claude Duet, for example, often benefits from having memories that are organized around the person it’s helping, and dreaming can be tuned to build memory in service of a particular goal.
指导记忆如何增长的指令因任务而异,并告诉 Claude 哪些信息真正重要。例如,LAB 基准中的一个任务模拟公司收购并提供一组合同。简报的评分标准是它是否正确描述了合同条款。
The instructions that guide how memory grows differ by task and tell Claude what information actually matters. For example, one task in the LAB benchmark simulates a company acquisition and provides a set of contracts. The brief is graded on whether it correctly describes the contract terms.
我们最初用于生成 wiki 的基本指令没有告诉 Claude **如何**使用 wiki。这个任务严重依赖合同条款。在构建 wiki 时,多个智能体同时写入一个共享的合同条款表(图 14 左)。
We started with basic instructions for generating the wiki that didn’t tell Claude _how_ to use the wiki. This task relied heavily on contract terms. While the wiki was being built, several agents wrote to a shared table of contract terms at the same time (Figure 14, left).
结果是一个缺少关键事实的 wiki。当我们对从 wiki 生成的简报进行评分时,Claude 只包含了 57 个事实中的 41 个(72%)。当我们查看出错原因时,发现共享表对合同条款的总结很差,并且完全遗漏了其中一些条款,因为智能体互相覆盖了对方的工作。
The result was a wiki that was missing key facts. When we graded a brief generated from the wiki, Claude had included only 41 of the 57 facts (72%). When we looked at what had gone wrong, we found that the shared table summarized the contract terms poorly and was missing some of them altogether, because agents had overwritten one another’s work.
合同条款是 wiki 中最重要的部分,但指令没有说明这一点。为了找到更好的指令,我们运行了一个循环,其中 Claude 反思其失败,提出对指令的修改,并重建 wiki。这个循环让 Claude 发现遵循合同条款的重要性,并相应地重写了构建 wiki 的指令。
The contract terms were the most important part of the wiki, but the instructions didn’t say that. To find better instructions, we ran a loop in which Claude reflected on its failures, proposed a change to the instructions, and rebuilt the wiki. This loop allowed Claude to discover how important it was to follow the contract terms, and it rewrote the instructions for building the wiki accordingly.
在 wiki 中,每份合同都拥有独立页面,由单个智能体撰写,逐字引用条款原文,而非将其概括进一张共享表格。当我们对基于这一改进后 wiki 生成的简报进行评分时,100% 的评分标准都能得到满足,较之前的 72% 有所提升。
Each contract got its own page in the wiki, written by a single agent, which quoted the terms word for word rather than summarizing them into a shared table. When we graded the brief generated from this improved wiki, 100% of the criteria could be met, up from 72%.
当 Claude 撰写记忆笔记时,它只能看到*该*特定会话中的上下文。Dreaming 可以跨多个会话进行搜索,以发现常见模式或纠正常见错误。
When Claude writes memory notes, it sees only the context in _that_ particular session. Dreaming can search across many sessions to find common patterns or correct common mistakes.
在为 LAB 基准构建 wiki 时(示例见图 14),Claude 有时会链接到尚未创建的页面。第二轮 dreaming 填补了其中许多空白。在一项任务中(图 15),dreaming 在 wiki 中发现了九处指向缺失页面的链接,并将它们重新指向同一文档的现有页面。它还新增了三个页面(包括一个索引页),以改进该 wiki。
When building the wikis for the LAB benchmark (as an example, see Figure 14), sometimes Claude would link to pages that hadn't yet been created. A second dreaming pass closed many of these gaps. On one task (Figure 15), dreaming found nine links to a missing page in the wiki and repointed them to an existing page for the same document. It also added three new pages, including an index, to improve the wiki.
关于如何为智能体设计记忆系统,仍有许多开放问题。其中之一是 Claude 应在何时创建记忆。当 Claude 处理任务时,它可以写入记忆存储(例如文件夹、文件或数据库),在源头捕捉显著性,并使重要细节可立即用于后续轮次或会话。但即时记录笔记不可避免地会把 Claude 的注意力从其主任务上引开。相比之下,dreaming 可以把记忆创建推到后台,但 dreaming 的任何输出可能无法立即被 Claude 访问。
There are many open questions about how to design memory systems for agents. One is when Claude should create memory. When Claude works on a task, it can write to a memory store (e.g., a folder, file, or database), capturing salience at the source and making important details immediately available for future turns or sessions. But logging notes in the moment inevitably draws Claude's attention away from its main task. In contrast, dreaming can push memory creation to the background, but any outputs from dreaming may not be immediately accessible to Claude.
另一个问题是如何让记忆可靠地保持最新。Claude 可能做出错误预测,或向记忆写入从一开始就错误的规则。随着环境变化,记忆也可能漂移并变得无关。Dreaming 是修正这些错误的一种方式(图 8),但它需要算力。指示 Claude 持续检查其 _所有_ 记忆可能并不实际。相反,或许重要的是将 dreaming 聚焦于可能变化的信息,或仔细调整 dreaming 的节奏(例如,基于记忆被访问的频率)。
Another question is how to keep memory reliably up to date. Claude can make incorrect predictions or write rules to memory that are wrong from the start. Memory can also drift and become irrelevant as the environment changes. Dreaming is one way to fix these mistakes (Figure 8), but it requires compute. Directing Claude to continuously check _all_ its memories might not be practical. It may be important instead to focus dreaming on information that's likely to change or to carefully tune the cadence of dreaming (e.g., based on how often memories are accessed).
将转录搜索视为 Claude 记忆的基础也是合理的。将表示(例如 wiki)与搜索结合(图 10)有时可能有效。但 Claude 需要使用算力来维护记忆表示,并确保它们不会累积错误或幻觉。
It's also reasonable to consider transcript search as the foundation of Claude's memory. Combining representations (e.g., wikis) with search (Figure 10) can sometimes be effective. But Claude needs to use compute to maintain memory representations and make sure they don't accumulate errors or hallucinations.
尽管 Claude 对显著性的判断已经改善,但没有模型能真正知道未来轮次会发生什么。Dreaming 让 Claude 有机会 _回望_,并利用算力来组织其记忆、修正错误,并维护使未来会话更容易访问记忆的表示。随着 Claude 能力增强,记忆系统将不断演进,而关于设计 Claude 的记忆系统,我们仍有很多要学习。
Even though Claude's judgment about salience has improved, no model can really know what will happen in future turns. Dreaming gives Claude an opportunity to look _backward_ and use compute to organize its memory, correct mistakes, and maintain representations that allow future sessions to access memory more readily. Memory systems will evolve as Claude gets more capable, and we still have a lot to learn about designing Claude's memory systems.
参见我们的 cookbook,了解如何开始使用 dreaming 和知识 wiki。
See our cookbooks for getting started with dreaming and knowledge wikis.