虽然大型语言模型（LLM）在语言理解和交互式决策任务中展现了令人印象深刻的能力，但其推理（如思维链提示）和行动（如行动计划生成）能力主要被作为独立主题研究。在本文中，我们探索了以交错方式使用 LLM 生成推理轨迹和任务特定行动，从而允许两者之间更大的协同作用：推理轨迹帮助模型诱导、跟踪和更新行动计划以及处理异常，而行动则允许模型与外部来源（如知识库或环境）交互以收集额外信息。我们将我们的方法命名为 ReAct，并将其应用于多种语言和决策任务，证明了其相对于最先进基线的有效性，以及相对于没有推理或行动组件的方法，提高了人类可解释性和可信度。具体来说，在问答（HotpotQA）和事实验证（Fever）任务中，ReAct 通过与简单的 Wikipedia API 交互，克服了思维链推理中常见的幻觉和错误传播问题，并生成了比没有推理轨迹的基线更可解释的类人任务解决轨迹。在两个交互式决策基准（ALFWorld 和 WebShop）上，ReAct 仅通过一个或两个上下文示例进行提示，就分别以 34%和 10%的绝对成功率优于模仿学习和强化学习方法。项目网站及代码：https://react-lm.github.io
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines, as well as improved human interpretability and trustworthiness over methods without reasoning or acting components. Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples. Project site with code: https://react-lm.github.io
核心贡献 · Key contributions
提出 ReAct，一种基于提示的范式，在 LLM 中协同推理与行动以解决通用任务。 Introduces ReAct, a prompt-based paradigm that synergizes reasoning and acting in LLMs for general task solving.
在多个基准上展示 ReAct 的有效性，在少样本设置中优于仅推理或仅行动的先前方法。 Demonstrates ReAct's effectiveness across diverse benchmarks, outperforming prior reasoning-only or acting-only methods in few-shot settings.
通过系统消融实验，展示推理任务中行动的重要性以及交互任务中推理的重要性。 Provides systematic ablations showing the importance of acting in reasoning tasks and reasoning in interactive tasks.
分析 ReAct 在提示下的局限性，并通过初步微调实验展示性能提升。 Analyzes limitations of ReAct under prompting and shows initial fine-tuning experiments that improve performance.
通过结合推理轨迹与外部行动，增强可解释性、可信度和可诊断性。 Enhances interpretability, trustworthiness, and diagnosability by combining reasoning traces with external actions.
在 ALFWorld 和 WebShop 上仅用一两个上下文示例即达到最先进结果。 Achieves state-of-the-art results on ALFWorld and WebShop with only one or two in-context examples.
局限 · Limitations
具有大动作空间的复杂任务需要更多示例，可能超出上下文学习的输入长度限制。 Complex tasks with large action spaces require more demonstrations, which may exceed input length limits of in-context learning.
ReAct 交错推理与行动的结构约束可能导致比思维链更多的推理错误。 ReAct's structural constraint of interleaving reasoning and action can lead to more reasoning errors than chain-of-thought.
非信息性搜索结果可能干扰模型推理，使其难以恢复。 Non-informative search results can derail model reasoning and make recovery difficult.
在 HotpotQA 等知识密集型任务上，由于推理灵活性降低，性能略逊于思维链。 Performance on knowledge-intensive tasks like HotpotQA slightly lags behind chain-of-thought due to reduced reasoning flexibility.
当前基于提示的方法在 WebShop 上仍远不及人类专家，尤其在产品探索和查询重构方面。 Current prompting-based method still far from expert human performance on WebShop, especially in product exploration and query reformulation.