开发安全且有用的通用人工智能系统需要我们在可扩展监督问题上取得进展，即监督可能在大多数相关技能上超越我们的系统。由于目前尚无广泛超越人类能力的系统，该问题的实证研究并不直接。本文讨论了我们思考这一问题的主要方式之一，重点关注如何通过实验进行研究。我们首先提出一个实验设计，其核心是那些人类专家能够成功但未经辅助的人类和当前通用 AI 系统失败的任务。接着，我们展示了一个概念验证实验，旨在证明该实验设计的关键特征，并通过两个问答任务（MMLU 和限时 QuALITY）展示其可行性。在这些任务中，我们发现通过聊天与不可靠的大型语言模型对话助手交互的人类参与者——这是一种可扩展监督的简单基线策略——其表现显著优于单独模型和未经辅助的人类自身。这些结果令人鼓舞，表明可扩展监督问题可以用现有模型进行研究，并支持了近期关于大型语言模型能有效协助人类完成困难任务的发现。
Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand. Empirical work on this problem is not straightforward, since we do not yet have systems that broadly exceed our abilities. This paper discusses one of the major ways we think about this problem, with a focus on ways it can be studied empirically. We first present an experimental design centered on tasks for which human specialists succeed but unaided humans and current general AI systems fail. We then present a proof-of-concept experiment meant to demonstrate a key feature of this experimental design and show its viability with two question-answering tasks: MMLU and time-limited QuALITY. On these tasks, we find that human participants who interact with an unreliable large-language-model dialog assistant through chat -- a trivial baseline strategy for scalable oversight -- substantially outperform both the model alone and their own unaided performance. These results are an encouraging sign that scalable oversight will be tractable to study with present models and bolster recent findings that large language models can productively assist humans with difficult tasks.
核心贡献 · Key contributions
提出了一种夹层实验范式，用于评估当前模型的可扩展监督技术。 Introduces a sandwiching experimental paradigm for evaluating scalable oversight techniques with present-day models.
证明人机团队在 MMLU 和限时 QuALITY 上的表现优于单独模型和未辅助人类。 Demonstrates that human–model teams outperform both the model alone and unaided humans on MMLU and time-limited QuALITY.
提供了人类与语言模型交互的有效策略的定性见解，如交叉验证和事实核查。 Provides qualitative insights into effective human strategies for interacting with language models, such as cross-examination and fact-checking.
表明与不可靠的大语言模型助手的简单聊天交互可作为可扩展监督的基线。 Shows that simple chat-based interaction with an unreliable LLM assistant can serve as a baseline for scalable oversight.
确立了现有 NLP 数据集可用于有效评估可扩展监督技术。 Establishes that existing NLP datasets can be used to productively evaluate scalable oversight techniques.
概述了可扩展监督的研究路线图，包括辩论、做市和递归奖励建模。 Outlines a research roadmap for scalable oversight, including debate, market-making, and recursive reward modeling.
局限 · Limitations
依赖多项选择题，比现实世界的监督任务更简单。 Relies on multiple-choice questions, which are simpler than real-world oversight tasks.
在每个问题后向参与者揭示正确答案，降低了识别可靠方法的难度。 Reveals correct answers to participants after each question, reducing the difficulty of identifying reliable methods.
排除了两名表现优异的异常参与者，可能导致结果偏差。 Excludes two high-performing outlier participants, potentially biasing results.
未测试除对话交互之外的微调或其他修改。 Does not test fine-tuning or other modifications beyond dialog interaction.
人类参与者有时因领域知识有限而接受模型自信的错误主张。 Human participants sometimes accept false confident claims from the model due to limited domain knowledge.
论文章节 · Sections（共 12）
摘要Abstract
1 引言1 Introduction
2 可扩展监督的研究范式2 A Research Paradigm for Scalable Oversight