Automated Red Teaming with GOAT: the Generative Offensive Agent Tester
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→红队测试评估大语言模型（LLM）在生成内容时是否违反安全训练中设定的规范、政策和规则。然而，现有的大多数自动化方法并不能代表人类与 AI 模型交互的典型方式。普通 AI 用户可能不具备对抗性机器学习方法的高级知识，也无法访问模型内部，且不会花费大量时间精心制作单个高效对抗性提示。相反，他们更可能利用网上常见的技巧以及 LLM 的多轮对话特性。虽然手动测试可以弥补这一差距，但效率低下且成本高昂。为解决这些限制，我们引入了生成式对抗代理测试器（GOAT），这是一个自动化代理红队测试系统，它模拟自然语言对抗对话，同时利用多种对抗性提示技术来识别 LLM 中的漏洞。我们通过提示通用模型，鼓励其对可用方法、当前目标模型的响应以及后续步骤进行推理，从而用 7 种红队攻击实例化 GOAT。我们的方法设计为可扩展且高效，允许人类测试人员专注于探索新的风险领域，而自动化则覆盖已知风险领域的规模化对抗压力测试。我们展示了 GOAT 的设计与评估，证明了其在识别最先进 LLM 漏洞方面的有效性，在 JailbreakBench 数据集上，对 Llama 3.1 的 ASR@10 达到 97%，对 GPT-4 达到 88%。
Red teaming assesses how large language models (LLMs) can produce content that violates norms, policies, and rules set during their safety training. However, most existing automated methods in the literature are not representative of the way humans tend to interact with AI models. Common users of AI models may not have advanced knowledge of adversarial machine learning methods or access to model internals, and they do not spend a lot of time crafting a single highly effective adversarial prompt. Instead, they are likely to make use of techniques commonly shared online and exploit the multiturn conversational nature of LLMs. While manual testing addresses this gap, it is an inefficient and often expensive process. To address these limitations, we introduce the Generative Offensive Agent Tester (GOAT), an automated agentic red teaming system that simulates plain language adversarial conversations while leveraging multiple adversarial prompting techniques to identify vulnerabilities in LLMs. We instantiate GOAT with 7 red teaming attacks by prompting a general-purpose model in a way that encourages reasoning through the choices of methods available, the current target model's response, and the next steps. Our approach is designed to be extensible and efficient, allowing human testers to focus on exploring new areas of risk while automation covers the scaled adversarial stress-testing of known risk territory. We present the design and evaluation of GOAT, demonstrating its effectiveness in identifying vulnerabilities in state-of-the-art LLMs, with an ASR@10 of 97% against Llama 3.1 and 88% against GPT-4 on the JailbreakBench dataset.