Large language models trained for safety and harmlessness remain susceptible to adversarial misuse, as evidenced by the prevalence of "jailbreak" attacks on early releases of ChatGPT that elicit undesired behavior. Going beyond recognition of the issue, we investigate why such attacks succeed and how they can be created. We hypothesize two failure modes of safety training: competing objectives and mismatched generalization. Competing objectives arise when a model's capabilities and safety goals conflict, while mismatched generalization occurs when safety training fails to generalize to a domain for which capabilities exist. We use these failure modes to guide jailbreak design and then evaluate state-of-the-art models, including OpenAI's GPT-4 and Anthropic's Claude v1.3, against both existing and newly designed attacks. We find that vulnerabilities persist despite the extensive red-teaming and safety-training efforts behind these models. Notably, new attacks utilizing our failure modes succeed on every prompt in a collection of unsafe requests from the models' red-teaming evaluation sets and outperform existing ad hoc jailbreaks. Our analysis emphasizes the need for safety-capability parity -- that safety mechanisms should be as sophisticated as the underlying model -- and argues against the idea that scaling alone can resolve these safety failure modes.
核心贡献 · Key contributions
识别出大语言模型安全训练的两个失败模式：竞争目标和错配泛化。 Identifies two failure modes of LLM safety training: competing objectives and mismatched generalization.
证明这些失败模式可被利用来创建有效的越狱攻击。 Demonstrates that these failure modes can be exploited to create effective jailbreak attacks.
表明 GPT-4 和 Claude v1.3 等最先进模型仍然容易受到越狱攻击。 Shows that state-of-the-art models like GPT-4 and Claude v1.3 remain vulnerable to jailbreaks.
提出安全-能力对等：安全机制必须与底层模型一样复杂。 Proposes safety-capability parity: safety mechanisms must be as sophisticated as the underlying model.
论证仅靠 Scaling（规模扩张）无法解决这些安全失败模式。 Argues that scaling alone cannot resolve these safety failure modes.
局限 · Limitations
仅限于黑盒访问；白盒攻击可能揭示更多漏洞。 Limited to black-box access; white-box attacks may reveal additional vulnerabilities.
仅对英文提示进行评估；对其他语言的泛化性未经测试。 Evaluation only on English prompts; generalization to other languages is untested.
攻击可能无法迁移到具有不同训练范式或架构的模型。 Attacks may not transfer to models with different training paradigms or architectures.
研究侧重于绕过拒绝机制；未涉及偏见或毒性等其他安全问题。 Study focuses on refusal bypass; other safety aspects like bias or toxicity are not addressed.
专有模型限制了可重复性和直接假设验证。 Proprietary models limit reproducibility and direct hypothesis confirmation.
论文章节 · Sections（共 15）
摘要Abstract
1 引言1 Introduction
1.1 相关工作1.1 Related Work
2 背景：安全训练的语言模型与越狱攻击2 Background: Safety-Trained Language Models and Jailbreak Attacks
2.1 受限行为的越狱攻击2.1 Jailbreak Attacks on Restricted Behaviors
2.2 评估安全训练的语言模型2.2 Evaluating Safety-trained Language Models
3 失败模式：竞争性目标与泛化失配3 Failure Modes: Competing Objectives and Generalization Mismatch
3.1 竞争目标3.1 Competing Objectives
3.2 不匹配的泛化3.2 Mismatched Generalization
4 越狱方法的实证评估4 Empirical Evaluation of Jailbreak Methods