我们研究了一种“越狱”技术——一种可以用来规避大型语言模型（LLM）开发者设置的安全护栏的方法。这种技术，我们称之为“多轮越狱”，对 Anthropic 自己的模型以及其他 AI 公司生产的模型都有效。我们提前向其他 AI 开发者通报了这一漏洞，并已在我们的系统上实施了缓解措施。该技术利用了 LLM 在过去一年中急剧增长的一个特性：上下文窗口。2023 年初，上下文窗口——LLM 作为输入可以处理的信息量——大约相当于一篇长文章的大小（约 4000 个 token）。现在，一些模型的上下文窗口比那时大数百倍——相当于几部长篇小说的大小（100 万个 token 或更多）。输入越来越大量信息的能力对 LLM 用户有明显的好处，但也带来了风险：容易受到利用更长上下文窗口的越狱攻击。
We investigated a “jailbreaking” technique — a method that can be used to evade the safety guardrails put in place by the developers of large language models (LLMs). The technique, which we call “many-shot jailbreaking”, is effective on Anthropic’s own models, as well as those produced by other AI companies. We briefed other AI developers about this vulnerability in advance, and have implemented mitigations on our systems. The technique takes advantage of a feature of LLMs that has grown dramatically in the last year: the context window. At the start of 2023, the context window—the amount of information that an LLM can process as its input—was around the size of a long essay (~4,000 tokens). Some models now have context windows that are hundreds of times larger — the size of several long novels (1,000,000 tokens or more). The ability to input increasingly-large amounts of information has obvious advantages for LLM users, but it also comes with risks: vulnerabilities to jailbreaks that exploit the longer context window.