Claude 如何为 AI 生成的文本添加水印

令牌采样、水印检测与移除的 48 分钟视频讲解 A 48-minute video walkthrough of token sampling, watermark detection, and removal

我最近在 Substack 上发布了一篇关于 Claude 新水印流程和实现的笔记。由于这是一个非常热门的话题，并引发了热烈的讨论，我想在解释其工作原理时，或许可以更详细一些。

I recently posted a Substack note about Claude's new watermarking process and implementation. Since it's such a popular topic and sparked such a lively discussion, I thought it might be interesting to go into a bit more detail when explaining how it works.

与通常的文字文章不同，我录制了一个关于该主题的小讲座（以改变我通常文章的风格）。因此，下面是视频以及文字记录。

Instead of the usual text article, I recorded a little lecture on the topic (to change it up a bit from my usual articles). So, below is the video along with a transcript.

最初，我计划制作 10 张幻灯片并录制一个 10 分钟的短视频。然而，在整理过程中，我在这里和那里添加了一些关键细节，最终导致幻灯片超过 50 张，录制时长达到 48 分钟。

Originally, I planned to make 10 slides and record a short 10-min video. However, while putting it together, I added some crucial details here and there, resulting in >50 slides and a 48 min recording.

不过，我希望现在它能很好地解释清楚！祝观看愉快！

I hope that this now explains it well, though! Happy watching!

* 如果你更喜欢使用 YouTube 播放器，我也有 YouTube 版本。

* I also have a YouTube version if you prefer using the YouTube player

视频文字记录 Video Transcript

注：以下文字记录为便于阅读略作编辑和清理，但保留了上述视频讲座的整体顺序和脉络。

Note: The transcript below is slightly edited and cleaned up for readability but preserves the overall order and flow of the video lecture above.

Claude 文本水印的工作原理 How Claude’s Text Watermarking Works

大家好。几天前，Anthropic 宣布将对其 Claude 模型的文本输出添加水印。随后我在社交媒体上发了一篇帖子，简要解释了其工作原理。这篇帖子相当受欢迎。不过，受欢迎的并不是水印本身，而是我给出的解释或背后的机制。那么，或许值得进一步展开，更详细地解释一下，因为那篇帖子只有一张图，却引发了许多问题和讨论。

Hi everyone. So, a few days ago, Anthropic announced that they will watermark the text outputs of their Claude models. I then did a social media post briefly explaining how that works. And yeah, this was quite the popular post. So not the watermarking itself was popular, but I guess the explanation or the mechanism behind it. Then, it might be worthwhile expanding this a bit to explain it in more detail, because this post only had one figure, and there were a lot of questions and discussions.

于是我想，不妨再多画几张图。我原本计划做大约 10 张幻灯片来逐步讲解，结果最后做成了 50 张。但我希望这能真正讲清楚这种水印技术的工作原理，以及水印本身可能如何失效或被移除等等。我认为这会是个有趣的话题，因为如今很多人都在使用大语言模型（LLM），同时也在互联网上消费大量可能由 LLM 生成的文本。

So, I thought, well, let’s make a few more figures. I actually originally planned to do like 10 slides and walk you through it. It ended up being 50 slides, but I hope this really explains how this watermarking technique works well, how watermarking itself can fail or be removed, and so forth. So I think it might be an interesting topic because a lot of people use LLMs these days and also consume a lot of text on the Internet that might be generated by LLMs.

而现在有了这种水印，我猜人们会担心水印会让文本质量变差，或者这种水印到底有什么好处？它意味着什么？我认为，如果我们能更好地理解水印是什么，那将大有裨益，然后我们就能自行判断这是好事还是坏事，以及其中的利弊得失。

And now there’s going to be this watermarking, and there’s this, I guess, fear of watermarking making text worse, or what’s actually the benefit of this watermarking? And so what does it mean? And I think if we understand a bit better what watermarking is, that goes a long way, and then we can make up our own minds about whether that’s a good thing or not, and so forth, like the pros and cons.

因此，我的目标就是解释其底层机制的工作原理，以及他们将如何实现这种类型的水印——文本水印。

So, my goal here is really to explain how the underlying mechanism works and how they are going to implement this type of watermarking, text watermarking.

这也是一个很好的例子，说明从头理解事物其实非常有用。这种水印技术也很好地解释了传统模型或一般 LLM 在底层是如何工作的。所以，你可能知道我喜欢从头开始做事情。比如，我写过几本书：《从零构建大语言模型》《从零构建推理模型》。我还有一些标注为“从零开始”的文章。

It’s also a great example to illustrate why understanding things from scratch is actually quite useful. This watermarking technique is also a nice way to explain how conventional models or LLMs in general work under the hood. So yeah, you may know I like doing things from scratch. Like, I have my books: Build a Large Language Model From Scratch, Build a Reasoning Model From Scratch. I have some articles labeled from scratch.

所以，对我来说，“从零开始”通常包括编码。所以这次不是编码相关的，但从零开始编码实际上是一项非常非常有用的技术，因为它确实帮助你理解某事物是如何实现的。然后从中我们可以得出我们的理解、图表、概念，因为如果我们不真正实现事物，如果没有代码，有时确实会含糊不清。当然，你知道，正如我意识到的，并不是每个人都在从零开始编码了。

So, for me, 'from scratch' often includes coding. So this one will not be coding-related, but coding from scratch is actually a very, very useful technique because it really helps you understand how something is implemented. And then from that we can derive our understanding, figures, concepts, because if we don't really implement things, if there's no code, it's really sometimes ambiguous. And of course, you know, as I realized, not everyone is coding from scratch anymore.

就像过去，从零开始编码是我们所拥有的一切。我的意思是，只有人类在编码。如今，编码可以由大语言模型（LLM）完成。然而，这并不意味着阅读代码不再有用，因为它承载了大量信息。所以在这个水印案例中，花时间从零开始编码一个 LLM 确实让你意识到内部的采样是如何实现的。我们仍然有一些相关的代码片段。

Like back in the day, coding something from scratch was all we had. I mean, there were only humans coding. Nowadays, coding can be done by LLMs. However, that doesn't mean reading code is no longer useful, because it carries a lot of information. So in this case here with this watermarking, spending some time coding an LLM from scratch really makes you realize how this sampling inside is implemented. We still have some relevant code snippets.

然后这反过来确实帮助我们理解，哦，水印应用在这个位置，这有如此这般的后果等等。所以我认为，即使人们可能不再从零开始编码，至少不是一直如此，能够为了教育目的从零开始构建某些东西以深入理解，以及为了研究目的以透明的方式操作它（而不是隐藏在层层抽象之中），仍然是有用的。

And then that really, in turn, helps us understand, oh, the watermarking is applied at this position, and this has so-and-so consequences and so forth. So I think even though people may not be coding from scratch, at least not all the time anymore, it is still useful being able to, let's say, build something from scratch for educational purposes to understand something deeply and then also for research purposes to manipulate this in a transparent way that is not hidden away in tons of layers of abstraction.

但除此之外，我认为这里只是一个巧合的愉快关系，因为对于这个幻灯片，我实际上使用了很多来自我从零开始编码材料的图表。

But that aside, I think it's just a coincidental nice relationship here because, for this slide deck, I actually used a lot of figures from my from-scratch coding materials.

所以几天前（这是 8 月 14 日），有一篇文章《Claude 的文本水印如何工作》，还有这篇文章在这里；它就像一个屏幕录制，所以幻灯片里可以包含所有内容，但有很多细节。他们实际上更新了几次，所以最初我读的时候，它要短得多。尽管如此，它非常概念化；有这样一个概述，而且，我的意思是，里面没有一张图。

So a few days ago (this is August 14), there was this article, How Claude's Text Watermark Works, and there was this article here; it's just like a screen recording, so it can have everything in the slides, but there's plenty of detail. They updated it actually a couple of times, so originally when I read this, it was a way shorter. Still, it is very, I guess, conceptual; there's like this overview, and there's, I mean, there's not a single figure in there.

因此，我们仍然难以理解他们试图做什么。他们解释了很多关于为什么要这样做，但没有解释如何做。他们链接了一篇论文，那篇论文也非常技术性。所以我认为退一步，从头开始理解他们想通过这种水印技术实现什么是合理的。

And so it’s still somewhat difficult to understand what they’re trying to do. They explain a lot about why they’re going to do it, but they don’t explain how. They link to a paper somewhere, which is also very technical. So I think it makes sense to take a step back and start from the beginning to understand what they’re trying to implement with this watermarking technique.

顺便说一下，水印的动机是让他们能够识别如果有人发布文本，他们可以说，哦，这段文本是由我们的 Claude Opus 4.8 模型生成的，例如，这样他们就有办法判断，好的，这段文本是 AI 生成的，因为它带有这个水印。而且这个水印对用户是不可见的，只有他们才能解码并发现文本是否带有他们的水印。

The motivation for watermarking, by the way, is for them to identify if someone posts text that they can say, oh, this text was generated by our Claude Opus 4.8 model, for example, so that they have a way to tell, OK, this text is AI-generated because it carries this watermark. And this watermark is invisible to users, so only they can decode it and find out whether the text has their watermark.

为什么只有他们能做到？我们将在本视频（希望不会太长）的后面部分讨论这个问题，但一次只讨论一件事。

Why can only they do it? We will get to that later in this (hopefully not too long a video), but one thing at a time.

LLM 文本生成的工作原理 How LLM Text Generation Works

因此，我想先做一个简短的序言，解释 LLM 中文本生成的工作原理，因为基于此，我们可以更容易地理解水印技术是如何工作的，以及这实际上并不是在其之上的一个庞大、昂贵的附加物。它实际上只是常规文本生成过程中的一个小调整。

So I wanted to start with a brief prelude to explain how text generation works in LLMs, because based on that we can then more easily understand how the watermarking works and that this is actually not a huge, expensive thing on top of it. It’s really just like a minor, I guess, tweak inside the regular text generation process.

例如，当我们使用类似 ChatGPT 的工具时，假设我问一个问题：德国的首都是，然后 ChatGPT 或其他 LLM（这只是一个例子）会回答“柏林”。在这种情况下，它生成了两个词元，即“柏林”和句号。但为了简单起见，我们假设它只生成一个词元。那么下一个词元就是“柏林”词元。这个词元在内部是如何生成的呢？

So when we are using something like ChatGPT, for example, let’s say I ask the question, the capital of Germany is, and yeah, ChatGPT or other LLMs, so this is just like an example would, for example, answer “Berlin”. So here, in this case, it’s generating two tokens, like “Berlin” and the period. But for simplicity, let’s assume it’s generating one token. So the next token is the “Berlin” token. How is this token generated internally?

当我们输入类似“德国的首都是”这样的内容并收到“柏林”这样的词元时，底层到底发生了什么？在幕后究竟发生了什么？

What is happening under the hood when we type something here like the capital of Germany is and receive a token like “Berlin” back? What is actually going on there behind the scenes?

在接下来的几张幻灯片中，我想简要谈谈生成下一个词元时底层发生的事情。

So in the next couple of slides, I want to briefly talk about what happens under the hood when this next token is generated.

再次假设我们的提示是“德国的首都是”。第一步是将其转换为词元 ID。因此，分词并将其转换为词元 ID 是开始阶段的主要步骤之一。这发生在外部，不在 LLM 内部。我们只是将文本转换为词元 ID，这是一种嵌入层可以处理的格式。

So assume again that our prompt is the capital of Germany is. And the first step here is to convert this into token IDs. So tokenizing it and converting it into token IDs is one of the main steps at the beginning. This is outside. It’s not inside the LLM; it’s outside of the LLM. So we are simply converting the text into token IDs. It’s just a format that embedding layers can work with.

然后这通过 LLM 处理，LLM 给出下一个词元的得分分布。

And then this passes through the LLM. The LLM gives us a score distribution for the next token.

同样，这只是 LLM 内部工作原理的简要概述。我不涉及 LLM 本身的机制；我在其他“从零开始构建 LLM”的视频和书籍中多次讲过。重要的是，当我们生成下一个词元（例如“Berlin”）时，此时我们有一个得分分布。这就是 LLM 产生的输出。

So again, this is just a brief overview of how LLMs work internally. I’m not covering the LLM machinery itself; I’ve talked about it many times in my other From Scratch LLMs videos and books. The important part is that when we generate the next token (for example, “Berlin”), we have, at this point, a distribution of scores. This is the output produced by the LLM.

在这里，我们看的是 logit 值。这些只是从负无穷到正无穷的得分，类似于一个得分范围。这里有一个例子，范围大约从 -8 或 -9 到 20。我们可以将这些转换为概率分布，但严格来说，根据采样方式的不同，这并不是必须的。所以你可以将 logit 值视为原始得分。

Here in this case, we’re looking at logit values. These are just scores from minus infinity to plus infinity, like a range of scores. Here’s an example, ranging from about -8 or -9 to 20. We could convert these into a probability distribution, but technically, it’s not strictly necessary depending on how we sample. So you can think of the logit values as the raw scores.

原始得分覆盖整个词汇表。

The raw scores go over the entire vocabulary.

这意味着 LLM 可能生成的每一个可能的词。现在，在词汇索引中，某个值（索引位置 19,846）获得了最高分。所以我将分布展开。如果你通过 LLM 运行这个提示，你甚至可能会看到更极端的情况：所有值都非常非常非常接近零，而“Berlin”可能会高得多。

That means every possible word that the LLM could generate. Now here, in the vocabulary index, a certain value (index position 19,846) receives the highest score. So I spread out the distribution. If you would run this prompt through an LLM, you would even see something more extreme: that everything is, like, very, very, very close to zero. And “Berlin” would probably be much, much higher even.

但为了展示几个峰值，让图看起来更有趣一些，我稍微放大了并展开了分布。现在这里，“Berlin” 得分最高，因为你可以把它看作是在这样一个非常具体的提示下最可能或最合理的下一个词。其他的，比如 Hamburg 或 Munich，可能是 LLM 会猜错的。

But just to show you a few peaks here so it looks a bit more interesting, I kind of zoomed in and spread out the distribution a bit. Now here, “Berlin” is the highest score because you can think of it as the most probable or plausible next token if I have a very specific prompt like this. So the other ones, I mean, it could be something like Hamburg or Munich that the LLM might guess incorrectly.

但如今，LLM 应该相当确定 “Berlin” 是正确答案。你在这里还看到词汇索引，它覆盖了整个词汇表。如今，LLM 有大约 25 万个可能的输出词元。我这里从 19,800 截断到 19,900，因为幻灯片上空间有限。如果使用真实的 25 万词汇量，分布会非常窄，我们几乎无法分辨或看到任何东西。

But nowadays an LLM should be fairly certain that “Berlin” is the correct answer here. You are also seeing here the vocabulary index. So that’s like over the whole vocabulary. Nowadays, LLMs have like 250,000 possible tokens as output. I’m truncating it here from 19,800 to 19,900 because there’s just so much space here on this slide. If I would have a very realistic vocabulary of 250,000 words, everything would be so narrow that we would barely even be able to tell or see anything on this distribution.

这只是为了教学目的而截断的。重点是，在常规文本生成中，我们会得到这个分数分布。现在我们要做的是看最高分。

So this is just truncated for educational purposes. The important point is that in regular text generation, we get this score distribution. Now, what we do is look at the highest score.

稍后我会更详细地介绍如何选择这个分数。它不一定就是最高的那个，但为了简单起见，我们假设这里取最高分。在这种情况下，它是 19,846。

I will get into more detail later on how this is selected. So it’s not necessarily precisely the highest one, but for simplicity, assume we are taking the highest score here. And in this case, it’s 19,846.

然后这个分数被去词元化，我们得到 “Berlin”。这就是这张幻灯片上的过程：从输入提示到转换为词元 ID 和词元化，传递给 LLM，得到分数分布，得到下一个词元，再转换回文本。

And this score is then detokenized, and we get “Berlin” back. So that is the process here on this slide: from an input prompt to conversion into token IDs and tokenization, passing it to the LLM, getting this score distribution, getting the next token, and converting it back into text.

然后这段文本会被追加到输入中。因此，如果我们的问题需要多个输出词元，我们会持续进行这个循环，直到答案完整。这通常意味着 LLM 会生成一个文本结束词元，例如在这里。为了简化，我只展示了一次迭代，其中它生成一个词元。但正如我所说，它会像那样继续下去，我们将修改后的输入反馈给 LLM 进行下一轮。

And then this text is appended to the input. So if we have a question that requires multiple output tokens, we keep going in this loop until the answer is complete. That usually means that the LLM generates an end-of-text token, for example, here. For simplicity, I’m showing you only one iteration where it generates one token. But yeah, as I said, it would kind of continue like that, where we are feeding back the modified input to the LLM for the next round.

现在，我们实际上如何对这个下一个词元进行采样呢？

Now, how do we actually sample this next token here?

下一个词采样如何工作 How Next-Token Sampling Works

我简单说过，我们可以从技术上直接选择得分最高的那个。这被称为贪心解码。这是一种方法。但大多数大语言模型（LLM），比如你使用它们时，它们并不会采用贪心解码，即总是选择得分最高的那个。因为如果你用其他提示词提问，得分最高的可能并不是我们想要的，那样的话模型就会记住训练数据。

I briefly said, well, we could just technically select the highest one, the one with the highest score. This is called greedy decoding. That’s one way to do it. But most LLMs, like if you use them, they don’t do greedy decoding where they always pick the highest one. Because if you ask it on some other prompt, it might not be what we want to always have the highest score, because then it would memorize the training data.

它总是会给出类似的回答等等。所以我们实际上常常希望输出有一些变化，但又不能太多以至于生成随机内容。其工作原理是，当我们从这个分布中采样时，首先通常将其转换为概率分数。

It would always kind of give the same response and so forth. So we actually often want some variation in the outputs, but not so much that it generates random stuff. So how it works is that, when we sample here from this distribution, we first typically convert it into probability scores.

这里我在图中展示了这些分数。为了简单起见，我使用 NumPy；无论你使用什么工具（例如 PyTorch），概念都是相同的。但我们假设这里用 NumPy 表示分数。那么我要做的就是计算 softmax。技术上，我会使用例如 Torch 或 PyTorch 中实现的 softmax 函数，该函数对于大值和小值都是数值稳定的，包括非常高的正值、非常低的正值和非常高的负值。

So here I just have these scores shown in this plot. I’m just using NumPy for simplicity; whatever tool you use (e.g., PyTorch), the same concepts apply. But let’s assume we have the scores here in NumPy. So what I would do is I would compute the softmax. Technically, I would use a softmax function implemented in Torch or PyTorch, for example, that is numerically stable for both large and small values, including very high positive values, very low positive values, and very high negative values.

这里我只是这样写出来。这是标准的 softmax，只是为了更易读。但细节在这里并不重要。

Here I’m just writing it out like that. That’s the canonical softmax, just to make it a bit more readable. But the details don’t matter here.

重要的是，经过这种转换后，这里的分数——我的意思是幻灯片上空间有限——这些分数加起来等于 1。所以这本质上是一种重新归一化。它们会被归一化使得总和为 1。这就是概率转换所做的全部：softmax 转换。那么一旦我们有了这些概率，我们就可以使用随机数或随机采样算法。

What matters is that after this conversion, the scores here, I mean, there’s only so much space on the slide, but the scores here, they would add up to one. So it’s essentially like a renormalization. So they would be normalized to sum up to one. That’s all that the probability conversion does: the softmax conversion. So then once we have these probabilities, we can use a random number or, like, a random sampling algorithm.

例如，在 NumPy 中，我们可以使用 choice 函数或方法。我们传入一个特定的随机种子到词汇索引中，然后——这是关键部分——我们将概率作为权重传入。所以，这些本质上就是：“某个 token 被选中的可能性有多大？”例如，如果“Berlin”在归一化步骤（即 softmax 步骤）之后有 99% 的概率，而其他 token 合计只有 1% 的概率，那么如果我们采样 100 次，其中 99 次会得到“Berlin”。

For example, here in NumPy, we could use the choice function or method. So this is with a specific random seed we are passing to the vocabulary indices. And then, and that’s the important part, we are passing the probabilities as the weights. So, these, essentially, yeah, are like: “How likely is a certain token to be selected?” So, for example, if “Berlin”, after this normalization step, the softmax step, has a 99% probability and the other ones together have a 1% probability, then if we would sample 100 times, 99 of the times, we would get “Berlin”.

在现实中的 LLM（例如训练良好的模型）中，“Berlin”的概率可能高达 99.999999% 左右。因此，你几乎总是会采样到“Berlin”，因为模型非常确信在这种情况下答案是“Berlin”。所以，这就是我们如何从这个分布中采样的。还有一些改进方法，如 top-k 采样或 top-p 采样，其中，简单来说，在 top-k 采样中，我们会选择前 100 个 token，然后仅对这前 100 个得分最高的 token 应用随机选择，以避免产生无意义的 token。

In realistic LLMs, for example, that are well trained, “Berlin” might receive a probability of 99.999999 or something like that. So you’re almost certainly always sampling “Berlin” because it’s very confident that the answer is “Berlin” in this particular case. So yeah, that is how we would sample from this distribution. There are modifications like top-k sampling or top-p sampling where, let’s say, just for simplicity in top-k sampling, we would select the top 100 tokens and then apply this random choice only to the top 100, the 100 highest-scoring ones, so that we don’t get nonsense tokens in there.

对于这个例子，这并不重要。我的意思是，这只是另一个需要解释的东西，所以我略过它。你可以假设这已经是使用 top-k 等方法得到的前 100 个 token。

For this example, it doesn’t really matter. I mean, it’s just like another thing to explain, so I’m skimming over this. So you can maybe assume that this is already the top 100 tokens using top-k or something like that.

所以，例如，这里有一个例子。如果我们采样 10,000 次，而“Berlin”的概率非常高（99.9%），我们会采样到“Berlin” 9,997 次，采样到单词“Hal”两次，以及一次“Moh”。这些基本上是无意义的 token。在这种情况下，LLM 很少会产生无意义的内容，因为正如我之前提到的，我稍微分散了这个分布，使其更有趣。

And so, for example, here’s an example. If we sample 10,000 times with a probability of “Berlin” being very high, 99.9, we would sample “Berlin” 9,997 times, sample the word “Hal” twice, and one “Moh”. And these are basically nonsense tokens. It rarely happens that, in this case, the LLM might produce nonsense because, as I mentioned before, I spread out this distribution a bit to make it more interesting.

一个真正的 LLM 可能会在 10,000 次采样中全部得到“Berlin”，因为“Berlin”的概率非常高。但这只是为了说明目的。

A real LLM would probably, 10,000 out of 10,000 times, sample “Berlin” because the probability of “Berlin” is so high. But this is for illustration purposes.

从采样到水印 From Sampling to Watermarking

现在我们简要讨论了 LLM 在底层是如何工作的，我认为这本身就是一个有趣的概念。但这个问题我之前已经讲过很多次了，所以不想再让你们感到无聊。我只是想为现在的内容设定一些背景，解释水印是如何工作的。

Now we briefly talked about how LLMs work under the hood, which I think is an interesting concept in itself. But I've talked about this many times before, so I don't want to bore you. I just wanted to set up some context for now, explaining how this watermarking works.

所以，我们提到在采样时我们会选择得分最高的词元。或者我们使用概率采样，这会导致大多数时候选择得分最高的词元之一。现在，这里有一个没有水印的另一个例子。我稍微改变了提示词。现在的提示词是：今天的天气是“冷”，可能的答案可以是，例如，“灰色”或“阴天”。因此，与“柏林”的例子相比，我认为“灰色”和“阴天”在某种程度上是可以互换的。

So, we mentioned that we select the highest-scoring token when sampling. Or we use probability sampling, which will lead to one of the highest-scoring tokens being selected most of the time. Now here's another example without watermarking. I changed the prompt slightly. Now the prompt is: today's weather is "cold," and a possible answer could be, for example, "gray" or "overcast". So in contrast to the "Berlin" example, I would say "gray" and "overcast" are kind of interchangeable.

对于这个提示词，考虑到完成这段文本或写下下一个词元的目标，它们都是合理的下一个词元。所以选择哪一个几乎就像抛硬币一样。没有一个在客观上比另一个更差。所以当我们进行随机采样时，由于它们都有相对较高的分数，而且因为两者都是合理的词元，它们的分数也相似地高，我们可能会得到其中一个或另一个。

They are both reasonable next tokens for this prompt, given the goal of completing this text or writing the next token. So it's almost like a coin flip which one we want to select. There is not really an objectively worse one of one or the other. So when we do the random sampling, because they also have relatively high scores and their scores are similarly high since they are both plausible tokens, we might get one or the other.

因此，如果我们多次重复采样，几乎一半的时间我们会得到“阴天”，几乎一半的时间我们会得到“灰色”。这就是为什么 LLM 在提供相同提示词时经常得到不同答案的原因。如果你使用相同的提示词并多次询问 LLM，你通常会得到略有不同的答案。这是因为在某些位置，两个可能的词元几乎同样可能，所以它会选择其中一个或另一个。

So almost half of the time we would get "overcast", and almost half of the time we would get "gray" if we repeat the sampling multiple times. And that's how LLMs often end up with different answers if you provide the same prompt. If you use the same prompt and you ask the LLM multiple times, you often get slightly different answers. And that's because at certain positions, two possible tokens are almost equally likely, so it will choose one or the other.

而那个词元随后会影响所有后续的词元，依此类推。

And that token would then influence all subsequent tokens, and so forth.

随机种子与确定性采样 Random Seeds and Deterministic Sampling

现在，我想简要谈谈随机数生成。例如，如果我们使用这样的随机数生成器，它会生成一个随机数序列。如果我再次运行它，这里的数字序列是不同的。所以你可以看到每次我们生成五个数字，它们都是不同的。如果我在这里设置随机种子，比如 1、2、3，然后多次运行，我们仍然会得到随机数，但现在它们都是相同的，对吧？

Now, I wanted to briefly talk about random number generation. For example, if we use a random number generator like this, it will generate a random sequence of numbers. If I run it again, the sequence of numbers is different. So you can see every time we produce five numbers, they are different. If I set the random seed here, like one, two, three, and I run this multiple times, we still get random numbers, but they are now all the same, right?

所以它们仍然是随机的。如果我们使用随机种子，我们仍然会得到彼此不同的随机数，但它们是可复现的。因此，无论我们是否使用随机种子，我们仍然会得到随机数。但使用随机种子，我们得到的是可复现的数字序列。所以请记住：这只是一个简单的入门介绍，我们稍后会用到这个概念。

So they are still random. If we use a random seed, we still get random numbers that are different from each other, but they are reproducible. So whether we use a random seed or not, we still get random numbers. But with a random seed, we get a reproducible sequence of numbers. So keep this in mind: this is just like a little primer, and we will use this concept in a few moments.

例如，我之前提到，如果我们随机采样，可能会得到“gray”或“overcast”。现在，如果我们使用特定的随机种子，比如 42，我们总是会选择“overcast”。我的意思是，这仍然是随机选择，但我们使其具有确定性。在这种情况下，给定这个提示，模型将始终选择“overcast”。

So, for example, I mentioned before that we might get either “gray” or “overcast” if we randomly sample. Now, if we use a specific random seed like 42, we would always, for example, select “overcast”. I mean, it’s still a random selection, but we make it deterministic. In this case, given this prompt, the model will always select “overcast”.

如果我们使用不同的随机种子，模型可能会选择“gray”。每次采样时，它总是会选择“gray”作为下一个词。所以这仍然是随机采样，但我们根据随机种子使其具有确定性。

If we use a different random seed, the model might select “gray”. Every time we sample, it will always select “gray” as the next token. So it’s still random sampling, but we are making it deterministic based on the random seed.

水印密钥的工作原理 How the Watermarking Key Works

因此，在水印技术中，Claude 的水印有点像设置随机种子的概念。但这个随机种子，并不是像某个人写下一个固定数字那样固定不变，而是使用一个秘密密钥，本质上类似于 API 密钥。从该密钥，结合前四个词，他们推导出这个随机种子。

So, in watermarking, Claude watermarking is kind of like the idea that it sets a random seed. But this random seed, instead of being like a number that is fixed based on, I don’t know, someone writing down a fixed number, they’re using a secret key that is essentially like an API key, a secret key, and from that key, together with the four previous words, they derive this random seed essentially.

但关键在于，如果我回到上一张幻灯片，它和这里是一样的：本质上有一个固定的随机种子，该随机种子总是选择相同的下一个词元。所以，例如，这里不是使用随机种子 99，而是使用一个秘密密钥，并利用前一词元的信息来推导这个随机种子。但稍后会有更多细节。

But the idea is that if I go back one slide, it’s the same as here: there’s essentially a fixed random seed, and that random seed always selects the same next token. Okay, so instead of using random seed 99 here, for example, they have a secret key and also use information about the previous tokens to derive this random seed. But more on that later.

因此，其思想是水印使文本生成在特定位置更具确定性。例如，如果我们在左侧有一些合理的文本。假设我有一个文本，内容为：

So the idea is that watermarking makes the text generation more deterministic in certain positions. So, for example, if we have these plausible texts on the left side. So if I have a text that says,

我可能选择“阴天”或“灰色”。下一句可能是：

I may either pick “overcast” or “gray”. And the next sentence could be,

这意味着我基本上可以说“安静”或“静止”。所以，在文本中有一些位置，词元选择的概率几乎相等，正如我们之前所见。这意味着，如果没有水印，当我们运行提示词或通过 LLM 处理提示词时，我们可能有时得到这个答案，有时得到那个答案，等等。

Which means basically I could say either “quiet” or “still”. So there are certain positions in the text where we have token choices where they are almost equally likely, like we have seen before. So that means if we are, this is without watermarking, if we are running the prompt, or given the prompt through the LLM, we might sometimes get this answer here, sometimes this answer, and so forth.

根据位置的数量，我们这里可能有 128 个可能的答案。当然，文本越长，我们可以互换术语的位置就越多，组合就越多，或者说可能的输出文本就越多。所以，例如，再次强调，一个可能的输出文本可能是

And based on the number of positions, we might have 128 possible answers here. And of course, the longer the text, the more positions we have where we can have terms interchangeably, the more combinations, or the more output texts, there are. So, for example, again, one possible output text could be

所以那是一种可能的文本。另一种可能的文本是

So that is one possible text. Another possible text is

但底线是，你可以看到这里生成了两个非常合理的文本，而且还有更多的组合。所以它们都是合理的。没有一个必然比另一个更好。它们只是，你知道，细微的差别。如果我们不使用水印，我们可能会得到其中一个，或者只是随机的，对吧？因为随机采样，我们可能会得到其中一个或另一个。

But yeah, the bottom line is that you can see there are two very reasonable texts here being generated, and there are more combinations. So they are all reasonable. There isn’t one that is necessarily better than the other. They’re just, you know, slight variations. And if we don’t use watermarking, we might get either one, or it’s just random, right? Because of the random sampling, we might get one or the other.

现在，如果我们固定随机种子，如我之前提到的，例如，如果随机种子是 99，我们可能总是得到这个文本。所以，使用随机种子，我们可以在某种程度上固定我们得到的答案，因为那时随机采样仍然是随机的，但它是确定性的，因为它是可复现的。它总是相同的。好的，所以那仍然是没有水印，但现在有了随机种子。

Now, if we fix the random seed, as I mentioned before, for example, if the random seed is 99, we might always get this text here. So, using a random seed, we can kind of fix which answer we get, because then the random sampling is still random, but it’s deterministic in the sense that it’s reproducible. It’s always going to be the same then. Okay, so that is still without watermarking, now with a random seed.

而水印本质上在做同样的事情。现在，他们不仅仅使用一个简单的随机种子，而是有一个所谓的随机密钥，这个随机密钥本质上参与选择文本。但我们已经可以说的是，在 Claude 的博客文章中，他们说水印不应该使文本变差。如果我们看看这个机制，是的，它为什么不会使文本变差就说得通了。

And the watermarking is essentially doing the same thing. Now, instead of just using a simple random seed, they have a so-called random key, where this random key is involved in selecting the text, essentially. But what we can already say is that, in the Claude blog post, they say the watermarking shouldn’t make the text worse. If we look at this mechanism, yeah, it makes sense why it would not make the text worse.

顺便说一句，我并不是在为水印辩护，我只是想解释一下。所以请不要怪罪传话的人。但我想说的是，对于最终用户而言，水印无非就是固定一个随机种子，使这种采样变得具有确定性，如果这么说合理的话。

By the way, I’m not defending watermarks here. I’m just trying to explain. So please don’t kill the messenger here. But what I’m trying to say is that the watermarking is nothing else for the end user than fixing a random seed and making this sampling kind of deterministic, if that makes sense.

水印应用的位置 Where the Watermark Is Applied

好的，到目前为止的总结是没有水印的情况。我们经常在没有随机种子的情况下采样，因为我知道大多数人甚至不使用随机种子。老实说，我不认为你一定能通过 Claude 和 OpenAI 的 API 做到这一点。我知道你可以在 Ollama 中做到，但我总是遇到一些问题，因为我在我的一本书的补充材料中使用 Ollama 来生成一些文本。

Okay, so the summary so far is without watermarking. We often sample without a random seed because I know most people don't even use one. I honestly don't think you can necessarily do it with the Claude and OpenAI APIs. I know you can do it in Ollama, but I also always had some problems with that because I used Ollama in one of my books for the bonus material to generate some texts.

我固定了随机种子，但它仍然不总是确定性的，等等。所以这很棘手。你的结果也可能因软件版本等因素而异。无论如何，在没有水印的情况下，我们进行随机采样。在右侧有水印的情况下，我们仍然进行随机采样。但除了完全随机的采样之外，我们还有一个水印密钥。

I was fixing the random seed, but it still wasn't always deterministic, and so forth. So it's tricky. Your mileage may also vary, depending on the software version and so forth. Anyways, so without watermarking, we have this random sampling. With watermarking on the right-hand side, we still have the random sampling. But in addition to just a random sampling being fully random, we have this watermarking key.

这个水印密钥被传递给随机种子生成器，以设置特定的随机种子，使其具有确定性。但这本质上非常相似，而且正如我提到的，从头理解事物有很多好处。现在我们知道水印应用在哪里了。所以这本质上是应用于采样阶段。它不是应用在 LLM 内部，这实际上是一个很酷的知识。

And this watermarking key is passed to the random seed generator to set a specific random seed, making this deterministic. But it's essentially very similar, and like I mentioned, there's a lot of benefit in terms of understanding things from scratch. And now we know essentially where this watermark is applied to. So this is essentially applied to the sampling. It's not applied inside the LLM, which is actually cool knowledge.

因此，他们不需要为此训练新的 LLM。他们可以使用现有的 LLM，只需在采样阶段应用水印。他们不需要重新训练任何东西。所以，这确实很有趣，对吧？

So they don't need to train a new LLM for that. They can just use an existing LLM, and they just apply it at this sampling stage. They don't have to retrain anything or anything like that. So yeah, that is actually interesting, right?

水印检测的工作原理 How Watermark Detection Works

但我们还没有完全结束。我还想讨论如何理解或判断文本是否带有水印。检测水印只有在拥有密钥的情况下才有可能。例如，如果我们有这些不同的文本，本质上，在文本生成之后，你在互联网上找到一些随机文本（例如，你在互联网某处找到这里的文本四），你想知道：这是否带有水印？

But we are not quite done yet. I would also like to discuss how we can understand or determine whether text is watermarked. Detecting the watermark is only possible if we have access to the key. For example, if we have these different texts, and essentially, after the text was generated, you find some random text on the internet (for example, you find this text number four here on the internet somewhere), you want to know: is this watermarked?

嗯，这是不可能知道的，因为要知道，你需要水印密钥。你需要这个评分函数，然后你必须用评分函数对文本进行评分。其思想是，如果分数超过某个阈值，那么文本就带有水印；否则，就没有。但作为最终用户，我们无法做到这一点，因为我们没有这个密钥。所以密钥对我们不可用。

Well, it’s impossible to know because, in order to know, you would need the watermarking key. You need this scoring function, and then you have to score the text with a scoring function. The idea is that if the score is above a certain threshold, then the text is watermarked; otherwise, it’s not. But as the end user, we can’t do this because we don’t have this key. So the key is not available to us.

只有 Anthropic 会拥有密钥。然而，在这篇博客文章中，他们提到他们当然会提供它，或者他们将为此开发一个 API 并使其可用。我不知道。老实说，我并非关联方。我不知道细节。我只是在这篇博客文章中读到的。我所知道的就这些。所以这个 API 可能对某些公司是私有的，比如说，X 或 Substack Notes，当他们想要标记 AI 生成的帖子时。

Only Anthropic will have the key. However, in this blog post, they mentioned that they are providing it, of course, or they’re going to develop an API for that that they will make available. I don’t know. Honestly, I’m not affiliated. I don’t know the details. I was just reading this in this blog post. That’s all I know. So that API might as well be private for some companies, like, let’s say, X or Substack Notes, when they want to label AI-generated posts.

他们可能会将其公开供最终用户使用。谁知道呢？我们只能等待。但是的，所以这里的底线是，水印检测只有在拥有水印密钥或他们将要开发的 API 的情况下才有可能。

They may make it public for end users to use. Who knows? We will have to wait on that. But yeah, so the bottom line here is that watermark detection is only possible if we have this watermarking key or, of course, the API that they are going to develop.

如何去除水印 How to Remove a Watermark

现在，去除水印很有意思。既然我们知道了水印的工作原理，也就知道了它的缺点。我的意思是，这确实高度依赖于特定位置上的特定词元。例如，在这段给定的文本中，如果这些彩色单词或词元就是水印位置，我们知道可以通过编辑这些位置来去除水印，对吧？如果我们改变所有这些位置上的单词，我们就能 100%地击败这个水印。

Now, removing the watermark is interesting. So now that we know how the watermarking works, we also know the shortcomings. I mean, this is really highly dependent on specific tokens in certain positions. So, for example, in this given text, if these colored words or tokens are the watermarking positions, we know that we could remove the watermark by editing this, right? If we change all the words at these positions, we would be 100% able to defeat this watermark.

不过，问题在于我们并不知道，对吧？

Now, the problem, though, is that we don’t know, right?

所以我们不知道这些词在哪里，因为我们没有生成水印。所以我们不知道该看哪些位置。因此，实际场景中，我们可以随机编辑文本。我们会随机更改几个单词，希望我们更改了足够多的位置来编辑水印。那是一种去除水印的方法。而且，由于我们也不知道哪些是得分最高的词，因为那需要我们访问大语言模型（LLM）并重新运行提示词来找出哪些词得分最高，所以我们只能猜测。

So we don’t know where these words are because we haven’t generated the watermark. So we don’t know which positions to look at. So the practical scenario here is that we could just randomly edit the text. So we would randomly change a few words and hope that we change enough positions to edit the watermark. So that would be one way to remove it. And since we also don’t know which are the highest-scoring ones, because that would require us to have access to the LLM and rerun the prompt through the LLM to find out which words are the highest-scoring, we can kind of only guess.

例如，我们可能会说，哦，我们把“overcast”换成“cloudy”，因为我们不知道“gray”得分很高，你知道吗？所以在这种情况下，可能直觉上会说“gray”，但有些情况可能不那么直观。所以我想说明的是，这只是一般的文本编辑，我们修改了位置，但我们仍然在猜测水印位置是什么。

So for example, we might say, oh, we replace “overcast” with “cloudy” because we don’t know that “gray” was high-scoring, you know? So in this case, it might be intuitive to say “gray”, but there might be cases where it’s not so intuitive. So what I’m trying to illustrate here is just some general text editing where we are modifying positions, but we are still kind of guessing what a watermark position is.

所以，由于我们不知道，我们只是在这里或那里添加了几个词。如果我们添加了足够的词，那也能击败水印。

So since we don’t know, we added just a few words here and there. And if we added enough words, that would also defeat the watermark.

水印评分函数的工作原理 How the Watermark Scoring Function Works

以上就是水印技术的简要介绍。我提到过有一个评分函数用来判断某段文本是否带有水印。我想在这里作为附加内容简要讲解一下这个评分函数的工作原理，因为这也是一个有趣的信息。它有点复杂，但理解评分函数的工作原理并非必需。

So that was watermarking in a nutshell. I mentioned that there is a scoring function to determine whether something is watermarked. I want to cover it as a bonus here. It's already a long video, but as a bonus, I wanted to briefly explain how this scoring function works because it's also interesting information. It's a bit complicated, and it's not essential to understand how the scoring function works.

但他们之所以采用特定的方式，是为了降低检测成本。否则，如果我往回翻一两页幻灯片，如果你想检查某段文本是否带有水印，即使他们想检查，也必须重新运行提示词以获得这些分数，然后应用水印随机种子生成文本，再进行比对。那将非常昂贵，因为基本上每段要比较的文本，你都得重新运行大语言模型。

But the reason they do it in a certain way is to make detection cheaper. Otherwise, if I go back one or two slides, if you wanted to check whether something is watermarked, even if they wanted to check, they would have to rerun the prompt to get the scores, then apply the watermarking random seed to get the text, and then compare. That would be very expensive because essentially for every text you want to compare, you would have to rerun the LLM.

你必须知道是哪个大语言模型，而且这实际上不可行，因为你往往连提示词都不知道，对吧？所以他们有一个技巧，我可以说，用来修改采样过程，使得在评分阶段不再需要使用大语言模型。在博客文章中，他们提到这个方法源自一篇论文。

You would have to know which LLM, and that would be really unfeasible because often you don't even know the prompt, right? So they have a trick that they use to, I would say, modify the sampling so that you don't use or don't need the LLM later on for the scoring stage. In the blog post, they mentioned that they derived this method from a paper.

那是一篇发表在《自然》杂志上的论文，这个方法叫做 SynthID-Text。那篇论文大概是一两年前发表的，来自谷歌。他们使用了一种类似的技术，称为 Claude 水印。我不确定，抱歉，我不确定他们是否确切使用了那种技术，但那是他们提到的。

It was a Nature paper, and this method is called SynthID-Text. That was a paper that came out maybe one or two years ago. It was by Google, and they use a similar technique they call Claude watermarking. I don't know, sorry, I don't know if they use exactly that technique, but that's the one they mentioned.

SynthID 文本与锦标赛采样 SynthID Text and Tournament Sampling

那么它是如何工作的呢？在我们看幻灯片之前，我们先看一下常规的概述：我们有一段文本，将其输入 LLM，得到 logit 分布，然后从分布中采样得到输出 token。在这里，在采样过程中，我们使用水印密钥和随机种子生成器。这仍然是正确的，仍然是正在发生的事情，但我想，对于这个 token 是如何采样的，还有更多的细微差别。

So how does it work? Before we looked at the slides, we looked at the regular, let's say, overview here, where we have some text. We put it through the LLM. We get this logit distribution and then we sample from the distribution and get the output token. And here, during the sampling, we use the watermarking key and the random seed generator. So this is still correct. This is still what's going on, but there is a bit more, I guess, nuance to how this token is sampled.

所以，他们不仅仅是使用，比如说，NumPy 的随机选择。他们在这里使用了更复杂的方法。

So they're not just using, let's say, NumPy's random choice. They're using something a bit more sophisticated here.

再次假设我们的上下文是“今天天气很冷”，我们想生成下一个 token。例如：“gray”、“overcast”、“gloomy”、“cloudy”。“gray”的概率是 50%，“overcast”是 30%，“gloomy”是 15%。假设“cloudy”是 0.05，其余的是 0。这里看起来当然有点不同。假设那是“gray”和“overcast”。我只是重复使用这个图。但现在想象这些是最可能的，比如“gray”和“overcast”，而其他所有词都非常小，除了“gloomy”和“cloudy”，也许。

So assume, again, our context is "the weather today is cold," and we want to generate the next token. So, for example: "gray", "overcast", "gloomy", "cloudy". "Gray" is 50% probability, "overcast" is 30%, "gloomy" is 15%. Let's say "cloudy" is 0.05 and the rest is, let's say, 0. Here it looks, of course, a bit different. Let's say that's "gray" and "overcast". I'm just reusing this figure. But now imagine these are the most likely ones, like "gray" and "overcast", and everything else is just very small, except "gloomy" and "cloudy," maybe.

所以本质上，考虑一个非常小的词汇表，这个例子中只有四个词，而不是这里的 50 个词，只是为了更简单。现在，正如我之前提到的，我们可以使用 NumPy 的随机选择，根据这些概率来采样下一个 token。

So essentially, think about just a very small vocabulary for this example of four words instead of all these 50 words here, just to make it even simpler. Now, as I mentioned before, we could use NumPy's random choice with these probabilities to sample the next token.

我们可以使用水印密钥和随机种子生成器使其具有确定性，并获得我们想要的特定水印。但正如我之前提到的，这将非常昂贵。不是采样本身，那没关系，这很便宜。但如果我们试图检查互联网上的随机文本，后续的检测将非常昂贵。

And we could use the watermarking key with this random seed generator to make it deterministic and get the certain watermark that we want. But as I mentioned before, this would be very expensive. Not the sampling itself. That doesn't matter. This is pretty cheap. But the detection later on would be very expensive if we are trying to check random text on the internet.

因此，相反，他们在采样和生成过程中也使用它，以便在检测阶段可以复用。他们使用的方法称为锦标赛采样。这是用来替代随机选择之类的概念。那么它是如何工作的呢？它看起来可能有点复杂，但实际上比看起来要简单得多，说实话。

So instead, what they use, they also use it during the sampling, during the generation, so that it can be reused later during detection. What they use is called tournament sampling. So this is instead of using something like random choice, they use a concept called tournament sampling. And so how does that work? It might look a bit complicated, but it looks really more complicated than it really is, to be honest.

你可能需要在某个时候暂停视频，仔细看一下这个图。但我认为它实际上比看起来要简单。一旦你掌握了它，就非常直接了。但让我试着在这里解释一下。我们仍然有这个上下文，然后我们有这些可能的或合理的下一个词，以及它们不同的概率。

So you might have to, I guess, stop the video at some point and just sit with the figure a bit. But I think it is actually simpler than it looks like. It’s like once you get the hang of it, it’s pretty straightforward. But let me try to explain here. So what we have is we have still this context, and then we have these probable or plausible next tokens with these different probabilities.

现在他们有了他们所谓的随机水印函数。

Now they have something they call random watermarking functions.

随机水印函数 Random Watermarking Functions

这里我们有三个水印函数，G1、G2 和 G3。实际上，它们可能有 30 个、50 个甚至更多。这里我只用三个，因为在这张幻灯片上更简单。只是更小，你知道，更适合放在幻灯片上。现在，如果我们看“gray”这个词，它可能会给出一个签名 101。我的意思是，如果我们用这个水印密钥生成这个随机种子，并且我们有三个函数 G1、G2、G3。

Here we have three watermarking functions, G1, G2, and G3. In reality, they might have 30, 50, or even more. Here I’m just using three because that is simpler on this slide. It’s just smaller, you know, like it fits better on the slide. Now, if we look at this word “gray”, this might give us a signature 101. With that, I mean, if we use this watermarking key to generate this random seed, and we have three functions, G1, G2, G3.

如果我放入“gray”这个词，我这里跳过的是，通常你会把“gray”和上下文中的前三个或四个词放在一起。所以是“cold”和“gray”。如果我把它和这个水印密钥一起放入 G1，我得到值 1。为什么？嗯，这就是这个函数的工作方式。它就像一个随机函数。随机函数要么返回 0 要么返回 1。在这种情况下，用这个随机密钥和这个令牌，它返回 1。

If I put the word “gray”, what I’m skipping here is that usually you put the word “gray” together with the four or three previous words from the context. So it’s “cold” and “gray”. If I put that into G1 together with this watermarking key, I get the value one. Why? Well, that’s just how this function works. It’s like a random function. The random function either returns zero or one. In this case, with this random key and this token, it returns one.

用相同的密钥，但不同的函数，你得到值 0。然后这里你又得到 1。所以如果我们有更多的函数（当然，30 个函数），这将是一个很长的 1 和 0 的字符串。

With the same key, but a different function, you get the value zero. And then here you get a one again. So if we have more functions (of course, 30 functions), this will be a very long string of ones and zeros.

它基本上就像一个位串，就像你有 0 和 1 的位。好的。所以这是针对“gray”的。所以我们通过使用这些水印函数得到签名 101。现在我们可以对所有其他的词做同样的事情。所以我们可以对“gray”做。我们可以对“overcast”、“gloomy”和“cloudy”做。所以每个词在这里都有不同的签名。例如，“overcast”是 0、1、0。

It’s basically like a bit string, like if you have bits of zeros and ones. Okay. So this is for “gray”. So we get the signature 101 through using these watermarking functions. Now we can do the same thing for all the other ones. So we can do it for “gray”. We can do it for “overcast”, “gloomy”, and “cloudy”. So each one has a different signature here. So, for example, “overcast” is zero, one, zero.

“Gloomy”有 0、0、1。“Cloudy”有 1、0、0。好的。所以我们现在有了这些位。下一步是所谓的锦标赛采样，我们只是将它们配对。

“Gloomy” has zero, zero, one. “Cloudy” has one, zero, zero. Okay. So we have these bits here now. The next step is a so-called tournament sampling where we just pair them.

锦标赛采样逐步解析 Tournament Sampling Step by Step

就像足球锦标赛的淘汰赛阶段或美式橄榄球的季后赛一样，总是有两支球队相互对抗。这里的思想类似：我们有一对词元，它们本质上在相互竞争，分数来自这些函数。我们从第一轮的第一个函数开始，所以有“cloudy”和“gray”。

Like a soccer tournament's knockout stages or the playoffs in American football, two teams always play against each other. The same idea applies here: we have a pair of tokens competing against each other, and the scores come from these functions. We start with the first function in the first round, so we have "cloudy" and "gray".

我们在这里查找：“gray”是 1，“cloudy”是 1，所以是 1 和 1。“overcast”和“gray”：“overcast”是 0，“gray”是 1，所以是 0 和 1。“gloomy”和“overcast”：“gloomy”是 0，“overcast”是 0，所以是 0 和 0。然后我们又有“gray”和“gray”，因为其他词元不够了，所以我们有一个重复。

We look up here: "gray" is a one and "cloudy" is a one, so one and one. "Overcast" and "gray": "overcast" is zero, "gray" is one, so zero and one. "Gloomy" and "overcast": "gloomy" is zero, "overcast" is zero, so zero and zero. Then we have "gray" and "gray" again because we are running out of other tokens, so we have one duplicate.

这是随机选择的，所以这里我们有 1 和 1。现在我们看结果：这是平局。在平局的情况下，我们也使用随机种子和水印密钥进行随机选择。这里，“cloudy”胜出。从这一对中，根据 G1，“gray”是赢家，因为它有 1，所以“gray”胜出。然后这里，“overcast”和“gray”是平局，随机选择，“gray”也被随机选中。

This is chosen randomly, so we have one and one here. Now we look at the results: this is a tie. In the case of a tie, we also select randomly using the random seed and the watermarking key. Here, "cloudy" survives. From this pair, according to G1, "gray" is the winner because it has a one, so "gray" survives. Then here, "overcast" and "gray" are a tie, randomly selected, and "gray" is also randomly selected.

所以我们现在有“cloudy”、“gray”、“overcast”和“gray”。我们进行下一轮比赛，使用 G2。根据 G2，“cloudy”是 0，“gray”也是 0，“overcast”是 1，“gray”也是 0（抱歉）。所以我们进入锦标赛的下一阶段。

So we now have "cloudy", "gray", "overcast", and "gray". We play the next round in this tournament, using G2. According to G2, "cloudy" has a zero, "gray" also has zero, "overcast" has one, and "gray" also has zero (sorry). So we proceed to the next stage of the tournament.

我们有一个平局，随机选择“gray”。这里，“overcast”是赢家。所以决赛是“gray”对“overcast”。我们再看分数：“gray”是 1，“overcast”是 0，所以“gray”是赢家。这就是词元“gray”被采样的过程。水印密钥在这里起什么作用？水印密钥，如果我回翻几页幻灯片，是用来通过随机水印函数生成这些分数的。

We have a tie, and we randomly select "gray". Here, "overcast" is the winner. So we have "gray" versus "overcast" in the final. We look at the scores again: "gray" has a one, "overcast" has a zero, so "gray" is the winner. That is how the token "gray" is sampled. What is the watermarking key doing here? The watermarking key, if I go back a few slides, is used to generate these scores via the random watermarking functions.

因此，水印密钥本质上决定了我们在这些阶段获得的值。水印密钥仍然至关重要；否则，这些签名将呈现不同的形态。

So the watermarking key essentially determines what values we obtain at these stages. The watermarking key remains crucial; otherwise, these signatures would appear different.

无需重跑 LLM 即可检测水印 Detecting Watermarks Without Rerunning the LLM

所以我们现在已经采样了下一个词元。这就是这个修改后的采样过程的工作原理。我们本可以使用 NumPy 的 random.choice。但它的缺点是，如果我们想对互联网上的随机文本进行评分，就必须重新运行 LLM。而使用这种技术，我们不需要。我稍后会展示。所以这种技术听起来确实很奇怪且繁琐，但它的优点是，我们现在可以更容易地对随机文本进行评分，而无需重新运行 LLM。

So we have now sampled the next token. And that's just how this modified sampling procedure works. We could have used NumPy's random.choice. But the shortcoming of that is that if we want to score random text on the internet, we would have to rerun the LLM. With this technique, we don't. I will show you in a moment. So this technique sounds really weird and cumbersome, but it has the advantage that we can now score random text more easily without having to rerun the LLM.

所以这基本上只是为了使检测更容易、更便宜。

So it's essentially just to make the detection easier and cheaper.

所以，例如，如果我们有一段新文本。我这里只是用相同的文本，但假设它是新文本。这是在采样之后，当我们进行评分时。假设我们在互联网上发现了这段文本。文本是“the weather today is cold and gray”，我们想知道这是否是 LLM 生成的。所以我们，或者 Claude/Anthropic，会拥有水印密钥和这些函数：G1、G2 和 G3。

So, for example, if we have a new text. So I'm just using the same text here, but let's assume it's new text. So this is after the sampling, when we are scoring. And let's say we are discovering this text on the internet. And the text is "the weather today is cold and gray", and we want to know if this is LLM-generated or not. So we would, or Claude/Anthropic would, have the watermarking key and these functions: G1, G2, and G3.

它会将这段文本通过这些函数。对于这里“gray”的一个位置，我们会得到 101，类似于我们在生成过程中得到的。所以这和之前一样。如果我们把这些位加起来，有两个位，对吧？这里一个和一个。所以它有两个位的信息，为了简单起见，我们这么说。这只是一个非常简单的说明。

And it would put this text through these functions. For the one position here for "gray", we would get 101, similar to what we got during the generation process. So this is the same as before. And this has, if we add up these bits, two bits, right? One and one here. So it has two bits of information, let's say, for simplicity. This is just a really simple illustration.

但假设我们在这里的“gray”位置得到分数 2。如果这个位置是另一个词“overcast”，我们会得到 1；如果是“gloomy”，我们也得到 1，“cloudy”也是 1。所以我只是对每一行求和，对吧？所以这就像我们在每个位置得到的分数。而这里我只关注最后一个位置。

But let's assume we get a score of two here for the "gray" in this position. If we had a different word here, "overcast," in this position, we would get one; if we get "gloomy," like we also have one, and "cloudy" one. So I'm just summing over each row here, right? So that's just like a score we would get at each position. And here I'm only looking at the last position.

如果我在其他位置也这样做，我会在不同位置得到不同的分数。例如，假设在第一个位置我得到 2，这里得到 2，对于“today”我得到 3，对于“is”得到 2，“cold”得到 2，“gray”得到 3。所以这里我应用了上一张幻灯片中展示的水印函数。

If I were to do this at other positions, I would get different scores at different positions. For example, let's assume at the first position I get a 2, here I get a 2, for "today" I get a 3, for "is" I get a 2, "cold" a 2, and "gray" a 3. So here I'm applying these watermarking functions as shown on the previous slide.

我只是将这三个函数的数值相加。水印函数非常廉价，因此你可以快速地对整个文本运行它们并获得这些分数。然后，基于这些分数，我可以计算平均比特数。所以如果我对这里的所有值取平均，假设得到 2.23。

And I'm just adding up these numbers across the three functions. The watermarking functions are very cheap, so you can quickly run them on the entire text and obtain these scores. Then, based on that, I can compute the average bits. So if I average over all these values here, let's say I get 2.23.

现在，如果我有一个略有不同的文本，例如，我将“today”换成“now”，将“gray”换成“overcast”，这些现在得到 1 和 1 的分数。如果我对整个字符串取平均，得到 1.71。为此，我不需要 LLM。我只需要水印密钥、随机种子生成器以及这些函数 G1、G2 和 G3。这就是我所需要的全部。

Now, if I have slightly different text, for example, I swapped "today" with "now" and "gray" with "overcast", these now get scores of 1 and 1. If I average over this whole string, I get 1.71. For this, I don't need an LLM. All I need is the watermarking key, the random seed generator, and these functions, G1, G2, and G3. That's all I need.

我不需要 LLM，就能得到这个分数。他们所做的是应用一个阈值。

I don't need the LLM, and I can get this score. What they do is apply a threshold.

例如，他们并不使用这个确切的阈值，但我们可以说：如果分数大于 2，则文本带有水印；如果分数小于 2，则没有水印。所以这里，如果分数大于 2，就是肯定的——这个文本带有水印。在这种情况下，1.71 不大于 2，所以这个文本没有水印。好的，这就是我们如何检测互联网上的随机文本是否带有水印的方法。

For example, they don't use this exact threshold, but we can say: if the score is greater than 2, then the text is watermarked; if the score is smaller than 2, it's not watermarked. So here, if the score is greater than 2, it's a yes—this text is watermarked. In this case, 1.71 is not greater than 2, so this text is not watermarked. Okay, so that's how we can detect whether random text on the internet is watermarked or not.

这本质上就是应用这些水印函数，然后对分数取平均并应用一个阈值。好的。

It's essentially just applying these watermarking functions and then averaging over the scores and applying a threshold. Okay.

再次强调，锦标赛采样主要是为了使检测更简单、更便宜。我们也可以使用类似 NumPy 的随机选择并设置随机种子，或者使采样具有确定性。但话又说回来，那样就很难对互联网上的任意文本进行评分了。

So again, the tournament sampling is mainly to make detection easier and cheaper. We could also use something like NumPy's random choice with a random seed or to make the sampling deterministic. But then again, it would be hard to score any text on the internet.

水印总结与局限 Watermarking Summary and Limitations

所以，总结还是一样的。无水印和水印之间的区别在于，我们在这里使用水印密钥来控制采样。在采样内部，我们采用了锦标赛采样。而且，正如我之前提到的，检测水印需要密钥和水印函数 G1 到 Gn。

So yeah, the summary is still the same, though. The thing that is different between no watermarking and watermarking is that we are controlling this sampling here with the watermarking key. And inside that, we have this tournament sampling. And yeah, as I mentioned before, detecting the watermarks requires the secret key and the watermarking functions G1 to Gn.

再说一遍，移除水印——因为我觉得有些人可能对此感兴趣——理想情况下需要编辑这里的所有位置。但由于我们不知道哪些位置被加了水印，而且在内部，他们选择的位置上的 token 概率是均等的。但可能有些位置并非如此。例如，对于“trees”这个词，我们可能甚至没有一个高分替代词，所以他们不会在那个位置加水印。

And again, removing the watermark, because I think that’s maybe interesting to some people, would ideally involve editing all the positions here. But since we don’t know which positions are watermarked and internally, they choose the positions so that they have equally likely tokens at those positions. And there might be positions where that’s not true. So here, for example, for “trees”, we might not even have an alternative word that is high scoring so they don’t watermark that position.

所以本质上，他们只在某些位置加水印。由于我们不知道哪些位置需要破坏或移除水印，我们不得不……

So they only do the watermarking at certain positions essentially. Since we don’t know which positions to kind of defeat or remove the watermark, we would...

……编辑文本中的多个位置。所以我认为，这对于 AI 生成文本的未来意味着，这实际上……

...have to edit several places in the text. So what I think that means for the future of AI-generated text is that this actually...

这对 AI 生成文本意味着什么 What This Means for AI-Generated Text

...可能导致更差的 AI 生成文本。所以我认为，如果有人喜欢在互联网上到处使用 AI 生成的文本，比如说有一个新闻网站喜欢用 AI 生成的文本写新闻，我不认为水印一定会阻止他们这样做。他们可能仍然想要生成 AI 文本，因为那是他们工作流程的一部分。所以我的猜测是，他们会使用另一个模型。

...might result in worse AI-generated text. So I think if there's a person who likes to use AI-generated text everywhere on the internet, let's say there's a news website that likes to use AI-generated text to write the news, I don't think watermarking will necessarily stop them from doing that. They will probably still want to generate AI-generated text because that's part of their workflow. So I think my guess is that they'll use another model.

他们只会用第二个模型来编辑文本，得到所谓的“编辑后的 AI 生成文本”。所以这使流程复杂化了。不是直接从 Claude 获取文本，而是现在用 Claude 生成 AI 文本，然后通过一个本地模型，最后得到编辑后的 AI 文本，这很可能不再有水印。那么为什么用本地模型？我只是认为本地模型，因为我认为所有的提供商——专有的 LLM，不仅是 Claude，还有谷歌——我的意思是，谷歌写了这篇论文，对吧？

They'll just use a second model to edit the text to get the so-called edited AI-generated text. So it's complicating the pipeline. Instead of getting the text directly from Claude, it's now using Claude to generate AI-generated text, passing it through a local model, and then having edited AI-generated text, which is likely not watermarked anymore. So why a local model? I just think a local model because I think all the providers - the proprietary LLMs, not only Claude, but also Google— I mean, Google wrote this paper, right?

所以我认为他们也在给 Gemini 文本加水印。而且我认为 OpenAI 可能已经在做或者也会这样做。我的意思是，我只是猜测，但我想象每个人可能都会做类似的事情，因为有欧盟法规要求这样做。而且根据博客文章，这显然也是 Claude 这样做的原因。是的，所以我认为本地模型可能不会，至少现在不会，实现这种水印。

So I'm thinking that they are also watermarking Gemini text. And I think OpenAI is probably already doing it or will do so as well. I mean, I'm just speculating, but I'm imagining everyone will probably do something like that because there's like an EU regulation that requires that. And that's, according to the blog post, apparently why Claude is doing it. Yeah, so I'm thinking local models may not, at least not yet, implement this watermarking.

所以我认为人们会使用本地模型，然后生成编辑后的 AI 文本。我的猜测是它会比原始文本稍差，因为对于本地模型，你可能现在使用的是较小的模型。所以，我的意思是，你也可以技术上直接使用本地模型生成文本。但在我看来，编辑文本比生成文本简单。所以对于文本的生成，你可能会使用非常昂贵的高端，我不知道，比如最高级、最昂贵的 Claude 模型来处理复杂的文本。

So I think people will just use a local model and then generate edited AI-generated text. And my guess is it will be slightly worse than the original text because for the local model, you might now be using a smaller model. So, I mean, you could also technically just use the local model directly to generate text. But in my view, editing text is simpler than generating text. So for the generation of the text, you might use a very expensive high-end, I don't know, like the highest, most expensive Claude model for complicated text.

然后你使用一个更便宜的本地模型来进行这些外科手术式的编辑，基本上就是这样。这很可能会发生。为什么更差？所以如果我们回顾一下我们刚刚添加随机位置的图形，你可能只是为了改变而改变单词。然后这有可能使文本变得更糟。所以你仍然可能有生成的文本，但它有点像被笨拙地编辑了。

And then you use a cheaper local model to make these surgical edits, essentially. That's probably what's going to happen. And why worse? So if we look back at this graphic where we just added random positions, you might be just changing words for the sake of changing them. And then it risks making the text worse. So you might still have generated text, but it's kind of like it's edited awkwardly.

结语 Final Thoughts

但无论如何，我在这里的目标是解释水印的工作原理，而不是其全球影响。我希望这种幕后、内部视角能有所帮助。水印并不像看起来那么复杂，但毕竟用了 52 页幻灯片，所以也不是很简单。希望你觉得这次讲座有用。下次再见。

But anyway, my goal here was to explain how the watermarking works, not the worldwide ramifications of it. I hope this behind-the-scenes, under-the-hood look is useful. The watermarking is not as complicated as it might seem, but it still took 52 slides, so it was not trivial either. I hope you found this lecture useful. Until next time, see you then.

附：如果你喜欢这种风格的更多讲解，我虽然不经常在 YouTube 上发布视频，但多年来已积累了 300 多个视频，你可以在这里的我的 YouTube 频道上找到。

PS: If you like more explainers in this style, I don't post videos to YouTube regularly, but I have accumulated over 300 videos over the years, which you can find on my YouTube channel here.