Claude 如何为 AI 生成的文本添加水印

48 分钟视频详解：Token 采样、水印检测与去除 A 48-Minute Video Walkthrough of Token Sampling, Watermark Detection, and Removal

我最近在 Substack 上发布了一篇关于 Claude 新水印流程及实现的笔记。由于这是一个非常热门的话题，并引发了热烈的讨论，我认为在解释其工作原理时更详细一些可能会很有趣。

I recently posted a Substack note about Claude's new watermarking process and implementation. Since it's such a popular topic and sparked such a lively discussion, I thought it might be interesting to go into a bit more detail when explaining how it works.

与通常的文字文章不同，我录制了一个关于该主题的小讲座（以区别于我平时的文章）。因此，下面是视频以及文字记录。

Instead of the usual text article, I recorded a little lecture on the topic (to change it up a bit from my usual articles). So, below is the video along with a transcript.

最初，我计划制作 10 张幻灯片并录制一个 10 分钟的短视频。然而，在制作过程中，我时不时地添加了一些关键细节，结果导致幻灯片超过 50 张，录制时长达到 48 分钟。

Originally, I planned to make 10 slides and record a short 10-min video. However, while putting it together, I added some crucial details here and there, resulting in >50 slides and a 48 min recording.

不过，我希望这能很好地解释清楚！观看愉快！

I hope that this now explains it well, though! Happy watching!

* 如果你更喜欢使用 YouTube 播放器，我也有一个 YouTube 版本

* I also have a YouTube version if you prefer using the YouTube player

视频文字稿 Video Transcript

注：为便于阅读，以下文字稿经过轻微编辑与整理，但保留了上述视频讲座的整体顺序与流程。

Note: The transcript below is slightly edited and cleaned up for readability but preserves the overall order and flow of the video lecture above.

Claude 文本水印技术的工作原理 How Claude’s Text Watermarking Works

大家好。几天前，Anthropic 宣布他们将对其 Claude 模型的文本输出添加水印。随后我在社交媒体上发帖简要解释了其工作原理。没错，这个帖子相当受欢迎。受欢迎的不是水印本身，而是对其原理或背后机制的解读。因此，值得进一步展开详细说明，因为原帖只有一张图，却引发了大量问题和讨论。

Hi everyone. So, a few days ago, Anthropic announced that they will watermark the text outputs of their Claude models. I then did a social media post briefly explaining how that works. And yeah, this was quite the popular post. So not the watermarking itself was popular, but I guess the explanation or the mechanism behind it. Then, it might be worthwhile expanding this a bit to explain it in more detail, because this post only had one figure, and there were a lot of questions and discussions.

所以我想，那就多画几张图吧。我原本计划做 10 张幻灯片来带大家过一遍，结果变成了 50 张，但我希望这能真正解释清楚这种水印技术如何运作、水印本身如何失效或被移除等等。我认为这可能是一个有趣的话题，因为如今很多人使用 LLM，也在互联网上消费大量可能由 LLM 生成的文本。

So, I thought, well, let's make a few more figures. I actually originally planned to do like 10 slides and walk you through it. It ended up being 50 slides, but I hope this really explains how this watermarking technique works well, how watermarking itself can fail or be removed, and so forth. So I think it might be an interesting topic because a lot of people use LLMs these days and also consume a lot of text on the Internet that might be generated by LLMs.

现在有了这种水印，人们担心水印会让文本质量变差，或者水印到底有什么好处？它意味着什么？我认为，如果我们能更好地理解水印是什么，这将大有帮助，然后我们就能自己判断这是否是好事，以及利弊如何。

And now there's going to be this watermarking, and there's this, I guess, fear of watermarking making text worse, or what's actually the benefit of this watermarking? And so what does it mean? And I think if we understand a bit better what watermarking is, that goes a long way, and then we can make up our own minds about whether that's a good thing or not, and so forth, like the pros and cons.

因此，我在这里的目标是解释底层机制如何工作，以及他们将如何实现这类水印——文本水印。

So, my goal here is really to explain how the underlying mechanism works and how they are going to implement this type of watermarking, text watermarking.

这也是一个很好的例子，说明为什么从头理解事物实际上非常有用。这种水印技术也是解释传统模型或一般 LLM 在底层如何工作的好方法。所以，你可能知道我喜欢从头做起。比如，我有书：《从头构建大型语言模型》、《从头构建推理模型》。我还有一些标为“从头开始”的文章。

It's also a great example to illustrate why understanding things from scratch is actually quite useful. This watermarking technique is also a nice way to explain how conventional models or LLMs in general work under the hood. So yeah, you may know I like doing things from scratch. Like, I have my books: Build a Large Language Model From Scratch, Build a Reasoning Model From Scratch. I have some articles labeled from scratch.

所以，对我来说，“从零开始”通常包括编码。所以这个不会与编码相关，但从零开始编码实际上是一种非常非常有用的技术，因为它确实能帮助你理解某事物是如何实现的。然后从中我们可以推导出我们的理解、图表和概念，因为如果我们不真正实现事物，如果没有代码，有时确实会很模糊。当然，你知道，正如我意识到的，不是每个人都在从零开始编码了。

So, for me, "from scratch" often includes coding. So this one will not be coding-related, but coding from scratch is actually a very, very useful technique because it really helps you understand how something is implemented. And then from that we can derive our understanding, figures, concepts, because if we don't really implement things, if there's no code, it's really sometimes ambiguous. And of course, you know, as I realized, not everyone is coding from scratch anymore.

就像过去，从零开始编码是我们唯一的方式。我的意思是，当时只有人类在编码。如今，编码可以由 LLM 完成。然而，这并不意味着阅读代码不再有用，因为它携带了大量信息。所以在这个水印的例子中，花些时间从零开始编码一个 LLM 确实让你意识到内部的采样是如何实现的。我们仍然有一些相关的代码片段。

Like back in the day, coding something from scratch was all we had. I mean, there were only humans coding. Nowadays, coding can be done by LLMs. However, that doesn't mean reading code is no longer useful, because it carries a lot of information. So in this case here with this watermarking, spending some time coding an LLM from scratch really makes you realize how this sampling inside is implemented. We still have some relevant code snippets.

然后这反过来真的帮助我们理解，哦，水印是在这个位置应用的，这有如此这般的后果等等。所以我认为，即使人们可能不再从零开始编码，至少不再总是如此，能够为了教育目的从零开始构建一些东西以深入理解某事物，然后也为了研究目的以透明的方式操纵它，而不是隐藏在层层抽象之后，这仍然是有用的。

And then that really, in turn, helps us understand, oh, the watermarking is applied at this position, and this has so-and-so consequences and so forth. So I think even though people may not be coding from scratch, at least not all the time anymore, it is still useful being able to, let's say, build something from scratch for educational purposes to understand something deeply and then also for research purposes to manipulate this in a transparent way that is not hidden away in tons of layers of abstraction.

但除此之外，我认为这里只是一个巧合的良性关系，因为对于这个幻灯片，我实际上使用了很多来自我从零开始编码材料的图表。

But that aside, I think it's just a coincidental nice relationship here because, for this slide deck, I actually used a lot of figures from my from-scratch coding materials.

所以几天前（这是 8 月 14 日），有一篇文章《Claude 的文本水印如何工作》，这里有一篇文章；它就像一个屏幕录制，所以幻灯片里可以包含所有内容，但有很多细节。他们实际上更新了几次，所以最初我读这个的时候，它要短得多。不过，它非常，我猜，概念性的；有这样一个概述，而且，我的意思是，里面没有一张图。

So a few days ago (this is August 14), there was this article, How Claude's Text Watermark Works, and there was this article here; it's just like a screen recording, so it can have everything in the slides, but there's plenty of detail. They updated it actually a couple of times, so originally when I read this, it was a way shorter. Still, it is very, I guess, conceptual; there's like this overview, and there's, I mean, there's not a single figure in there.

因此，理解他们试图做什么仍然有些困难。他们解释了很多关于为什么要做这件事，但没有解释如何做。他们在某处链接到了一篇论文，那篇论文也非常技术性。所以我认为，也许退一步，从头开始，来理解他们试图用这种水印技术实现什么，是有道理的。

And so it's kind of still hard to understand what they're trying to do. So they explain a lot about why they're going to do it, but they don't explain how. They're linking to one paper somewhere there, which is very technical also. So I do think it makes sense maybe to take a step back and start at the beginning to kind of understand what they're trying to implement here with this watermarking technique.

顺便说一下，水印的动机是让他们能够识别，如果有人发布了一些文本，他们可以说，哦，这段文本是由我们的 Claude Opus 4.8 模型生成的，例如，这样他们就有办法判断，OK，这段文本是 AI 生成的，因为它带有这个水印。而且这个水印对用户是不可见的，所以只有他们才能解码并找出文本是否带有他们的水印。

And so the motivation, by the way, of watermarking is for them to identify if someone posts some text that they can say, oh, this text was generated by our Claude Opus 4.8 model, for example, so that they have a way to tell, OK, this text is AI-generated because it carries this watermark. And this watermark is invisible to users, so only they can decode it and find out whether the text has their watermark.

为什么只有他们能做到？我们稍后在这个（希望不会太长的视频）中会谈到，但一次只讲一件事。

Why can only they do it? We will get to that later in this (hopefully not too long a video), but one thing at a time.

LLM 文本生成的工作原理 How LLM Text Generation Works

因此，我想先简要介绍一下 LLM 中文本生成的工作原理，因为基于这一点，我们就能更容易地理解水印是如何工作的，并且这实际上并不是一个庞大且昂贵的东西。它真的只是常规文本生成过程中的一个小调整。

So I wanted to start with a brief prelude to explain how text generation works in LLMs, because based on that we can then more easily understand how the watermarking works and that this is actually not a huge, expensive thing on top of it. It’s really just like a minor, I guess, tweak inside the regular text generation process.

例如，当我们使用类似 ChatGPT 这样的工具时，假设我问一个问题：“德国的首都是”，那么 ChatGPT 或其他 LLM 就会回答“柏林”。所以在这里，它生成了两个 token，比如“柏林”和句号。但为了简单起见，我们假设它生成一个 token。那么下一个 token 就是“柏林”这个 token。这个 token 在内部是如何生成的呢？

So when we are using something like ChatGPT, for example, let’s say I ask the question, the capital of Germany is, and yeah, ChatGPT or other LLMs, so this is just like an example would, for example, answer “Berlin”. So here, in this case, it’s generating two tokens, like “Berlin” and the period. But for simplicity, let’s assume it’s generating one token. So the next token is the “Berlin” token. How is this token generated internally?

当我们在输入框中输入类似“德国的首都是”这样的内容，并收到一个像“柏林”这样的 token 时，底层发生了什么？幕后究竟发生了什么？

What is happening under the hood when we type something here like the capital of Germany is and receive a token like “Berlin” back? What is actually going on there behind the scenes?

因此，在接下来的几张幻灯片中，我想简要谈谈当生成下一个 token 时底层发生了什么。

So in the next couple of slides, I want to briefly talk about what happens under the hood when this next token is generated.

再次假设我们的提示是“德国的首都是”。这里的第一步是将其转换为 token ID。因此，对其进行 token 化并转换为 token ID 是开始阶段的主要步骤之一。这是在外部进行的。它不在 LLM 内部，而是在 LLM 外部。所以我们只是将文本转换为 token ID。这只是一种嵌入层可以处理的格式。

So assume again that our prompt is the capital of Germany is. And the first step here is to convert this into token IDs. So tokenizing it and converting it into token IDs is one of the main steps at the beginning. This is outside. It’s not inside the LLM; it’s outside of the LLM. So we are simply converting the text into token IDs. It’s just a format that embedding layers can work with.

然后这通过 LLM。LLM 给我们下一个词的分数分布。

And then this passes through the LLM. And the LLM gives us a score distribution for the next token.

所以再次，这只是 LLM 内部工作原理的简要概述。所以我不涵盖 LLM 机制本身。我在其他 From Scratch LLMs 视频和书中多次谈到过。重要的是，当我们生成下一个词（例如，“Berlin”）时，此时我们有一个分数分布。所以这是 LLM 产生的输出。

So again, this is just like a brief overview of how LLMs work internally. So I’m not covering the LLM machinery itself. I talked about it many times in my other From Scratch LLMs videos and books. The important part is that when we generate the next token (for example, “Berlin”), we have, at this point, a distribution of scores. So this is the output produced by the LLM.

在这种情况下，我们看的是 logit 值。所以这些只是从负无穷到正无穷的分数，像一个分数范围。这里有一个例子，范围大约从 -8 或 -9 到 20。我们可以将这些转换为概率分布，但技术上，根据我们如何采样，这并非严格必要。但所以你可以将 logit 值视为原始分数。

Here in this case, we’re looking at logit values. So these are just scores from minus infinity to plus infinity, like a range of scores. Here’s an example, ranging from about -8 or -9 to 20. We could convert these into a probability distribution, but technically, it’s not strictly necessary depending on how we sample. But so you can think of the logit values as the raw scores.

而原始分数覆盖整个词汇表。

And the raw scores go over the entire vocabulary.

这意味着 LLM 可能生成的每个可能的词。现在这里，在词汇索引中，某个值（索引位置 19,846）获得最高分。所以我展开了分布。如果你通过 LLM 运行这个提示，你甚至会看到更极端的情况：一切都非常非常非常接近零。而“Berlin”可能会更高得多。

That means every possible word that the LLM could generate. Now here, in the vocabulary index, a certain value (index position 19,846) receives the highest score. So I spread out the distribution. If you would run this prompt through an LLM, you would even see something more extreme: that everything is, like, very, very, very close to zero. And “Berlin” would probably be much, much higher even.

但为了展示几个，你知道，像这里的峰值，让它看起来更有趣一些，我稍微放大并展开了分布。现在这里，“Berlin”得分最高，因为你可以认为它是最可能的或最合理的下一个词，如果我有一个非常具体的提示词。所以其他的，我的意思是，可能是像 Hamburg 或 Munich 这样的，LLM 可能会猜错。

But just to show you a few, you know, like peaks here so it looks a bit more interesting, I kind of zoomed in and spread out the distribution a bit. Now here, “Berlin” is the highest score because you can think of it as the most, I guess, probable or plausible next token if I have a very specific prompt like this. So the other ones, I mean, it could be something like Hamburg or Munich that the LLM might guess incorrectly.

但如今，LLM 应该相当确定“Berlin”是这里的正确答案。你在这里还看到了词汇索引。所以那是整个词汇表。如今，LLM 有大约 250,000 个可能的词元作为输出。我在这里从 19,800 截断到 19,900，因为这张幻灯片上空间有限。如果我有一个非常现实的 250,000 词的词汇表，一切都会非常狭窄，我们几乎无法在这个分布上分辨或看到任何东西。

But nowadays an LLM should be fairly certain that “Berlin” is the correct answer here. You are also seeing here the vocabulary index. So that’s like over the whole vocabulary. Nowadays, LLMs have like 250,000 possible tokens as output. I’m truncating it here from 19,800 to 19,900 because there’s just so much space here on this slide. If I would have a very realistic vocabulary of 250,000 words, everything would be so narrow that we would barely even be able to tell or see anything on this distribution.

所以这只是为了教学目的而截断的。重要的一点是，在常规文本生成中，我们得到这个分数分布。现在，我们所做的是查看最高分。

So this is just truncated for educational purposes. The important point is that in regular text generation, we get this score distribution. Now, what we do is look at the highest score.

我稍后会详细说明这是如何选择的。所以它不一定是精确的最高分，但为了简单起见，假设我们取这里的最高分。在这个例子中，是 19,846。

I will get into more detail later on how this is selected. So it’s not necessarily precisely the highest one, but for simplicity, assume we are taking the highest score here. And in this case, it’s 19,846.

然后这个分数被反词元化，我们得到“Berlin”。所以这就是这张幻灯片上的过程：从输入提示词到转换为词元 ID 和词元化，传递给 LLM，得到这个分数分布，获取下一个词元，并将其转换回文本。

And this score is then detokenized, and we get “Berlin” back. So that is the process here on this slide: from an input prompt to conversion into token IDs and tokenization, passing it to the LLM, getting this score distribution, getting the next token, and converting it back into text.

然后这段文本会被追加到输入中。因此，如果一个问题需要多个输出 token，我们就会在这个循环中持续进行，直到答案完整为止。这通常意味着 LLM 会生成一个文本结束（end-of-text）token，例如这里所示。为简单起见，我只展示了一次迭代，即生成一个 token 的过程。但正如我所说，它会以这种方式继续下去，也就是我们把修改后的输入再反馈给 LLM，进行下一轮。

And then this text is appended to the input. So if we have a question that requires multiple output tokens, we keep going in this loop until the answer is complete. That usually means that the LLM generates an end-of-text token, for example, here. For simplicity, I'm showing you only one iteration where it generates one token. But yeah, as I said, it would kind of continue like that, where we are feeding back the modified input to the LLM for the next round.

那么，我们实际上是如何在这里采样下一个 token 的呢？

Now, how do we actually sample this next token here?

下一个词采样如何工作 How Next-Token Sampling Works

我简要说过，技术上我们可以直接选择最高的那个，即得分最高的。这称为贪婪解码。这是一种方法。但大多数 LLM，比如你使用它们时，它们不会进行贪婪解码，总是选择最高的。因为如果你在其他提示上问它，可能并不总是希望得分最高，因为那样它会记住训练数据。

I briefly said, well, we could just technically select the highest one, the one with the highest score. This is called greedy decoding. That's one way to do it. But most LLMs, like if you use them, they don't do greedy decoding where they always pick the highest one. Because if you ask it on some other prompt, it might not be what we want to always have the highest score, because then it would memorize the training data.

它总是会给出相同的回应等等。所以我们实际上常常希望输出有一些变化，但不要太多以至于生成随机的东西。所以它的工作方式是，当我们从这个分布中采样时，我们通常首先将其转换为概率分数。

It would always kind of give the same response and so forth. So we actually often want some variation in the outputs, but not so much that it generates random stuff. So how it works is that, when we sample here from this distribution, we first typically convert it into probability scores.

所以这里我只是在图中展示了这些分数。为简单起见，我使用 NumPy；无论你使用什么工具（例如 PyTorch），相同的概念都适用。但假设我们在这里的 NumPy 中有这些分数。那么我会做的是计算 softmax。技术上，我会使用在 Torch 或 PyTorch 中实现的 softmax 函数，例如，该函数对于大值和小值（包括非常高的正值、非常低的正值和非常高的负值）都是数值稳定的。

So here I just have these scores shown in this plot. I'm just using NumPy for simplicity; whatever tool you use (e.g., PyTorch), the same concepts apply. But let's assume we have the scores here in NumPy. So what I would do is I would compute the softmax. Technically, I would use a softmax function implemented in Torch or PyTorch, for example, that is numerically stable for both large and small values, including very high positive values, very low positive values, and very high negative values.

这里我只是这样写出来。那是标准的 softmax，只是为了让它更易读一些。但这里的细节并不重要。

Here I'm just writing it out like that. That's the canonical softmax, just to make it a bit more readable. But the details don't matter here.

重要的是，经过这种转换后，这里的分数，我的意思是，幻灯片上只有这么多空间，但这里的分数，它们会加起来等于一。所以它本质上就像重新归一化。所以它们会被归一化以总和为一。这就是概率转换所做的全部：softmax 转换。那么一旦我们有了这些概率，我们就可以使用随机数或类似的随机采样算法。

What matters is that after this conversion, the scores here, I mean, there's only so much space on the slide, but the scores here, they would add up to one. So it's essentially like a renormalization. So they would be normalized to sum up to one. That's all that the probability conversion does: the softmax conversion. So then once we have these probabilities, we can use a random number or, like, a random sampling algorithm.

例如，在 NumPy 中，我们可以使用 `choice` 函数或方法。这里我们向词汇索引传递一个特定的随机种子。然后，重要的是，我们将概率作为权重传递。所以，这些本质上就是：“某个 token 被选中的可能性有多大？”例如，如果“Berlin”在经过归一化步骤（即 softmax 步骤）后具有 99% 的概率，而其他所有 token 加起来有 1% 的概率，那么如果我们采样 100 次，其中 99 次我们会得到“Berlin”。

For example, here in NumPy, we could use the `choice` function or method. So this is with a specific random seed we are passing to the vocabulary indices. And then, and that's the important part, we are passing the probabilities as the weights. So, these, essentially, yeah, are like: “How likely is a certain token to be selected?” So, for example, if “Berlin”, after this normalization step, the softmax step, has a 99% probability and the other ones together have a 1% probability, then if we would sample 100 times, 99 of the times, we would get “Berlin”.

在真实的 LLM 中，例如训练良好的模型，“Berlin”的概率可能达到 99.999999 左右。因此你几乎总是采样到“Berlin”，因为在这个特定情况下，模型非常确信答案是“Berlin”。所以，这就是我们如何从这个分布中采样。有一些改进方法，如 top-k 采样或 top-p 采样，简单来说，在 top-k 采样中，我们会选择前 100 个 token，然后仅对这前 100 个（得分最高的 100 个）应用随机选择，这样我们就不会得到无意义的 token。

In realistic LLMs, for example, that are well trained, “Berlin” might receive a probability of 99.999999 or something like that. So you're almost certainly always sampling “Berlin” because it's very confident that the answer is “Berlin” in this particular case. So yeah, that is how we would sample from this distribution. There are modifications like top-k sampling or top-p sampling where, let's say, just for simplicity in top-k sampling, we would select the top 100 tokens and then apply this random choice only to the top 100, the 100 highest-scoring ones, so that we don't get nonsense tokens in there.

对于这个例子，这并不重要。我的意思是，这只是另一个需要解释的事情，所以我略过它。所以你可以假设这已经是使用 top-k 等方法得到的前 100 个 token。

For this example, it doesn't really matter. I mean, it's just like another thing to explain, so I'm skimming over this. So you can maybe assume that this is already the top 100 tokens using top-k or something like that.

例如，这里有一个例子。如果我们采样 10,000 次，而“Berlin”的概率非常高，为 99.9，我们会采样到“Berlin”9,997 次，采样到单词“Hal”两次，以及“Moh”一次。这些基本上是无意义的 token。在这种情况下，LLM 很少会产生无意义的内容，因为正如我之前提到的，我稍微展开了这个分布以使其更有趣。

And so, for example, here's an example. If we sample 10,000 times with a probability of “Berlin” being very high, 99.9, we would sample “Berlin” 9,997 times, sample the word “Hal” twice, and one “Moh”. And these are basically nonsense tokens. It rarely happens that, in this case, the LLM might produce nonsense because, as I mentioned before, I spread out this distribution a bit to make it more interesting.

真实的 LLM 很可能在 10,000 次中采样 10,000 次都是“Berlin”，因为“Berlin”的概率非常高。但这只是为了说明目的。

A real LLM would probably, 10,000 out of 10,000 times, sample “Berlin” because the probability of “Berlin” is so high. But this is for illustration purposes.

从采样到水印 From Sampling to Watermarking

现在我们简要讨论了 LLM 在底层是如何工作的，我认为这本身就是一个有趣的概念。但我之前已经谈过很多次了，所以不想让你们感到无聊。我只是想先设定一些背景，解释这种水印是如何工作的。

Now we briefly talked about how LLMs work under the hood, which I think is kind of an interesting concept in itself. But I've talked about this many times before, so I don't want to bore you. I just wanted to set up some context for now, explaining how this watermarking works.

所以，我们提到在采样时选择得分最高的词元。或者我们使用这种概率采样，这会导致大多数时候选择得分最高的词元之一。现在这里有一个没有水印的另一个例子。我稍微改变了提示。现在的提示是：今天的天气是“冷”，一个可能的答案可能是，例如，“灰色”或“阴天”。所以与“柏林”的例子相比，我会说“灰色”和“阴天”在某种程度上是可以互换的。

So, we mentioned that we select the highest-scoring token when sampling. Or we use this probability sampling, which will lead to one of the highest-scoring tokens being selected most of the time. Now here's another example without watermarking. I changed the prompt slightly. Now the prompt is: today's weather is "cold," and a possible answer could be, for example, "gray" or "overcast". So in contrast to the "Berlin" example, I would say "gray" and "overcast" kind of are interchangeable.

给定完成文本或写下一个词元的目标，它们都是这个提示的合理下一个词元。所以选择哪一个几乎就像抛硬币一样。两者之间并没有真正客观上更差的一个。所以当我们进行随机采样时，因为它们也有相对较高的分数，而且由于它们都是合理的词元，它们的分数相似地高，我们可能会得到其中一个或另一个。

They are both reasonable next tokens for this prompt, given the goal of completing this text or writing the next token. So it's almost like a coin flip which one we want to select. There is not really an objectively worse one of one or the other. So when we do the random sampling, because they also have relatively high scores and their scores are similarly high since they are both plausible tokens, we might get one or the other.

所以如果我们多次重复采样，几乎一半的时间我们会得到“阴天”，几乎一半的时间我们会得到“灰色”。这就是为什么如果你提供相同的提示，LLM 通常会得到不同的答案。如果你使用相同的提示并多次询问 LLM，你通常会得到稍微不同的答案。这是因为在某些位置，两个可能的词元几乎同样可能，所以它会选择其中一个或另一个。

So almost half of the time we would get "overcast", and almost half of the time we would get "gray" if we repeat the sampling multiple times. And that's how LLMs often end up with different answers if you provide the same prompt. If you use the same prompt and you ask the LLM multiple times, you often get slightly different answers. And that's because at certain positions, two possible tokens are almost equally likely, so it will choose one or the other.

然后这个词元会影响所有后续的词元，以此类推。

And that token would then influence all subsequent tokens, and so forth.

随机种子与确定性采样 Random Seeds and Deterministic Sampling

现在，我想简要谈谈随机数生成。例如，如果我们使用这样一个随机数生成器，它会生成一个随机数序列。如果我再运行一次，这里的数字序列就不同了。所以你可以看到，每次我们生成五个数字，它们都是不同的。如果我在这里设置随机种子，比如 1、2、3，然后多次运行，我们仍然会得到随机数，但它们现在全都一样了，对吧？

Now, I wanted to briefly talk about random number generation. So, for example, if we use a random number generator like this, it will generate a random sequence of numbers. If I run it again, the sequence of numbers is different here. So you can see every time we produce five numbers, they are different. If I set the random seed here, like one, two, three, and I run this multiple times, we still get random numbers, but they are now all the same, right?

所以它们仍然是随机的。如果我们使用随机种子，我们仍然会得到彼此不同的随机数，但它们是可复现的。因此，无论是否使用随机种子，我们都会得到随机数。但使用随机种子时，我们会得到一个可复现的数字序列。所以请记住这一点：这只是一个小的入门介绍，我们稍后就会用到这个概念。

So they are still random. If we use a random seed, we still get random numbers that are different from each other, but they are reproducible. So whether we use a random seed or not, we still get random numbers. But with a random seed, we get a reproducible sequence of numbers. So keep this in mind: this is just like a little primer, and we will use this concept in a few moments.

例如，我之前提到过，如果我们进行随机采样，可能会得到“gray”或“overcast”。现在，如果我们使用一个特定的随机种子，比如 42，我们就会总是，例如，选择“overcast”。我的意思是，这仍然是随机选择，但我们使它变得确定。在这种情况下，给定这个提示，模型将总是选择“overcast”。

So, for example, I mentioned before that we might get either "gray" or "overcast" if we randomly sample. Now, if we use a specific random seed like 42, we would always, for example, select "overcast". I mean, it's still a random selection, but we make it deterministic. In this case, given this prompt, the model will always select "overcast".

如果我们使用不同的随机种子，模型可能会选择“gray”。每次我们采样时，它都会总是选择“gray”作为下一个词。所以这仍然是随机采样，但我们基于随机种子使其变得确定。

If we use a different random seed, the model might select "gray". Every time we sample, it will always select "gray" as the next token. So it's still random sampling, but we are making it deterministic based on the random seed.

水印密钥的工作原理 How the Watermarking Key Works

那么，在水印技术中，Claude 的水印有点像设置一个随机种子的概念。但这个随机种子并不是基于某个固定数字（比如某人写下的固定数字），而是使用一个密钥，本质上类似于 API 密钥、一个秘密密钥，然后从这个密钥出发，结合前四个词，他们本质上推导出这个随机种子。

So, in watermarking, Claude watermarking is kind of like the idea that it sets a random seed. But this random seed, instead of being like a number that is fixed based on, I don't know, someone writing down a fixed number, they're using a secret key that is essentially like an API key, a secret key, and from that key, together with the four previous words, they derive this random seed essentially.

但思路是，如果我回到上一张幻灯片，就和这里一样：本质上有一个固定的随机种子，而这个随机种子总是选择相同的下一个词元。好的，所以这里不是使用随机种子 99，而是他们有一个秘密密钥，并且还利用前几个词元的信息来推导这个随机种子。但更多细节稍后再说。

But the idea is that if I go back one slide, it's the same as here: there's essentially a fixed random seed, and that random seed always selects the same next token. Okay, so instead of using random seed 99 here, for example, they have a secret key and also use information about the previous tokens to derive this random seed. But more on that later.

所以思路是，水印使得文本生成在某些位置上更加确定。例如，如果我们左侧有这些合理的文本。那么如果我有一段文本说，

So the idea is that watermarking makes the text generation more deterministic in certain positions. So, for example, if we have these plausible texts on the left side. So if I have a text that says,

今天天气寒冷且

The weather today is cold and

我可能选择“阴天”或“灰色”。而下一句可能是，

I may either pick “overcast” or “gray”. And the next sentence could be,

然后是“light”或“gentle”

and then "light" or "gentle"

然后微风“moving”或“blowing”穿过树林，街道似乎“quiet”或“still”。

And then breeze is "moving" or "blowing" through the trees, and the streets seem "quiet" or "still".

这意味着基本上我可以说“quiet”或“still”。所以文本中有某些位置，我们有的词元选择几乎同样可能，就像我们之前看到的那样。所以这意味着，如果我们——这是没有水印的情况——如果我们运行提示，或者通过 LLM 给出提示，我们有时可能会得到这个答案，有时得到那个答案，等等。

Which means basically I could say either "quiet" or "still". So there are certain positions in the text where we have token choices where they are almost equally likely, like we have seen before. So that means if we are, this is without watermarking, if we are running the prompt, or given the prompt through the LLM, we might sometimes get this answer here, sometimes this answer, and so forth.

根据位置的数量，我们这里可能有 128 种可能的答案。当然，文本越长，我们可以互换使用词语的位置就越多，组合就越多，或者输出文本就越多。所以，例如，再次，一个可能的输出文本可能是

And based on the number of positions, we might have 128 possible answers here. And of course, the longer the text, the more positions we have where we can have terms interchangeably, the more combinations, or the more output texts, there are. So, for example, again, one possible output text could be

今天的天气寒冷且阴沉。微风穿过树林，街道似乎很安静。我想我会待在家里，喝杯茶读本书。

The weather today is cold and overcast. A light breeze is moving through the trees, and the streets seem quiet. I think I’ll stay home and read a book with a cup of tea.

所以这是一种可能的文本。另一种可能的文本是

So that is one possible text. Another possible text is

今天的天气寒冷而阴沉。微风拂过树梢，街道似乎很安静。我想我会待在家里，捧着一杯茶读一本小说。顺便说一句，外面其实也在下雨。我不知道这个麦克风有多好，但在这里的语境下还挺合适的。

The weather today is cold and gray. A gentle breeze is blowing through the trees, and the streets seem still. I think I’ll stay inside and read a novel with a mug of tea. By the way, it’s also actually raining outside. I don’t know how good this microphone is, but it’s kind of a very fitting context here.

但总之，底线是你可以看到这里生成了两段非常合理的文本，而且还有更多的组合。所以它们都是合理的。没有哪一个必然比另一个更好。它们只是，你知道，细微的变化。如果我们不使用水印，我们可能会得到其中任意一个，或者就是随机的，对吧？由于随机采样，我们可能会得到这个或那个。

But yeah, the bottom line is that you can see there are two very reasonable texts here being generated, and there are more combinations. So they are all reasonable. There isn’t one that is necessarily better than the other. They’re just, you know, slight variations. And if we don’t use watermarking, we might get either one, or it’s just random, right? Because of the random sampling, we might get one or the other.

现在，如果我们固定随机种子，就像我之前提到的，例如，如果随机种子是 99，我们可能总是得到这里的这段文本。所以，使用随机种子，我们可以在某种程度上固定我们得到的答案，因为这样随机采样仍然是随机的，但它是确定性的，因为它是可复现的。那么它总是会是一样的。好了，所以那仍然是没有水印的情况，现在有了随机种子。

Now, if we fix the random seed, as I mentioned before, for example, if the random seed is 99, we might always get this text here. So, using a random seed, we can kind of fix which answer we get, because then the random sampling is still random, but it’s deterministic in the sense that it’s reproducible. It’s always going to be the same then. Okay, so that is still without watermarking, now with a random seed.

而水印本质上做的是同样的事情。现在，他们不是只使用一个简单的随机种子，而是有一个所谓的随机密钥，这个随机密钥本质上参与了文本的选择。但我们已经可以说的是，在 Claude 的博客文章中，他们说水印不应该使文本变差。如果我们看看这个机制，是的，它不会使文本变差是有道理的。

And the watermarking is essentially doing the same thing. Now, instead of just using a simple random seed, they have a so-called random key, where this random key is involved in selecting the text, essentially. But what we can already say is that, in the Claude blog post, they say the watermarking shouldn’t make the text worse. If we look at this mechanism, yeah, it makes sense why it would not make the text worse.

顺便说一句，我在这里并不是在为水印辩护，我只是想解释清楚。所以请不要迁怒于传信人。但我想说的是，对最终用户而言，水印不过是固定一个随机种子、让这种采样在某种程度上变得确定，如果这样说能讲得通的话。

By the way, I'm not defending watermarks here. I'm just trying to explain. So please don't kill the messenger here. But what I'm trying to say is that the watermarking is nothing else for the end user than fixing a random seed and making this sampling kind of deterministic, if that makes sense.

水印的应用位置 Where the Watermark Is Applied

好的，到目前为止的总结是不带水印的情况。我们经常在不使用随机种子的情况下进行采样，因为我知道大多数人甚至不使用随机种子。老实说，我不认为在 Claude 和 OpenAI 的 API 中一定能做到这一点。我知道在 Ollama 中可以做到，但我也总是遇到一些问题，因为我在我的一本书的附加材料中使用了 Ollama 来生成一些文本。

Okay, so the summary so far is without watermarking. We often sample without a random seed because I know most people don’t even use one. I honestly don’t think you can necessarily do it with the Claude and OpenAI APIs. I know you can do it in Ollama, but I also always had some problems with that because I used Ollama in one of my books for the bonus material to generate some texts.

我固定了随机种子，但它仍然不总是确定性的，等等。所以这很棘手。你的体验也可能因软件版本等因素而异。总之，在不带水印的情况下，我们有这种随机采样。在右侧带水印的情况下，我们仍然有随机采样。但除了完全随机的随机采样之外，我们还有这个水印密钥。

I was fixing the random seed, but it still wasn’t always deterministic, and so forth. So it’s tricky. Your mileage may also vary, depending on the software version and so forth. Anyways, so without watermarking, we have this random sampling. With watermarking on the right-hand side, we still have the random sampling. But in addition to just a random sampling being fully random, we have this watermarking key.

而这个水印密钥被传递给随机种子生成器，以设置特定的随机种子，使其具有确定性。但本质上非常相似，正如我提到的，从头理解事物有很多好处。现在我们知道水印本质上应用在哪里了。所以它本质上应用于采样阶段。它并没有应用于 LLM 内部，这实际上是一个很酷的知识点。

And this watermarking key is passed to the random seed generator to set a specific random seed, making this deterministic. But it’s essentially very similar, and like I mentioned, there’s a lot of benefit in terms of understanding things from scratch. And now we know essentially where this watermark is applied to. So this is essentially applied to the sampling. It’s not applied inside the LLM, which is actually cool knowledge.

因此，他们不需要为此训练一个新的 LLM。他们可以直接使用现有的 LLM，并且只需在采样阶段应用它。他们不需要重新训练任何东西或类似的事情。所以，是的，这实际上很有趣，对吧？

So they don’t need to train a new LLM for that. They can just use an existing LLM, and they just apply it at this sampling stage. They don’t have to retrain anything or anything like that. So yeah, that is actually interesting, right?

水印检测的工作原理 How Watermark Detection Works

但我们还没完全讲完。我还想谈谈我们如何理解或判断文本是否带有水印。因此，只有当我们拥有密钥时，才能检测水印。例如，如果我们有这些不同的文本，本质上，在文本生成之后，你在互联网上发现了一些随机文本（比如，你在互联网某处发现了这里的第四段文本），你想知道：这段文本有水印吗？

But we are not quite done yet. I would also like to talk about how we can understand or see whether text is watermarked. So detecting the watermark is only possible if we have access to the key. So, for example, if we have these different texts, and essentially, after the text was generated, you find some random text on the internet (for example, you find this text number four here on the internet somewhere), you want to know: is this watermarked?

嗯，这不可能知道，因为要知道的话，你需要水印密钥。你需要这个评分函数，然后你必须用评分函数对文本进行评分。然后思路是，如果分数高于某个阈值，那么文本就有水印。否则，就没有水印。但作为最终用户，我们无法这样做，因为我们没有这个密钥。所以密钥对我们不可用。

Well, it’s impossible to know because, in order to know, you would need the watermarking key. You need this scoring function, and then you have to score basically the text with a scoring function. And then the idea is that if the score is above a certain threshold, then the text is watermarked. Otherwise, it’s not watermarked. But as the end user, we can’t do this because we don’t have this key. So the key is not available to us.

只有 Anthropic 会拥有密钥。然而，在这篇博客文章中，他们提到他们当然会提供它，或者他们将为此开发一个 API 并使其可用。我不知道。老实说，我不隶属于他们。我不知道细节。我只是在这篇博客文章中读到的。这就是我所知道的。所以那个 API 也可能对某些公司私有，比如，X 或 Substack Notes，当他们想要标记 AI 生成的帖子时。

Only Anthropic will have the key. However, in this blog post, they mentioned that they are providing it, of course, or they’re going to develop an API for that that they will make available. I don’t know. Honestly, I’m not affiliated. I don’t know the details. I was just reading this in this blog post. That’s all I know. So that API might as well be private for some companies, like, let’s say, X or Substack Notes, when they want to label AI-generated posts.

他们可能会公开让最终用户使用。谁知道呢？我们将不得不等待。但是的，所以这里的底线是，只有当我们拥有这个水印密钥，或者当然，他们将要开发的 API 时，水印检测才可能实现。

They may make it public for end users to use. Who knows? We will have to wait on that. But yeah, so the bottom line here is that watermark detection is only possible if we have this watermarking key or, of course, the API that they are going to develop.

如何去除水印 How to Remove a Watermark

现在，去除水印很有趣。既然我们知道了水印是如何工作的，也就知道了其缺点。我的意思是，这高度依赖于特定位置的特定 token。例如，在给定文本中，如果这些彩色单词或 token 是水印位置，我们知道可以通过编辑来去除水印，对吧？如果我们更改这些位置上的所有单词，就能 100% 击败这个水印。

Now, removing the watermark is interesting. So now that we know how the watermarking works, we also know the shortcomings. I mean, this is really highly dependent on specific tokens in certain positions. So, for example, in this given text, if these colored words or tokens are the watermarking positions, we know that we could remove the watermark by editing this, right? If we change all the words at these positions, we would be 100% able to defeat this watermark.

然而，问题在于我们并不知道，对吧？

Now, the problem, though, is that we don’t know, right?

我们不知道这些单词在哪里，因为我们没有生成水印。所以我们不知道要查看哪些位置。这里的实际场景是，我们可以随机编辑文本。我们会随机更改几个单词，并希望更改足够多的位置来编辑水印。这是一种去除水印的方法。而且，我们也不知道哪些是得分最高的词，因为那需要访问 LLM 并重新通过 LLM 运行提示来找出哪些词得分最高，我们只能猜测。

So we don’t know where these words are because we haven’t generated the watermark. So we don’t know which positions to look at. So the practical scenario here is that we could just randomly edit the text. So we would randomly change a few words and hope that we change enough positions to edit the watermark. So that would be one way to remove it. And since we also don’t know which are the highest-scoring ones, because that would require us to have access to the LLM and rerun the prompt through the LLM to find out which words are the highest-scoring, we can kind of only guess.

例如，我们可能会说，哦，我们把“overcast”替换为“cloudy”，因为我们不知道“gray”得分很高，你明白吗？所以在这种情况下，说“gray”可能很直观，但有些情况可能不那么直观。我想说明的只是一些通用的文本编辑，我们修改位置，但仍然在猜测水印位置是什么。

So for example, we might say, oh, we replace “overcast” with “cloudy” because we don’t know that “gray” was high-scoring, you know? So in this case, it might be intuitive to say “gray”, but there might be cases where it’s not so intuitive. So what I’m trying to illustrate here is just some general text editing where we are modifying positions, but we are still kind of guessing what a watermark position is.

由于我们不知道，我们只是在这里那里添加几个词。如果我们添加足够的词，也能击败水印。

So since we don’t know, we added just a few words here and there. And if we added enough words, that would also defeat the watermark.

水印评分函数的工作原理 How the Watermark Scoring Function Works

所以，这就是水印技术的简要概述。我提到过，有一个评分函数可以用来判断某内容是否带有水印。我想在这里作为额外内容来讲解一下。视频已经很长了，但作为额外内容，我想简要解释一下这个评分函数的工作原理，因为这也是有趣的信息。它有点复杂。理解评分函数的工作原理并不是必需的。

So yeah, that was the watermarking in a nutshell. I mentioned that there is a scoring function to find out whether something is watermarked. And I want to do it as a bonus here. It’s already a long video, but as a bonus here, I wanted to briefly also explain how this scoring function works because that is also interesting information. It’s a bit complicated. It’s not essential to understand how the scoring function works.

但他们采用特定方式的原因是为了降低检测成本。因为否则，如果我回到上一张或上两张幻灯片，如果你想检查某内容是否带有水印，甚至他们自己想检查，他们都必须重新运行提示来获得这些分数，然后应用这个水印随机种子来得到文本，再进行对比。那将非常昂贵，因为本质上，对于每一个你想对比的文本，你都必须重新运行 LLM。

But the reason why they do it the certain way they do is to make the detection cheaper. Because otherwise, if I go back one slide or two slides, if you wanted to check if something is watermarked, if even they wanted to check, they would have to rerun the prompt to get these scores and then apply this watermarking random seed to get this text and then compare. And that would be very expensive because then essentially every text you want to compare, you would have to rerun the LLM.

你必须知道是哪个 LLM，而这将非常不可行，因为你通常甚至不知道提示是什么，对吧？所以，他们使用了一个技巧，可以说，修改采样过程，这样你之后在评分阶段就不使用或不需要 LLM 了。在博客文章中，他们提到这个方法源自一篇论文。

You have to know which LLM, and that would be really unfeasible because you often also don’t even know the prompt, right? So yeah, so they have like a trick that they use to, yeah, I would say, modify the sampling so that you don’t use or don’t need the LLM later on for the scoring stage. And in the blog post, they mentioned that they derived this method from a paper.

那是一篇《自然》论文，这个方法叫做 SynthID-Text。所以那是一篇大约一两年前发表的论文。它来自谷歌，他们使用了一种类似的技术，他们称之为 Claude 水印。我不知道，抱歉，我不知道他们是否使用了完全相同的技术，但那就是他们提到的技术。

It was a Nature paper, and this method is called SynthID-Text. So that was like a paper that came out maybe one or two years ago. It was by Google, and they use a similar technique they call Claude watermarking. I don’t know, sorry, I don’t know if they use exactly that technique, but that’s the one they mentioned.

SynthID Text 与锦标赛采样 SynthID Text and Tournament Sampling

那么它是如何工作的呢？在我们看幻灯片之前，我们先看了这里的常规概述，其中我们有一些文本。我们将其输入 LLM。我们得到这个 logit 分布，然后从分布中采样并得到输出 token。而在这里，在采样过程中，我们使用了水印密钥和随机种子生成器。所以这仍然是正确的。这仍然是正在发生的事情，但我想这个 token 是如何被采样的，还有更多的细微差别。

So how does it work? Before we looked at the slides, we looked at the regular overview here, where we have some text. We put it through the LLM. We get this logit distribution and then we sample from the distribution and get the output token. And here, during the sampling, we use the watermarking key and the random seed generator. So this is still correct. This is still what's going on, but there is a bit more nuance, I guess, to how this token is sampled.

所以他们不只是使用，比如说，NumPy 的随机选择。他们在这里使用了更复杂的东西。

So they're not just using, let's say, NumPy's random choice. They're using something a bit more sophisticated here.

所以再次假设，我们的上下文是“今天天气很冷”，我们想要生成下一个 token。例如：“gray”、“overcast”、“gloomy”、“cloudy”。“Gray”的概率是 50%，“overcast”是 30%，“gloomy”是 15%。假设“cloudy”是 0.05，其余的是 0。这里看起来当然有点不同。假设那是“gray”和“overcast”。我只是重复使用这个图。但现在想象这些是最可能的，比如“gray”和“overcast”，而其他一切都很小，也许除了“gloomy”和“cloudy”。

So assume, again, our context is "the weather today is cold," and we want to generate the next token. So, for example: "gray", "overcast", "gloomy", "cloudy". "Gray" is 50% probability, "overcast" is 30, "gloomy" is 15. Let's say "cloudy" is 0.05 and the rest is, let's say, 0. Here it looks, of course, a bit different. Let's say that's "gray" and "overcast". I'm just reusing this figure. But now imagine these are the most likely ones, like "gray" and "overcast", and everything else is just very small, except "gloomy" and "cloudy," maybe.

所以本质上，为了更简单，在这个例子中只考虑一个非常小的词汇表，四个词，而不是这里的 50 个词。现在，正如我之前提到的，我们可以使用 NumPy 的随机选择，用这些概率来采样下一个 token。

So essentially, think about just a very small vocabulary for this example of four words instead of all these 50 words here, just to make it even simpler. Now, as I mentioned before, we could use NumPy's random choice with these probabilities to sample the next token.

并且我们可以使用水印密钥和这个随机种子生成器来使其确定，并得到我们想要的特定水印。但正如我之前提到的，这将非常昂贵。不是采样本身。那没关系。这相当便宜。但如果我们试图检查互联网上的随机文本，后来的检测将非常昂贵。

And we could use the watermarking key with this random seed generator to make it deterministic and get the certain watermark that we want. But as I mentioned before, this would be very expensive. Not the sampling itself. That doesn't matter. This is pretty cheap. But the detection later on would be very expensive if we are trying to check random text on the internet.

因此，他们改用另一种方式——在采样过程中、即生成过程中也使用它，以便之后在检测阶段可以复用。他们所用的方法称为锦标赛采样（tournament sampling）。也就是说，他们不使用类似随机选择的做法，而是采用一种称为锦标赛采样的概念。那么它是如何工作的呢？它看起来可能有点复杂，但说实话，它看起来比实际要复杂得多。

So instead, what they use—they also use it during the sampling, during the generation, so that it can be reused later during detection. What they use is called tournament sampling. So instead of using something like random choice, they use a concept called tournament sampling. And so how does that work? It might look a bit complicated, but it looks really more complicated than it really is, to be honest.

所以你可能需要在某个时刻暂停视频，对着图稍微琢磨一下。但我认为它实际上比看起来要简单。一旦你掌握了窍门，它就相当直接。不过让我试着在这里解释一下。我们所拥有的是，我们仍然有这个上下文，然后我们有这些可能的或看似合理的下一个词元，以及它们各自不同的概率。

So you might have to, I guess, stop the video at some point and just sit with the figure a bit. But I think it is actually simpler than it looks. It's like once you get the hang of it, it's pretty straightforward. But let me try to explain here. So what we have is we still have this context, and then we have these probable or plausible next tokens with these different probabilities.

现在他们有一种东西，称为随机水印函数（random watermarking functions）。

Now they have something they call random watermarking functions.

随机水印函数 Random Watermarking Functions

这里我们有三个水印函数：G1、G2 和 G3。实际上，它们可能有 30 个、50 个甚至更多。这里我只用三个，因为这样在这张幻灯片上更简单。它更小，你知道，就像更适合放在幻灯片上。现在，如果我们看“gray”这个词，这可能会给我们一个签名 101。我的意思是，如果我们使用这个水印密钥来生成这个随机种子，并且我们有三个函数 G1、G2、G3。

Here we have three watermarking functions, G1, G2, and G3. In reality, they might have 30, 50, or even more. Here I’m just using three because that is simpler on this slide. It’s just smaller, you know, like it fits better on the slide. Now, if we look at this word “gray”, this might give us a signature 101. With that, I mean, if we use this watermarking key to generate this random seed, and we have three functions, G1, G2, G3.

如果我放入“gray”这个词，我在这里跳过的是，通常你会把“gray”这个词与上下文中前面四个或三个词放在一起。所以是“cold”和“gray”。如果我把这个和这个水印密钥一起放入 G1，我得到值 1。为什么？嗯，这个函数就是这样工作的。它就像一个随机函数。随机函数要么返回 0 要么返回 1。在这种情况下，用这个随机密钥和这个 token，它返回 1。

If I put the word “gray”, what I’m skipping here is that usually you put the word “gray” together with the four or three previous words from the context. So it’s “cold” and “gray”. If I put that into G1 together with this watermarking key, I get the value one. Why? Well, that’s just how this function works. It’s like a random function. The random function either returns zero or one. In this case, with this random key and this token, it returns one.

使用相同的密钥，但不同的函数，你得到值 0。然后这里你又得到一个 1。所以如果我们有更多的函数（当然，30 个函数），这将是一个非常长的 1 和 0 的字符串。

With the same key, but a different function, you get the value zero. And then here you get a one again. So if we have more functions (of course, 30 functions), this will be a very long string of ones and zeros.

它基本上就像一个位串，就像你有 0 和 1 的位。好的。所以这是对于“gray”。所以我们通过使用这些水印函数得到签名 101。现在我们可以对所有其他的做同样的事情。所以我们可以对“gray”做。我们可以对“overcast”、“gloomy”和“cloudy”做。所以每一个在这里有一个不同的签名。例如，“overcast”是 0, 1, 0。

It’s basically like a bit string, like if you have bits of zeros and ones. Okay. So this is for “gray”. So we get the signature 101 through using these watermarking functions. Now we can do the same thing for all the other ones. So we can do it for “gray”. We can do it for “overcast”, “gloomy”, and “cloudy”. So each one has a different signature here. So, for example, “overcast” is zero, one, zero.

“Gloomy”有 0, 0, 1。“Cloudy”有 1, 0, 0。好的。所以我们现在有了这些位。下一步是所谓的锦标赛采样，我们只是将它们配对。

“Gloomy” has zero, zero, one. “Cloudy” has one, zero, zero. Okay. So we have these bits here now. The next step is a so-called tournament sampling where we just pair them.

锦标赛采样逐步解析 Tournament Sampling Step by Step

就像你知道的，像足球锦标赛的淘汰赛阶段，或者像美式橄榄球的季后赛，总是有两支队伍相互对抗。这有点类似同样的思路。我们有一对词元，它们本质上是在相互竞争。而分数来自这里的这些函数。所以我们在第一轮从第一个函数开始。于是我们有“cloudy”和“gray”。

Like, you know, like a soccer tournament, the knockout (KO) stages, or like the playoffs in American football, you always have two teams playing against each other. And that's kind of like the same idea. We have a pair of tokens, and they're playing against each other, essentially. And the scores, they come from these functions here. So we start with the first function in the first round. So we have "cloudy" and "gray".

所以我们在这里查找：“gray”是 1，“cloudy”是 1。好的。所以是 1 和 1。“Overcast”和“gray”。那么“overcast”是 0，“gray”是 1。所以我们有 0 和 1。“Gloomy”和“overcast”。这里“gloomy”是 0，“overcast”是 0。所以是 0 和 0。然后我们又有“gray”和“gray”，因为其他词不够了。所以有一个重复。

So we look up here: "gray" is a one and "cloudy" is a one. Okay. So one and one. "Overcast" and "gray". So "overcast" is zero, "gray" is one. So we have zero, one. "Gloomy" and "overcast". So here we have "gloomy" zero, "overcast" zero. So zero, zero. And then we have "gray" and "gray" again, because we are running out. So we don't have enough of the others. So we have one duplicate.

所以这是随机选择的。这里你得到 1 和 1。现在我们看结果。这是平局。在平局的情况下，我们也使用随机种子和水印密钥进行随机选择。所以这里，“cloudy”胜出。而从这个，根据 G1，“gray”是赢家，因为它有 1。所以“gray”胜出。然后这里，“overcast”和“gray”是平局，随机选择，而“gray”也被随机选中。

So this is chosen randomly. And so you have one and one here. Now we look at the results. So this is a tie. In the case of a tie, we also select randomly using, you know, the random seed and the watermarking key. So here, "cloudy" survives. And from this one, G1 is, according to G1, "gray" is the winner because it has the one. So "gray" survives. And then here, "overcast" and "gray" are a tie, randomly selected, and "gray" also randomly selected.

所以我们现在有“cloudy”和“gray”以及“overcast”和“gray”。我们进行这个锦标赛的下一轮。在下一轮中，我们使用 G2。根据 G2，“cloudy”在这里是 0。“Gray”也是 0。“Overcast”是 1。而“gray”也是 0，抱歉。于是，再次进入锦标赛的下一阶段。

So we have now "cloudy" and "gray" and "overcast" and "gray". And we play the next round in this tournament. So in this next round, we use G2. So according to G2, "cloudy" has a zero here. "Gray" also has zero. "Overcast" has one. And "gray" also has zero, sorry. And so, the next stage of the tournament again.

所以我们有一个平局。我们随机选择“gray”。而这里“overcast”是赢家。因此我们在决赛中有“gray”对“overcast”。然后我们再次看分数。“gray”是 1。“Overcast”是 0。所以“gray”是赢家。这就是词元“gray”被采样的方式。水印密钥在这里起什么作用？水印密钥，如果我往回翻几页，是用来通过这些随机水印函数生成这些分数的。

So we have a tie. We randomly select "gray". And here we have "overcast" as the winner. And so we have "gray" versus "overcast" in the final. And then we look again at the scores. So "gray" has a one. "Overcast" is a zero. So "gray" is the winner. And that's how the token "gray" is sampled. What is the watermarking key doing here? So the watermarking key, if I go back a few slides, is selected for generating these scores using these random watermarking functions.

因此，水印密钥本质上决定了我们在这些阶段得到什么值。因此，水印密钥仍然非常重要；否则，这些签名看起来会有所不同。

So the watermarking key essentially determines what values we get at these stages. The watermarking key is therefore still very important; otherwise, these signatures would look different.

无需重新运行 LLM 的水印检测 Detecting Watermarks Without Rerunning the LLM

现在我们已经采样了下一个词。这就是这个修改后的采样过程的工作方式。我们本可以使用 NumPy 的 random.choice。但其缺点是，如果我们想对互联网上的随机文本进行评分，我们将不得不重新运行 LLM。而使用这种技术，我们不需要。我稍后会展示给你看。所以这种技术听起来真的很奇怪和繁琐，但它的优点是，我们现在可以更容易地对随机文本进行评分，而无需重新运行 LLM。

So we now have sampled the next token. And that's just how this modified sampling procedure works. We could have used NumPy's random.choice. But the shortcoming of that is that if we want to score random text on the internet, we would have to rerun the LLM. With this technique, we don't. I will show you in a moment. So this technique sounds really weird and cumbersome, but it has the advantage that we can now score random text more easily without having to rerun the LLM.

所以本质上只是为了使得检测更容易和更便宜。

So it's essentially just to make the detection easier and cheaper.

例如，如果我们有一个新文本。这里我只是使用相同的文本，但假设它是新文本。这是在采样之后，当我们进行评分时。假设我们在互联网上发现这个文本。文本是“今天天气冷且灰暗”，我们想知道这是否是 LLM 生成的。因此，我们，或者 Claude/Anthropic，将拥有水印密钥和这些函数：G1、G2 和 G3。

So, for example, if we have a new text. So I'm just using the same text here, but let's assume it's new text. So this is after the sampling, when we are scoring. And let's say we are discovering this text on the internet. And the text is "the weather today is cold and gray", and we want to know if this is LLM-generated or not. So we would, or Claude/Anthropic would, have the watermarking key and these functions: G1, G2, and G3.

然后它将把这个文本通过这些函数。对于“灰暗”这个位置，我们会得到 101，类似于我们在生成过程中得到的。所以这和之前一样。如果我们把这些比特加起来，两个比特，对吧？这里一个 1 和一个 1。所以它有两个比特的信息，简单来说。这只是一个非常简单的说明。

And it would put this text through these functions. For the one position here for "gray", we would get 101, similar to what we got during the generation process. So this is the same as before. And this has, if we add up these bits, two bits, right? One and one here. So it has two bits of information, let's say, for simplicity. This is just a really simple illustration.

但假设我们在这个位置对“灰暗”得到分数 2。如果在这个位置我们有一个不同的词，“阴天”，我们会得到 1；如果得到“阴沉”，我们也有 1；而“多云”也是 1。所以我只是对每一行求和，对吧？所以这就像我们在每个位置得到的分数。而这里我只看了最后一个位置。

But let's assume we get a score of two here for the "gray" in this position. If we had a different word here, "overcast", in this position, we would get one; if we get "gloomy", like we also have one; and "cloudy" one. So I'm just summing over each row here, right? So that's just like a score we would get at each position. And here I'm only looking at the last position.

如果我在其他位置这样做，我会在不同位置得到不同的分数。例如，假设在第一个位置我得到 2 分。这里我得到 2 分。对于“today”，我得到 3 分。对于“is”，我得到 2 分。“Cold”得 2 分。“gray”得 3 分。所以这里我应用了上一张幻灯片中展示的水印函数。

If I would do this at other positions, I would get a different score at different positions. So, for example, let's assume at the first position I get a two. Here I get a two. For "today", I get a three. For "is", I get a two. "Cold", two. And "gray", three. So here I'm applying these watermarking functions as I've shown on the previous slide.

我只是在三个函数中将这些数字相加。水印函数非常廉价。因此你可以快速地在整个文本上运行它们并得到这些分数。然后基于此，我可以计算平均比特数。所以如果我对这里的所有值取平均，假设我得到 2.23。

And I'm just adding up these numbers across the three functions. And the watermarking functions are very cheap. So you can just quickly run them on the whole text and get these scores. And then based on that, I can compute the average bits. So if I just average over all these values here, let's say I get 2.23.

现在，如果我有一个稍微不同的文本，这里我把“today”换成了“now”，把“gray”换成了“overcast”。这些现在得到 1 分和 1 分。如果我对整个字符串取平均，那么我得到 1.71。因此，我不需要 LLM。我只需要水印密钥、随机种子生成器以及这些函数 G1、G2 和 G3。这就是我所需要的全部。

Now, if I have slightly different text, so here I swapped "today" with "now" and "gray" with "overcast". These now get a score of one and one. And if I average over this whole string, then I get a 1.71. And so for that, I don't need an LLM. All I need is the watermarking key, the random seed generator, and these functions, G1, G2, and G3. And that's all I need.

我不需要 LLM。我可以得到这个分数。他们所做的就是应用一个阈值。

I don't need the LLM. And I can get this score here. And what they do is apply a threshold.

例如，我的意思是，他们不使用这个确切的阈值。但例如，我们可以说如果分数大于 2，那么文本是水印的。如果分数小于 2，则不是水印的。所以这里，如果分数大于 2，就是“是”。所以是的，这是水印的。在这种情况下，1.71 不大于 2。所以这个文本不是水印的。好的。这就是我们检测互联网上随机文本是否带有水印的方法。

So, for example, I mean, they don't use this exact threshold. But for example, we can say if the score is greater than two, then the text is watermarked. If the score is smaller than two, it's not watermarked. So here, if the score is greater than two, it's a yes. So yes, this is watermarked. In this case, 1.71 is not greater than two. So this text is not watermarked. Okay. So that's just the way we can then detect whether random text on the internet is watermarked or not.

本质上就是应用这些水印函数，然后对分数取平均并施加一个阈值。好的。

It's essentially just applying these watermarking functions and then averaging over the scores and applying a threshold. Okay.

所以，锦标赛采样主要是为了让检测更容易、成本更低。我们也可以使用类似 NumPy 的带随机种子的随机选择来使采样具有确定性。但话又说回来，那样的话就很难对互联网上的任何文本进行评分了。

So again, the tournament sampling is mainly to make detection easier and cheaper. We could also use something like NumPy's random choice with a random seed to make the sampling deterministic. But then again, it would be hard to score any text on the internet.

水印技术总结与局限性 Watermarking Summary and Limitations

所以，总结仍然是一样的。不过，无水印和水印之间的区别在于，我们用水印密钥控制这里的采样。在那里面，我们有这种锦标赛采样。而且，正如我之前提到的，检测水印需要秘密密钥和水印函数 G1 到 Gn。

So yeah, the summary is still the same, though. The thing that is different between no watermarking and watermarking is that we are controlling this sampling here with the watermarking key. And inside that, we have this tournament sampling. And yeah, as I mentioned before, detecting the watermarks requires the secret key and the watermarking functions G1 to Gn.

再者，去除水印——因为我觉得这可能对某些人来说很有趣——理想情况下需要编辑这里的所有位置。但既然我们不知道哪些位置有水印，而且在内部，他们选择那些位置使得在这些位置上词元具有相等的可能性。可能有些位置并非如此。所以这里，例如对于“trees”，我们甚至可能没有高分的替代词，因此他们不对那个位置加水印。

And again, removing the watermark, because I think that's maybe interesting to some people, would ideally involve editing all the positions here. But since we don't know which positions are watermarked and internally, they choose the positions so that they have equally likely tokens at those positions. And there might be positions where that's not true. So here, for example, for “trees”, we might not even have an alternative word that is high scoring so they don't watermark that position.

所以他们本质上只在某些位置进行水印。既然我们不知道哪些位置需要击败或去除水印，我们将……

So they only do the watermarking at certain positions essentially. Since we don't know which positions to kind of defeat or remove the watermark, we would...

……不得不编辑文本中的几个地方。所以我认为这对 AI 生成文本的未来意味着，这实际上……

...have to edit several places in the text. So what I think that means for the future of AI-generated text is that this actually...

这对 AI 生成文本意味着什么 What This Means for AI-Generated Text

……可能导致更差的 AI 生成文本。所以我认为，如果有人在互联网上到处使用 AI 生成的文本，比如说有个新闻网站喜欢用 AI 生成的文本写新闻，我不认为水印能阻止他们这么做。他们可能仍然想要生成 AI 文本，因为那是他们工作流程的一部分。所以我的猜测是他们会用另一个模型。

…might result in worse AI-generated text. So I think if there's a person who likes to use AI-generated text everywhere on the internet, let's say there's a news website that likes to use AI-generated text to write the news, I don't think watermarking will necessarily stop them from doing that. They will probably still want to generate AI-generated text because that's part of their workflow. So I think my guess is that they'll use another model.

他们只会用第二个模型来编辑文本，得到所谓的编辑后的 AI 生成文本。所以这使流程复杂化了。现在不是直接从 Claude 获取文本，而是用 Claude 生成 AI 文本，将其通过一个本地模型，然后得到编辑后的 AI 生成文本，这很可能不再有水印。那么为什么用本地模型？我只是认为本地模型是因为我认为所有的提供商——专有 LLM，不仅是 Claude，还有谷歌——我的意思是，谷歌写了这篇论文，对吧？

They'll just use a second model to edit the text to get the so-called edited AI-generated text. So it's complicating the pipeline. Instead of getting the text directly from Claude, it's now using Claude to generate AI-generated text, passing it through a local model, and then having edited AI-generated text, which is likely not watermarked anymore. So why a local model? I just think a local model because I think all the providers—the proprietary LLMs, not only Claude, but also Google—I mean, Google wrote this paper, right?

所以我认为他们也在给 Gemini 文本加水印。而且我认为 OpenAI 可能已经在做了，或者也会这么做。我的意思是，我只是在推测，但我想每个人可能都会做类似的事情，因为好像有欧盟法规要求这样做。根据博客文章，这显然就是 Claude 这样做的原因。是的，所以我认为本地模型可能不会，至少目前还不会，实施这种水印。

So I'm thinking that they are also watermarking Gemini text. And I think OpenAI is probably already doing it or will do so as well. I mean, I'm just speculating, but I'm imagining everyone will probably do something like that because there's like an EU regulation that requires that. And that's, according to the blog post, apparently why Claude is doing it. Yeah, so I'm thinking local models may not, at least not yet, implement this watermarking.

所以我认为人们只会使用本地模型，然后生成编辑后的 AI 文本。我猜它会比原始文本稍差一些，因为对于本地模型，你现在可能使用的是更小的模型。所以，我的意思是，从技术上讲，你也可以直接使用本地模型来生成文本。但在我看来，编辑文本比生成文本更简单。所以对于文本的生成，你可能会使用非常昂贵的高端模型，我不知道，比如最贵、最高端的 Claude 模型来处理复杂文本。

So I think people will just use a local model and then generate edited AI-generated text. And my guess is it will be slightly worse than the original text because for the local model, you might now be using a smaller model. So, I mean, you could also technically just use the local model directly to generate text. But in my view, editing text is simpler than generating text. So for the generation of the text, you might use a very expensive high-end, I don't know, like the highest, most expensive Claude model for complicated text.

然后你使用更便宜的本地模型来进行这些外科手术式的编辑，基本上就是这样。这很可能就是将会发生的事情。为什么更差？所以如果我们回顾一下这个图形，我们只是添加了随机位置，你可能只是为了改变而改变词语。然后这就有使文本变差的风险。所以你仍然可能得到生成的文本，但它有点像被笨拙地编辑过。

And then you use a cheaper local model to make these surgical edits, essentially. That's probably what's going to happen. And why worse? So if we look back at this graphic where we just added random positions, you might be just changing words for the sake of changing them. And then it risks making the text worse. So you might still have generated text, but it's kind of like it's edited awkwardly.

最终思考 Final Thoughts

不过无论如何，我在这里的目标是解释水印是如何工作的，而不是，比如说，它在全球范围内的影响。但我希望这种幕后、深入内部的视角是有用的。水印并不像它看起来那么复杂，但我想它仍然有 52 张幻灯片，所以它也并非超级简单。所以我希望你觉得这堂小讲座有用。那么，下次见，到时候见。

But anyway, my goal here was to explain how the watermarking works, and not, let's say, the worldwide ramifications of that. But I hope this kind of behind-the-scenes, under-the-hood look is useful. The watermarking is not as complicated as it might seem, but I think it was still 52 slides, so it was also not super trivial. So I hope you found this little lecture useful. And yeah, until next time, see you then.

附言：如果你喜欢更多这种风格的讲解，我并不经常在 YouTube 上发布视频，但多年来我已经积累了 300 多个视频，你可以在我的 YouTube 频道这里找到它们。

PS: If you like more explainers in this style, I don't post videos to YouTube regularly, but I have accumulated over 300 videos over the years, which you can find on my YouTube channel here.