ChatGPT 在做什么……以及它为什么有效？

Stephen Wolfram Stephen Wolfram(https://www.stephenwolfram.com/)

* * [](https://x.com/stephen_wolfram "X")

* * [](https://x.com/stephen_wolfram "X")

* [](https://www.facebook.com/Stephen-Wolfram/ "Facebook")

* [](https://www.facebook.com/Stephen-Wolfram/ "Facebook")

* [](https://www.linkedin.com/in/stephenwolfram "LinkedIn")

* [](https://www.linkedin.com/in/stephenwolfram "LinkedIn")

* [](https://soundcloud.com/stephenwolfram/ "SoundCloud")

* [](https://soundcloud.com/stephenwolfram/ "SoundCloud")

* [](https://www.twitch.tv/stephen_wolfram "Twitch")

* [](https://www.twitch.tv/stephen_wolfram "Twitch")

* 概率从何而来？

* Where Do the Probabilities Come From?

* 机器学习与神经网络的训练

* Machine Learning, and the Training of Neural Nets

* 神经网络训练的实践与经验

* The Practice and Lore of Neural Net Training

* “一个足够大的网络当然无所不能！”

* "Surely a Network That's Big Enough Can Do Anything!"

* 意义空间与语义运动定律

* Meaning Space and Semantic Laws of Motion

* 语义语法与计算语言的力量

* Semantic Grammar and the Power of Computational Language

* 那么……ChatGPT 在做什么，它为什么有效？

* So ... What Is ChatGPT Doing, and Why Does It Work?

ChatGPT 在做什么……它为什么有效？

What Is ChatGPT Doing … and Why Does It Work?

它只是一次添加一个词 It’s Just Adding One Word at a Time

ChatGPT 能够自动生成哪怕表面上读起来像人类写的文本，这很了不起，也出乎意料。但它是如何做到的呢？为什么它能工作？我在这里的目的是粗略概述 ChatGPT 内部发生的事情，然后探讨为什么它能如此出色地生成我们可能认为有意义的文本。我一开始就应该说明，我将关注大局，虽然我会提到一些工程细节，但不会深入探讨。（而且，我所说的本质同样适用于其他当前的“大型语言模型”[LLM]以及 ChatGPT。）

That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, and unexpected. But how does it do it? And why does it work? My purpose here is to give a rough outline of what’s going on inside ChatGPT—and then to explore why it is that it can do so well in producing what we might consider to be meaningful text. I should say at the outset that I’m going to focus on the big picture of what’s going on—and while I’ll mention some engineering details, I won’t get deeply into them. (And the essence of what I’ll say applies just as well to other current “large language models” [LLMs] as to ChatGPT.)

首先要解释的是，ChatGPT 始终试图做的基本上是生成它已有文本的“合理延续”，其中“合理”指的是“人们可能会写出的内容，基于看到数十亿网页等上人们写的内容”。

The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.”

假设我们有文本“_The best thing about AI is its ability to_”。想象扫描数十亿页人类写的文本（比如在网页和数字化书籍中），找到这个文本的所有实例，然后看下一个词出现的频率。ChatGPT 有效地做了类似的事情，只是（正如我将解释的）它不查看字面文本；它寻找在某种意义上“在含义上匹配”的东西。但最终结果是它生成一个可能的下一个词的排名列表，以及“概率”：

So let’s say we’ve got the text “_The best thing about AI is its ability to_”. Imagine scanning billions of pages of human-written text (say on the web and in digitized books) and finding all instances of this text—then seeing what word comes next what fraction of the time. ChatGPT effectively does something like this, except that (as I’ll explain) it doesn’t look at literal text; it looks for things that in a certain sense “match in meaning”. But the end result is that it produces a ranked list of words that might follow, together with “probabilities”:

值得注意的是，当 ChatGPT 做类似写文章这样的事情时，它本质上只是反复询问“给定到目前为止的文本，下一个词应该是什么？”——然后每次添加一个词。（更准确地说，正如我将解释的，它添加一个“token”，可能只是一个词的一部分，这就是为什么它有时能“造出新词”。）

And the remarkable thing is that when ChatGPT does something like write an essay what it’s essentially doing is just asking over and over again “given the text so far, what should the next word be?”—and each time adding a word. (More precisely, as I’ll explain, it’s adding a “token”, which could be just a part of a word, which is why it can sometimes “make up new words”.)

但是，好吧，每一步它都会得到一个带有概率的词列表。但它实际上应该选择哪一个添加到它正在写的文章（或其他东西）中呢？人们可能会认为它应该选择“排名最高”的词（即被赋予最高“概率”的词）。但这就是一点巫术开始渗入的地方。因为出于某种原因——也许有一天我们会有科学式的理解——如果我们总是选择排名最高的词，我们通常会得到一篇非常“平淡”的文章，似乎从未“展现任何创造力”（甚至有时逐字重复）。但如果我们有时（随机地）选择排名较低的词，我们会得到一篇“更有趣”的文章。

But, OK, at each step it gets a list of words with probabilities. But which one should it actually pick to add to the essay (or whatever) that it’s writing? One might think it should be the “highest-ranked” word (i.e. the one to which the highest “probability” was assigned). But this is where a bit of voodoo begins to creep in. Because for some reason—that maybe one day we’ll have a scientific-style understanding of—if we always pick the highest-ranked word, we’ll typically get a very “flat” essay, that never seems to “show any creativity” (and even sometimes repeats word for word). But if sometimes (at random) we pick lower-ranked words, we get a “more interesting” essay.

这里存在随机性，这意味着如果我们多次使用相同的提示，我们每次可能会得到不同的文章。并且，与巫术的想法一致，有一个特定的所谓“温度”参数，它决定了使用较低排名词的频率，对于文章生成，事实证明“温度”0.8 似乎最好。（值得强调的是，这里没有使用任何“理论”；这只是实践中发现有效的方法。例如，“温度”的概念之所以存在，是因为碰巧使用了统计物理学中熟悉的指数分布，但至少据我们所知，没有“物理”联系。）

The fact that there’s randomness here means that if we use the same prompt multiple times, we’re likely to get different essays each time. And, in keeping with the idea of voodoo, there’s a particular so-called “temperature” parameter that determines how often lower-ranked words will be used, and for essay generation, it turns out that a “temperature” of 0.8 seems best. (It’s worth emphasizing that there’s no “theory” being used here; it’s just a matter of what’s been found to work in practice. And for example the concept of “temperature” is there because exponential distributions familiar from statistical physics happen to be being used, but there’s no “physical” connection—at least so far as we know.)

在我们继续之前，我应该解释一下，为了阐述的目的，我大多不会使用 ChatGPT 中的完整系统；相反，我通常会使用一个更简单的 GPT-2 系统，它有一个很好的特性，即足够小，可以在标准台式计算机上运行。因此，对于我展示的几乎所有内容，我都能包含明确的 Wolfram 语言代码，你可以立即在你的计算机上运行。（点击这里的任何图片复制其背后的代码。）

Before we go on I should explain that for purposes of exposition I’m mostly not going to use the full system that’s in ChatGPT; instead I’ll usually work with a simpler GPT-2 system, which has the nice feature that it’s small enough to be able to run on a standard desktop computer. And so for essentially everything I show I’ll be able to include explicit Wolfram Language code that you can immediately run on your computer. (Click any picture here to copy the code behind it.)

例如，以下是如何获取上面的概率表。首先，我们必须检索底层的“语言模型”神经网络：

For example, here’s how to get the table of probabilities above. First, we have to retrieve the underlying “language model” neural net:

稍后，我们将深入这个神经网络内部，并讨论它是如何工作的。但现在，我们可以将这个“网络模型”作为一个黑盒应用于我们已有的文本，并询问模型认为应该跟随的前 5 个词及其概率：

Later on, we’ll look inside this neural net, and talk about how it works. But for now we can just apply this “net model” as a black box to our text so far, and ask for the top 5 words by probability that the model says should follow:

这将结果转化为一个明确的格式化“数据集”：

This takes that result and makes it into an explicit formatted “dataset”:

以下是如果反复“应用模型”——每一步添加具有最高概率的词（在此代码中指定为模型的“决策”）——会发生什么：

Here’s what happens if one repeatedly “applies the model”—at each step adding the word that has the top probability (specified in this code as the “decision” from the model):

如果继续下去会发生什么？在这种（“零温度”）情况下，输出的内容很快就会变得相当混乱和重复：

What happens if one goes on longer? In this (“zero temperature”) case what comes out soon gets rather confused and repetitive:

但是，如果不是总是选择“顶部”词，而是有时随机选择“非顶部”词（“随机性”对应于“温度”0.8）呢？同样可以构建文本：

But what if instead of always picking the “top” word one sometimes randomly picks “non-top” words (with the “randomness” corresponding to “temperature” 0.8)? Again one can build up text:

每次这样做时，都会做出不同的随机选择，文本也会不同——如下面这 5 个例子：

And every time one does this, different random choices will be made, and the text will be different—as in these 5 examples:

值得指出的是，即使在第一步，也有许多可能的“下一个词”可供选择（在温度 0.8 下），尽管它们的概率下降得相当快（是的，这个对数-对数图上的直线对应于 _n_–1 的“幂律”衰减，这是语言一般统计的典型特征）：

It’s worth pointing out that even at the first step there are a lot of possible “next words” to choose from (at temperature 0.8), though their probabilities fall off quite quickly (and, yes, the straight line on this log-log plot corresponds to an _n_–1“power-law” decay that’s very characteristic of the general statistics of language):

那么如果继续下去会发生什么？这里是一个随机例子。它比顶部词（零温度）情况好，但充其量仍然有点奇怪：

So what happens if one goes on longer? Here’s a random example. It’s better than the top-word (zero temperature) case, but still at best a bit weird:

这是用最简单的 GPT-2 模型（2019 年）完成的。使用更新更大的 GPT-3 模型，结果更好。以下是用相同“提示”但使用最大 GPT-3 模型生成的顶部词（零温度）文本：

This was done with the simplest GPT-2 model (from 2019). With the newer and bigger GPT-3 models the results are better. Here’s the top-word (zero temperature) text produced with the same “prompt”, but with the biggest GPT-3 model:

这里是一个“温度 0.8”下的随机例子：

And here’s a random example at “temperature 0.8”:

概率从何而来？ Where Do the Probabilities Come From?

好的，ChatGPT 总是根据概率选择下一个词。但这些概率从何而来？让我们从一个更简单的问题开始。考虑一次生成一个字母（而不是单词）的英文文本。我们如何计算每个字母的概率？

OK, so ChatGPT always picks its next word based on probabilities. But where do those probabilities come from? Let’s start with a simpler problem. Let’s consider generating English text one letter (rather than word) at a time. How can we work out what the probability for each letter should be?

一个非常基础的方法是取一段英文文本样本，计算其中不同字母出现的频率。例如，下面是对维基百科上关于“猫”的文章中字母的计数：

A very minimal thing we could do is just take a sample of English text, and calculate how often different letters occur in it. So, for example, this counts letters in the Wikipedia article on “cats”:

结果相似但不完全相同（“o”在“狗”文章中更常见，因为它出现在单词“dog”本身中）。不过，如果我们取足够大的英文文本样本，最终应该能得到相当一致的结果：

The results are similar, but not the same (“o” is no doubt more common in the “dogs” article because, after all, it occurs in the word “dog” itself). Still, if we take a large enough sample of English text we can expect to eventually get at least fairly consistent results:

以下是根据这些概率生成的一串字母示例：

Here’s a sample of what we get if we just generate a sequence of letters with these probabilities:

我们可以通过将空格视为具有一定概率的字母来将其拆分为“单词”：

We can break this into “words” by adding in spaces as if they were letters with a certain probability:

我们可以通过强制“单词长度”分布与英语中的分布一致来更好地生成“单词”：

We can do a slightly better job of making “words” by forcing the distribution of “word lengths” to agree with what it is in English:

这里我们并没有恰好得到任何“实际单词”，但结果看起来稍好一些。然而，要更进一步，我们需要做的不仅仅是随机独立地选择每个字母。例如，我们知道如果有一个“q”，下一个字母基本上必须是“u”。

We didn’t happen to get any “actual words” here, but the results are looking slightly better. To go further, though, we need to do more than just pick each letter separately at random. And, for example, we know that if we have a “q”, the next letter basically has to be “u”.

以下是单个字母的概率图：

Here’s a plot of the probabilities for letters on their own:

以下是显示典型英文文本中字母对（“2-gram”）概率的图。可能的第一个字母显示在页面横向，第二个字母显示在纵向：

And here’s a plot that shows the probabilities of pairs of letters (“2-grams”) in typical English text. The possible first letters are shown across the page, the second letters down the page:

例如，我们看到“q”列除了“u”行之外都是空白（零概率）。好了，现在让我们不再一次生成一个字母的“单词”，而是使用这些“2-gram”概率一次查看两个字母来生成它们。以下是结果示例——其中恰好包含几个“实际单词”：

And we see here, for example, that the “q” column is blank (zero probability) except on the “u” row. OK, so now instead of generating our “words” a single letter at a time, let’s generate them looking at two letters at a time, using these “2-gram” probabilities. Here’s a sample of the result—which happens to include a few “actual words”:

如果有足够多的英文文本，我们不仅可以很好地估计单个字母或字母对（2-gram）的概率，还可以估计更长字母序列的概率。如果我们使用逐渐变长的 n-gram 概率生成“随机单词”，会发现它们逐渐变得“更真实”：

With sufficiently much English text we can get pretty good estimates not just for probabilities of single letters or pairs of letters (2-grams), but also for longer runs of letters. And if we generate “random words” with progressively longer _n_-gram probabilities, we see that they get progressively “more realistic”:

现在让我们假设——或多或少像 ChatGPT 那样——我们处理的是整个单词，而不是字母。英语中大约有 40,000 个相当常用的单词。通过查看大量英文文本语料库（例如几百万本书，总共几千亿个单词），我们可以估计每个单词的常见程度。利用这一点，我们可以开始生成“句子”，其中每个单词是独立随机选取的，概率与其在语料库中出现的概率相同。以下是我们得到的一个示例：

But let’s now assume—more or less as ChatGPT does—that we’re dealing with whole words, not letters. There are about 40,000 reasonably commonly used words in English. And by looking at a large corpus of English text (say a few million books, with altogether a few hundred billion words), we can get an estimate of how common each word is. And using this we can start generating “sentences”, in which each word is independently picked at random, with the same probability that it appears in the corpus. Here’s a sample of what we get:

毫不奇怪，这是无意义的。那么如何做得更好呢？就像字母一样，我们可以开始考虑不仅仅是单个单词的概率，还有单词对或更长 n-gram 的概率。对于单词对，以下是 5 个示例结果，所有情况都以单词“cat”开头：

Not surprisingly, this is nonsense. So how can we do better? Just like with letters, we can start taking into account not just probabilities for single words but probabilities for pairs or longer _n_-grams of words. Doing this for pairs, here are 5 examples of what we get, in all cases starting from the word “cat”:

它看起来稍微更“合理”了。我们可以想象，如果我们能够使用足够长的 n-gram，基本上就能得到一个“ChatGPT”——也就是说，我们会得到能够生成具有“正确整体文章概率”的文章长度单词序列的东西。但问题在于：迄今为止写下的英文文本远远不足以推断这些概率。

It’s getting slightly more “sensible looking”. And we might imagine that if we were able to use sufficiently long _n_-grams we’d basically “get a ChatGPT”—in the sense that we’d get something that would generate essay-length sequences of words with the “correct overall essay probabilities”. But here’s the problem: there just isn’t even close to enough English text that’s ever been written to be able to deduce those probabilities.

在网页抓取中可能有几千亿个单词；在已数字化的书籍中可能还有另外一千亿个单词。但有了 40,000 个常用单词，仅可能的 2-gram 数量就已经是 16 亿——而可能的 3-gram 数量是 60 万亿。因此，我们无法从现有的文本中估计所有这些概率。而当我们到达 20 个单词的“文章片段”时，可能性的数量比宇宙中的粒子数还要多，所以从某种意义上说，它们永远不可能全部被写下来。

In a crawl of the web there might be a few hundred billion words; in books that have been digitized there might be another hundred billion words. But with 40,000 common words, even the number of possible 2-grams is already 1.6 billion—and the number of possible 3-grams is 60 trillion. So there’s no way we can estimate the probabilities even for all of these from text that’s out there. And by the time we get to “essay fragments” of 20 words, the number of possibilities is larger than the number of particles in the universe, so in a sense they could never all be written down.

那么我们能做什么？核心思想是建立一个模型，让我们能够估计序列应该出现的概率——即使我们从未在查看的文本语料库中明确看到过这些序列。而 ChatGPT 的核心正是一个所谓的“大型语言模型”（LLM），它被构建来很好地估计这些概率。

So what can we do? The big idea is to make a model that lets us estimate the probabilities with which sequences should occur—even though we’ve never explicitly seen those sequences in the corpus of text we’ve looked at. And at the core of ChatGPT is precisely a so-called “large language model” (LLM) that’s been built to do a good job of estimating those probabilities.

什么是模型？ What Is a Model?

假设你想知道（就像伽利略在 16 世纪末所做的那样）从比萨斜塔的每一层扔下的炮弹需要多长时间才能落地。好吧，你可以直接测量每种情况并制作一个结果表格。或者你可以做理论科学的本质：建立一个模型，提供某种计算答案的程序，而不是仅仅测量和记住每种情况。

Say you want to know (as Galileo did back in the late 1500s) how long it’s going to take a cannon ball dropped from each floor of the Tower of Pisa to hit the ground. Well, you could just measure it in each case and make a table of the results. Or you could do what is the essence of theoretical science: make a model that gives some kind of procedure for computing the answer rather than just measuring and remembering each case.

让我们想象我们有一些（有点理想化的）数据，显示炮弹从不同楼层落下所需的时间：

Let’s imagine we have (somewhat idealized) data for how long the cannon ball takes to fall from various floors:

我们如何找出从没有明确数据的楼层落下所需的时间？在这个特定情况下，我们可以使用已知的物理定律来计算。但假设我们只有数据，而不知道支配它的底层规律。那么我们可以做一个数学猜测，比如也许我们应该使用一条直线作为模型：

How do we figure out how long it’s going to take to fall from a floor we don’t explicitly have data about? In this particular case, we can use known laws of physics to work it out. But say all we’ve got is the data, and we don’t know what underlying laws govern it. Then we might make a mathematical guess, like that perhaps we should use a straight line as a model:

我们可以选择不同的直线。但这条直线平均而言最接近我们给出的数据。从这条直线，我们可以估计任何楼层下落的时间。

We could pick different straight lines. But this is the one that’s on average closest to the data we’re given. And from this straight line we can estimate the time to fall for any floor.

我们怎么知道在这里尝试使用一条直线？在某种程度上，我们不知道。这只是一个数学上简单的东西，而且我们习惯于这样一个事实：我们测量的许多数据都很好地符合数学上简单的东西。我们可以尝试更复杂的数学形式——比如 _a_ + _b_ _x_ + _c_ _x_ 2——然后在这种情况下我们做得更好：

How did we know to try using a straight line here? At some level we didn’t. It’s just something that’s mathematically simple, and we’re used to the fact that lots of data we measure turns out to be well fit by mathematically simple things. We could try something mathematically more complicated—say _a_ + _b_ _x_ + _c_ _x_ 2—and then in this case we do better:

然而，事情可能会变得很糟糕。比如，这是用 _a_ + _b_/_x_ + _c_ sin(_x_) 能得到的最佳结果：

Things can go quite wrong, though. Like here’s the best we can do with _a_ + _b_/_x_ + _c_ sin(_x_):

值得理解的是，从来不存在“无模型的模型”。你使用的任何模型都有某种特定的底层结构——然后有一组“你可以调节的旋钮”（即你可以设置的参数）来拟合你的数据。而在 ChatGPT 的情况下，使用了大量这样的“旋钮”——实际上有 1750 亿个。

It is worth understanding that there’s never a “model-less model”. Any model you use has some particular underlying structure—then a certain set of “knobs you can turn” (i.e. parameters you can set) to fit your data. And in the case of ChatGPT, lots of such “knobs” are used—actually, 175 billion of them.

但值得注意的是，ChatGPT 的底层结构——只有那么多参数——足以建立一个模型，能够“足够好”地计算下一个词的概率，从而为我们提供合理的文章长度的文本。

But the remarkable thing is that the underlying structure of ChatGPT—with “just” that many parameters—is sufficient to make a model that computes next-word probabilities “well enough” to give us reasonable essay-length pieces of text.

类人任务的模型 Models for Human-Like Tasks

我们上面给出的例子涉及为本质上来自简单物理学的数值数据建立模型——几个世纪以来我们就知道“简单数学适用”。但对于 ChatGPT，我们必须为人类大脑产生的那种人类语言文本建立模型。对于这样的事情，我们（至少目前）还没有像“简单数学”这样的东西。那么它的模型可能是什么样的呢？

The example we gave above involves making a model for numerical data that essentially comes from simple physics—where we’ve known for several centuries that “simple mathematics applies”. But for ChatGPT we have to make a model of human-language text of the kind produced by a human brain. And for something like that we don’t (at least yet) have anything like “simple mathematics”. So what might a model of it be like?

在讨论语言之前，我们先讨论另一个类人任务：识别图像。作为这个任务的一个简单例子，让我们考虑数字的图像（是的，这是一个经典的机器学习例子）：

Before we talk about language, let’s talk about another human-like task: recognizing images. And as a simple example of this, let’s consider images of digits (and, yes, this is a classic machine learning example):

我们可以做的一件事是为每个数字获取一组样本图像：

One thing we could do is get a bunch of sample images for each digit:

然后，要判断我们输入的图像是否对应某个特定数字，我们可以简单地将它与我们拥有的样本进行逐像素比较。但作为人类，我们似乎做得更好——因为即使数字是手写的，并且有各种修改和变形，我们仍然能够识别它们：

Then to find out if an image we’re given as input corresponds to a particular digit we could just do an explicit pixel-by-pixel comparison with the samples we have. But as humans we certainly seem to do something better—because we can still recognize digits, even when they’re for example handwritten, and have all sorts of modifications and distortions:

当我们为上面的数值数据建立模型时，我们可以获取给定的数值 _x_，并针对特定的 _a_ 和 _b_ 直接计算 _a + b x_。因此，如果我们将这里每个像素的灰度值视为某个变量 _x i_，是否存在所有这些变量的某个函数——当计算时——能告诉我们图像是哪个数字？事实证明，构造这样的函数是可能的。毫不奇怪，它并不特别简单。一个典型的例子可能涉及大约五十万次数学运算。

When we made a model for our numerical data above, we were able to take a numerical value _x_ that we were given, and just compute _a + b x_ for particular _a_ and _b_. So if we treat the gray-level value of each pixel here as some variable _x i_ is there some function of all those variables that—when evaluated—tells us what digit the image is of? It turns out that it’s possible to construct such a function. Not surprisingly, it’s not particularly simple, though. And a typical example might involve perhaps half a million mathematical operations.

但最终结果是，如果我们把图像的像素值集合输入到这个函数中，输出将是指定图像对应哪个数字的数字。稍后，我们将讨论如何构造这样的函数，以及神经网络的概念。但现在，让我们把这个函数当作一个黑箱，我们输入手写数字的图像（作为像素值数组），然后得到它们对应的数字：

But the end result is that if we feed the collection of pixel values for an image into this function, out will come the number specifying which digit we have an image of. Later, we’ll talk about how such a function can be constructed, and the idea of neural nets. But for now let’s treat the function as black box, where we feed in images of, say, handwritten digits (as arrays of pixel values) and we get out the numbers these correspond to:

但这里到底发生了什么？假设我们逐渐模糊一个数字。一开始，我们的函数仍然“识别”它，这里识别为“2”。但很快它就“丢失了”，并开始给出“错误”的结果：

But what’s really going on here? Let’s say we progressively blur a digit. For a little while our function still “recognizes” it, here as a “2”. But soon it “loses it”, and starts giving the “wrong” result:

但我们为什么说它是“错误”的结果？在这种情况下，我们知道所有图像都是通过模糊一个“2”得到的。但如果我们的目标是建立一个模型来模拟人类在识别图像方面的能力，真正要问的问题是，如果一个人看到这些模糊图像之一，并且不知道它来自哪里，他会怎么做。

But why do we say it’s the “wrong” result? In this case, we know we got all the images by blurring a “2”. But if our goal is to produce a model of what humans can do in recognizing images, the real question to ask is what a human would have done if presented with one of those blurred images, without knowing where it came from.

如果我们从函数中得到的结果通常与人类所说的相符，那么我们就有了一个“好模型”。而非平凡的科学事实是，对于像这样的图像识别任务，我们现在基本上知道如何构造实现这一点的函数。

And we have a “good model” if the results we get from our function typically agree with what a human would say. And the nontrivial scientific fact is that for an image-recognition task like this we now basically know how to construct functions that do this.

我们能否“数学上证明”它们有效？嗯，不能。因为要做到这一点，我们必须有一个关于人类行为的数学理论。以“2”图像为例，改变几个像素。我们可能认为，只有几个像素“错位”，我们仍然应该认为图像是“2”。但这应该到什么程度？这是一个人类视觉感知的问题。是的，对于蜜蜂或章鱼来说，答案无疑会不同——对于假设的外星人来说，可能完全不同。

Can we “mathematically prove” that they work? Well, no. Because to do that we’d have to have a mathematical theory of what we humans are doing. Take the “2” image and change a few pixels. We might imagine that with only a few pixels “out of place” we should still consider the image a “2”. But how far should that go? It’s a question of human visual perception. And, yes, the answer would no doubt be different for bees or octopuses—and potentially utterly different for putative aliens.

神经网络 Neural Nets

那么，我们用于图像识别等任务的典型模型实际上是如何工作的？当前最流行且最成功的方法是使用神经网络。神经网络在 20 世纪 40 年代被发明，其形式与今天的使用方式惊人地相似，可以看作是大脑工作方式的简单理想化。

OK, so how do our typical models for tasks like image recognition actually work? The most popular—and successful—current approach uses neural nets. Invented—in a form remarkably close to their use today—in the 1940s, neural nets can be thought of as simple idealizations of how brains seem to work.

在人脑中，大约有 1000 亿个神经元（神经细胞），每个神经元每秒最多能产生大约一千次电脉冲。这些神经元连接成一个复杂的网络，每个神经元都有树状分支，使其能够将电信号传递给可能数千个其他神经元。粗略地说，某个神经元在特定时刻是否产生电脉冲取决于它从其他神经元接收到的脉冲，而不同的连接具有不同的“权重”。

In human brains there are about 100 billion neurons (nerve cells), each capable of producing an electrical pulse up to perhaps a thousand times a second. The neurons are connected in a complicated net, with each neuron having tree-like branches allowing it to pass electrical signals to perhaps thousands of other neurons. And in a rough approximation, whether any given neuron produces an electrical pulse at a given moment depends on what pulses it’s received from other neurons—with different connections contributing with different “weights”.

当我们“看到一幅图像”时，发生的事情是：当来自图像的光子落在我们眼睛后部的（感光）细胞上时，它们会在神经细胞中产生电信号。这些神经细胞连接到其他神经细胞，最终信号经过整个神经元层序列。正是在这个过程中，我们“识别”了图像，最终“形成想法”，认为我们“看到了一个 2”（也许最后还会做一些事情，比如大声说出“二”这个词）。

When we “see an image” what’s happening is that when photons of light from the image fall on (“photoreceptor”) cells at the back of our eyes they produce electrical signals in nerve cells. These nerve cells are connected to other nerve cells, and eventually the signals go through a whole sequence of layers of neurons. And it’s in this process that we “recognize” the image, eventually “forming the thought” that we’re “seeing a 2” (and maybe in the end doing something like saying the word “two” out loud).

上一节中的“黑箱”函数就是这种神经网络的“数学化”版本。它恰好有 11 层（尽管只有 4 个“核心层”）：

The “black-box” function from the previous section is a “mathematicized” version of such a neural net. It happens to have 11 layers (though only 4 “core layers”):

这个神经网络并没有什么特别的“理论推导”；它只是 1998 年作为工程构造出来的东西，并且被发现有效。（当然，这与我们描述大脑是通过生物进化过程产生的说法并没有太大不同。）

There’s nothing particularly “theoretically derived” about this neural net; it’s just something that—back in 1998—was constructed as a piece of engineering, and found to work. (Of course, that’s not much different from how we might describe our brains as having been produced through the process of biological evolution.)

好的，但像这样的神经网络是如何“识别事物”的呢？关键在于吸引子的概念。假设我们有手写的数字 1 和 2 的图像：

OK, but how does a neural net like this “recognize things”? The key is the notion of attractors. Imagine we’ve got handwritten images of 1’s and 2’s:

我们以某种方式希望所有的 1 都“被吸引到一个地方”，所有的 2 都“被吸引到另一个地方”。或者换一种说法，如果一幅图像在某种程度上“更接近 1 而不是 2”，我们希望它最终到达“1 的位置”，反之亦然。

We somehow want all the 1’s to “be attracted to one place”, and all the 2’s to “be attracted to another place”. Or, put a different way, if an image is somehow “closer to being a 1” than to being a 2, we want it to end up in the “1 place” and vice versa.

作为一个简单的类比，假设我们在平面上有某些位置，用点表示（在现实生活中，它们可能是咖啡店的位置）。那么我们可以想象，从平面上的任何点出发，我们总是希望到达最近的点（即我们总是去最近的咖啡店）。我们可以通过将平面划分为由理想化的“分水岭”分隔的区域（“吸引子盆地”）来表示这一点：

As a straightforward analogy, let’s say we have certain positions in the plane, indicated by dots (in a real-life setting they might be positions of coffee shops). Then we might imagine that starting from any point on the plane we’d always want to end up at the closest dot (i.e. we’d always go to the closest coffee shop). We can represent this by dividing the plane into regions (“attractor basins”) separated by idealized “watersheds”:

我们可以将其视为实现了一种“识别任务”，其中我们并不是在做诸如“给定图像看起来最像哪个数字”之类的事情，而是直接看给定点离哪个点最近。（我们在这里展示的“Voronoi 图”设置是在二维欧几里得空间中分隔点；数字识别任务可以被认为是在做非常类似的事情——但在一个由每幅图像中所有像素的灰度级构成的 784 维空间中。）

We can think of this as implementing a kind of “recognition task” in which we’re not doing something like identifying what digit a given image “looks most like”—but rather we’re just, quite directly, seeing what dot a given point is closest to. (The “Voronoi diagram” setup we’re showing here separates points in 2D Euclidean space; the digit recognition task can be thought of as doing something very similar—but in a 784-dimensional space formed from the gray levels of all the pixels in each image.)

那么，我们如何让神经网络“执行识别任务”呢？让我们考虑这个非常简单的例子：

So how do we make a neural net “do a recognition task”? Let’s consider this very simple case:

我们的目标是获取一个对应于位置{x,y}的“输入”，然后将其“识别”为离它最近的三个点之一。或者换句话说，我们希望神经网络计算一个关于{x,y}的函数，如下所示：

Our goal is to take an “input” corresponding to a position {_x_,_y_}—and then to “recognize” it as whichever of the three points it’s closest to. Or, in other words, we want the neural net to compute a function of {_x_,_y_} like:

那么，我们如何用神经网络做到这一点呢？最终，神经网络是一个理想化“神经元”的连接集合——通常按层排列——一个简单的例子是：

So how do we do this with a neural net? Ultimately a neural net is a connected collection of idealized “neurons”—usually arranged in layers—with a simple example being:

每个“神经元”实际上被设置为评估一个简单的数值函数。为了“使用”网络，我们只需在顶部输入数字（比如我们的坐标 x 和 y），然后让每一层的神经元“评估它们的函数”并将结果通过网络向前传递——最终在底部产生最终结果：

Each “neuron” is effectively set up to evaluate a simple numerical function. And to “use” the network, we simply feed numbers (like our coordinates _x_ and _y_) in at the top, then have neurons on each layer “evaluate their functions” and feed the results forward through the network—eventually producing the final result at the bottom:

在传统的（受生物学启发的）设置中，每个神经元实际上有一组来自前一层神经元的“传入连接”，每个连接被分配一个特定的“权重”（可以是正数或负数）。给定神经元的值是通过将“前一层神经元”的值乘以它们对应的权重，然后将它们相加并加上一个常数——最后应用一个“阈值”（或“激活”）函数来确定的。用数学术语来说，如果一个神经元有输入 x = {x1, x2, ...}，那么我们计算 f[w·x + b]，其中权重 w 和常数 b 通常对网络中的每个神经元选择不同；函数 f 通常是相同的。

In the traditional (biologically inspired) setup each neuron effectively has a certain set of “incoming connections” from the neurons on the previous layer, with each connection being assigned a certain “weight” (which can be a positive or negative number). The value of a given neuron is determined by multiplying the values of “previous neurons” by their corresponding weights, then adding these up and adding a constant—and finally applying a “thresholding” (or “activation”) function. In mathematical terms, if a neuron has inputs _x_ = {_x_ 1, _x_ 2 …} then we compute _f_[_w_ . _x_ + _b_], where the weights _w_ and constant _b_ are generally chosen differently for each neuron in the network; the function _f_ is usually the same.

计算 w·x + b 只是矩阵乘法和加法的问题。“激活函数”f 引入了非线性（并最终导致了非平凡的行为）。常用的激活函数有多种；这里我们只使用 Ramp（或 ReLU）：

Computing _w_ . _x_ + _b_ is just a matter of matrix multiplication and addition. The “activation function” _f_ introduces nonlinearity (and ultimately is what leads to nontrivial behavior). Various activation functions commonly get used; here we’ll just use Ramp (or ReLU):

对于我们希望神经网络执行的每个任务（或者等价地，对于我们希望它评估的每个整体函数），我们将有不同的权重选择。（而且——正如我们稍后将讨论的——这些权重通常是通过使用机器学习从我们想要的输出示例中“训练”神经网络来确定的。）

For each task we want the neural net to perform (or, equivalently, for each overall function we want it to evaluate) we’ll have different choices of weights. (And—as we’ll discuss later—these weights are normally determined by “training” the neural net using machine learning from examples of the outputs we want.)

最终，每个神经网络都对应于某个整体数学函数——尽管写出来可能很混乱。对于上面的例子，它将是：

Ultimately, every neural net just corresponds to some overall mathematical function—though it may be messy to write out. For the example above, it would be:

ChatGPT 的神经网络也对应于这样一个数学函数——但实际上有数十亿项。

The neural net of ChatGPT also just corresponds to a mathematical function like this—but effectively with billions of terms.

但让我们回到单个神经元。以下是一个具有两个输入（表示坐标 x 和 y）的神经元在权重和常数（以及 Ramp 作为激活函数）的各种选择下可以计算的函数的一些例子：

But let’s go back to individual neurons. Here are some examples of the functions a neuron with two inputs (representing coordinates _x_ and _y_) can compute with various choices of weights and constants (and Ramp as activation function):

那么上面那个更大的网络呢？嗯，这是它计算的结果：

But what about the larger network from above? Well, here’s what it computes:

它并不完全“正确”，但接近我们上面展示的“最近点”函数。

It’s not quite “right”, but it’s close to the “nearest point” function we showed above.

让我们看看其他一些神经网络会发生什么。在每种情况下，正如我们稍后将解释的，我们使用机器学习来找到最佳的权重选择。然后我们在这里展示具有这些权重的神经网络计算的结果：

Let’s see what happens with some other neural nets. In each case, as we’ll explain later, we’re using machine learning to find the best choice of weights. Then we’re showing here what the neural net with those weights computes:

更大的网络通常能更好地逼近我们目标函数。并且在每个“吸引子盆地”的“中间”，我们通常能得到我们想要的精确答案。但在边界处——神经网络“难以做出决定”的地方——事情可能会更混乱。

Bigger networks generally do better at approximating the function we’re aiming for. And in the “middle of each attractor basin” we typically get exactly the answer we want. But at the boundaries—where the neural net “has a hard time making up its mind”—things can be messier.

对于这种简单的数学风格“识别任务”，什么是“正确答案”是清楚的。但在识别手写数字的问题中，就不那么清楚了。如果有人把“2”写得非常糟糕以至于看起来像“7”等等，该怎么办？尽管如此，我们仍然可以询问神经网络如何区分数字——这给出了一种指示：

With this simple mathematical-style “recognition task” it’s clear what the “right answer” is. But in the problem of recognizing handwritten digits, it’s not so clear. What if someone wrote a “2” so badly it looked like a “7”, etc.? Still, we can ask how a neural net distinguishes digits—and this gives an indication:

我们能否“数学地”说明网络是如何做出区分的？不能完全。它只是“做神经网络该做的事”。但事实证明，这通常与人类所做的区分相当一致。

Can we say “mathematically” how the network makes its distinctions? Not really. It’s just “doing what the neural net does”. But it turns out that that normally seems to agree fairly well with the distinctions we humans make.

让我们看一个更复杂的例子。假设我们有猫和狗的图片。并且我们有一个经过训练可以区分它们的神经网络。以下是在一些示例上它可能做的事情：

Let’s take a more elaborate example. Let’s say we have images of cats and dogs. And we have a neural net that’s been trained to distinguish them. Here’s what it might do on some examples:

现在甚至更不清楚什么是“正确答案”了。一只穿着猫服的狗呢？等等。无论输入什么，神经网络都会生成一个答案，并且在某种程度上与人类可能的方式相当一致。正如我上面所说，这不是一个我们可以“从第一原理推导”的事实。它只是经验上被发现为真的事情，至少在某些领域是这样。但这是神经网络有用的一个关键原因：它们以某种方式捕捉了一种“类似人类”的做事方式。

Now it’s even less clear what the “right answer” is. What about a dog dressed in a cat suit? Etc. Whatever input it’s given the neural net will generate an answer, and in a way reasonably consistent with how humans might. As I’ve said above, that’s not a fact we can “derive from first principles”. It’s just something that’s empirically been found to be true, at least in certain domains. But it’s a key reason why neural nets are useful: that they somehow capture a “human-like” way of doing things.

给自己看一张猫的图片，然后问“为什么这是一只猫？”也许你会开始说“嗯，我看到它尖尖的耳朵等等。”但要解释你是如何识别出这张图片是猫的并不容易。只是你的大脑以某种方式弄清楚了。但对于大脑来说，没有办法（至少目前）“进入内部”看看它是如何弄清楚的。对于（人工）神经网络呢？嗯，当你展示一张猫的图片时，很容易看到每个“神经元”做了什么。但即使进行基本的可视化通常也非常困难。

Show yourself a picture of a cat, and ask “Why is that a cat?”. Maybe you’d start saying “Well, I see its pointy ears, etc.” But it’s not very easy to explain how you recognized the image as a cat. It’s just that somehow your brain figured that out. But for a brain there’s no way (at least yet) to “go inside” and see how it figured it out. What about for an (artificial) neural net? Well, it’s straightforward to see what each “neuron” does when you show a picture of a cat. But even to get a basic visualization is usually very difficult.

在我们上面用于“最近点”问题的最终网络中，有 17 个神经元。在用于识别手写数字的网络中，有 2190 个。而在我们用于识别猫和狗的网络中，有 60650 个。通常，可视化相当于 60650 维空间是非常困难的。但因为这是一个为处理图像而设置的网络，它的许多神经元层被组织成数组，就像它正在查看的像素数组一样。

In the final net that we used for the “nearest point” problem above there are 17 neurons. In the net for recognizing handwritten digits there are 2190. And in the net we’re using to recognize cats and dogs there are 60,650. Normally it would be pretty difficult to visualize what amounts to 60,650-dimensional space. But because this is a network set up to deal with images, many of its layers of neurons are organized into arrays, like the arrays of pixels it’s looking at.

那么我们可以通过一组派生图像来表示第一层神经元的状态——其中许多我们可以很容易地解释为诸如“没有背景的猫”或“猫的轮廓”之类的东西：

then we can represent the states of neurons at the first layer by a collection of derived images—many of which we can readily interpret as being things like “the cat without its background”, or “the outline of the cat”:

到第 10 层时，解释正在发生的事情就更难了：

By the 10th layer it’s harder to interpret what’s going on:

但总的来说，我们可以说神经网络正在“挑选某些特征”（也许尖耳朵是其中之一），并使用这些特征来确定图像是什么。但这些特征是我们有名称的特征吗——比如“尖耳朵”？大多数不是。

But in general we might say that the neural net is “picking out certain features” (maybe pointy ears are among them), and using these to determine what the image is of. But are those features ones for which we have names—like “pointy ears”? Mostly not.

我们的大脑是否使用类似的特征？我们大多不知道。但值得注意的是，像我们这里展示的神经网络的前几层似乎挑选出图像的某些方面（如物体的边缘），这些方面似乎与我们已知的大脑视觉处理第一级所挑选的方面相似。

Are our brains using similar features? Mostly we don’t know. But it’s notable that the first few layers of a neural net like the one we’re showing here seem to pick out aspects of images (like edges of objects) that seem to be similar to ones we know are picked out by the first level of visual processing in brains.

但假设我们想要一个神经网络中“猫识别”的理论。我们可以说：“看，这个特定的网络做到了”——这立即让我们对“问题有多难”有了一些感觉（例如，需要多少神经元或层）。但至少到目前为止，我们还没有办法“给出一个叙述性描述”来说明网络在做什么。也许这是因为它在计算上确实是不可约的，并且没有通用的方法可以找出它做什么，除非明确地跟踪每一步。或者也许只是我们还没有“弄清楚科学”，并识别出允许我们总结正在发生的事情的“自然法则”。

But let’s say we want a “theory of cat recognition” in neural nets. We can say: “Look, this particular net does it”—and immediately that gives us some sense of “how hard a problem” it is (and, for example, how many neurons or layers might be needed). But at least as of now we don’t have a way to “give a narrative description” of what the network is doing. And maybe that’s because it truly is computationally irreducible, and there’s no general way to find what it does except by explicitly tracing each step. Or maybe it’s just that we haven’t “figured out the science”, and identified the “natural laws” that allow us to summarize what’s going on.

当我们讨论用 ChatGPT 生成语言时，我们会遇到同样的问题。同样，目前还不清楚是否有办法“总结它在做什么”。但语言的丰富性和细节（以及我们对它的经验）可能使我们比图像走得更远。

We’ll encounter the same kinds of issues when we talk about generating language with ChatGPT. And again it’s not clear whether there are ways to “summarize what it’s doing”. But the richness and detail of language (and our experience with it) may allow us to get further than with images.

机器学习与神经网络训练 Machine Learning, and the Training of Neural Nets

到目前为止，我们一直在讨论“已经知道”如何执行特定任务的神经网络。但神经网络之所以如此有用（在脑中可能也是如此），不仅在于它们原则上可以执行各种任务，还在于它们可以逐步地“从示例中训练”来执行这些任务。

We’ve been talking so far about neural nets that “already know” how to do particular tasks. But what makes neural nets so useful (presumably also in brains) is that not only can they in principle do all sorts of tasks, but they can be incrementally “trained from examples” to do those tasks.

当我们制作一个区分猫和狗的神经网络时，我们实际上不需要编写一个（例如）显式寻找胡须的程序；相反，我们只需展示大量猫和狗的示例，然后让网络从这些示例中“机器学习”如何区分它们。

When we make a neural net to distinguish cats from dogs we don’t effectively have to write a program that (say) explicitly finds whiskers; instead we just show lots of examples of what’s a cat and what’s a dog, and then have the network “machine learn” from these how to distinguish them.

关键在于，训练后的网络会从它看到的特定示例中“泛化”。正如我们上面所见，网络不仅仅识别它看到的示例猫图像的特定像素模式；相反，神经网络以某种方式根据我们认为是某种“一般猫性”的东西来区分图像。

And the point is that the trained network “generalizes” from the particular examples it’s shown. Just as we’ve seen above, it isn’t simply that the network recognizes the particular pixel pattern of an example cat image it was shown; rather it’s that the neural net somehow manages to distinguish images on the basis of what we consider to be some kind of “general catness”.

那么神经网络训练实际上是如何工作的呢？本质上，我们始终试图找到能够使神经网络成功复现我们给出的示例的权重。然后我们依赖神经网络以“合理”的方式在这些示例之间进行“插值”（或“泛化”）。

So how does neural net training actually work? Essentially what we’re always trying to do is to find weights that make the neural net successfully reproduce the examples we’ve given. And then we’re relying on the neural net to “interpolate” (or “generalize”) “between” these examples in a “reasonable” way.

让我们看一个比上面最近点问题更简单的问题。我们尝试让神经网络学习函数：

Let’s look at a problem even simpler than the nearest-point one above. Let’s just try to get a neural net to learn the function:

对于这个任务，我们需要一个只有一个输入和一个输出的网络，比如：

For this task, we’ll need a network that has just one input and one output, like:

但是我们应该使用什么权重呢？对于每一组可能的权重，神经网络都会计算某个函数。例如，以下是它使用几组随机选择的权重时的表现：

But what weights, etc. should we be using? With every possible set of weights the neural net will compute some function. And, for example, here’s what it does with a few randomly chosen sets of weights:

是的，我们可以清楚地看到，在这些情况下，没有一个能接近复现我们想要的函数。那么如何找到能够复现该函数的权重呢？

And, yes, we can plainly see that in none of these cases does it get even close to reproducing the function we want. So how do we find weights that will reproduce the function?

基本思想是提供大量的“输入→输出”示例来“学习”——然后尝试找到能够复现这些示例的权重。以下是使用逐渐增多的示例进行训练的结果：

The basic idea is to supply lots of “input → output” examples to “learn from”—and then to try to find weights that will reproduce these examples. Here’s the result of doing that with progressively more examples:

在这个“训练”的每个阶段，网络中的权重都会逐步调整——我们看到最终我们得到一个成功复现所需函数的网络。那么如何调整权重呢？基本思想是在每个阶段查看我们“离目标有多远”——然后以使我们更接近目标的方式更新权重。

At each stage in this “training” the weights in the network are progressively adjusted—and we see that eventually we get a network that successfully reproduces the function we want. So how do we adjust the weights? The basic idea is at each stage to see “how far away we are” from getting the function we want—and then to update the weights in such a way as to get closer.

为了找出“我们离目标有多远”，我们计算通常称为“损失函数”（有时也称为“成本函数”）的东西。这里我们使用一个简单的（L2）损失函数，它只是我们得到的值与真实值之间差值的平方和。我们看到，随着训练过程的进行，损失函数逐渐减小（遵循一条因任务而异的“学习曲线”）——直到我们达到网络（至少很好地近似）成功复现所需函数的点：

To find out “how far away we are” we compute what’s usually called a “loss function” (or sometimes “cost function”). Here we’re using a simple (L2) loss function that’s just the sum of the squares of the differences between the values we get, and the true values. And what we see is that as our training process progresses, the loss function progressively decreases (following a certain “learning curve” that’s different for different tasks)—until we reach a point where the network (at least to a good approximation) successfully reproduces the function we want:

好了，最后一个需要解释的关键部分是如何调整权重以减少损失函数。正如我们所说，损失函数给出了我们得到的值与真实值之间的“距离”。但“我们得到的值”在每个阶段由当前版本的神经网络及其权重决定。现在想象权重是变量——比如 _w i_。我们想找出如何调整这些变量的值以最小化依赖于它们的损失。

Alright, so the last essential piece to explain is how the weights are adjusted to reduce the loss function. As we’ve said, the loss function gives us a “distance” between the values we’ve got, and the true values. But the “values we’ve got” are determined at each stage by the current version of neural net—and by the weights in it. But now imagine that the weights are variables—say _w i_. We want to find out how to adjust the values of these variables to minimize the loss that depends on them.

例如，想象（在实际使用的典型神经网络的一个极其简化的版本中）我们只有两个权重 _w_ 1 和 _w_ 2。那么损失作为 _w_ 1 和 _w_ 2 的函数可能看起来像这样：

For example, imagine (in an incredible simplification of typical neural nets used in practice) that we have just two weights _w_ 1 and _w_ 2. Then we might have a loss that as a function of _w_ 1 and _w_ 2 looks like this:

数值分析提供了多种技术来寻找这种情况下的最小值。但一种典型的方法是逐步跟随从我们之前的 _w_ 1, _w_ 2 出发的最陡下降路径：

Numerical analysis provides a variety of techniques for finding the minimum in cases like this. But a typical approach is just to progressively follow the path of steepest descent from whatever previous _w_ 1, _w_ 2 we had:

就像水从山上流下一样，唯一能保证的是这个过程最终会到达表面的某个局部最小值（“山湖”）；它很可能无法达到最终的全局最小值。

Like water flowing down a mountain, all that’s guaranteed is that this procedure will end up at some local minimum of the surface (“a mountain lake”); it might well not reach the ultimate global minimum.

在“权重景观”上找到最陡下降路径是否可行并不明显。但微积分解决了这个问题。正如我们上面提到的，人们总是可以将神经网络视为计算一个数学函数——该函数依赖于其输入和权重。现在考虑对这些权重求导。事实证明，微积分的链式法则实际上让我们能够“解开”神经网络中连续层执行的操作。结果是，我们可以在至少局部近似的情况下，“反转”神经网络的操作，并逐步找到最小化与输出相关的损失的权重。

It’s not obvious that it would be feasible to find the path of the steepest descent on the “weight landscape”. But calculus comes to the rescue. As we mentioned above, one can always think of a neural net as computing a mathematical function—that depends on its inputs, and its weights. But now consider differentiating with respect to these weights. It turns out that the chain rule of calculus in effect lets us “unravel” the operations done by successive layers in the neural net. And the result is that we can—at least in some local approximation—“invert” the operation of the neural net, and progressively find weights that minimize the loss associated with the output.

上面的图片展示了在只有 2 个权重的极其简单的情况下我们可能需要进行的这种最小化。但事实证明，即使有更多的权重（ChatGPT 使用 1750 亿个），仍然可以进行最小化，至少在某种近似程度上。事实上，大约在 2011 年发生的“深度学习”的重大突破与一个发现有关，即在某种意义上，当涉及大量权重时，进行（至少近似）最小化可能比权重较少时更容易。

The picture above shows the kind of minimization we might need to do in the unrealistically simple case of just 2 weights. But it turns out that even with many more weights (ChatGPT uses 175 billion) it’s still possible to do the minimization, at least to some level of approximation. And in fact the big breakthrough in “deep learning” that occurred around 2011 was associated with the discovery that in some sense it can be easier to do (at least approximate) minimization when there are lots of weights involved than when there are fairly few.

换句话说——有点反直觉——用神经网络解决更复杂的问题可能比解决更简单的问题更容易。其粗略原因似乎是，当有大量“权重变量”时，我们有一个高维空间，其中有“许多不同的方向”可以引导我们到达最小值——而变量较少时，更容易陷入局部最小值（“山湖”），从那里“没有方向可以出去”。

In other words—somewhat counterintuitively—it can be easier to solve more complicated problems with neural nets than simpler ones. And the rough reason for this seems to be that when one has a lot of “weight variables” one has a high-dimensional space with “lots of different directions” that can lead one to the minimum—whereas with fewer variables it’s easier to end up getting stuck in a local minimum (“mountain lake”) from which there’s no “direction to get out”.

值得指出的是，在典型情况下，有许多不同的权重集合都会给出性能几乎相同的神经网络。通常在实际的神经网络训练中，会有很多随机选择——导致“不同但等效的解”，比如这些：

It’s worth pointing out that in typical cases there are many different collections of weights that will all give neural nets that have pretty much the same performance. And usually in practical neural net training there are lots of random choices made—that lead to “different-but-equivalent solutions”, like these:

但每个这样的“不同解”都会至少略有不同的行为。如果我们要求，例如，在给出训练示例的区域之外进行“外推”，我们可能会得到截然不同的结果：

But each such “different solution” will have at least slightly different behavior. And if we ask, say, for an “extrapolation” outside the region where we gave training examples, we can get dramatically different results:

但哪个是“正确”的呢？实际上无法判断。它们都与“观察到的数据一致”。但它们都对应于不同的“固有”方式来“思考”在“框外”该做什么。有些可能在我们人类看来比其他更“合理”。

But which of these is “right”? There’s really no way to say. They’re all “consistent with the observed data”. But they all correspond to different “innate” ways to “think about” what to do “outside the box”. And some may seem “more reasonable” to us humans than others.

神经网络训练的实践与经验 The Practice and Lore of Neural Net Training

特别是在过去十年中，训练神经网络的艺术取得了许多进展。是的，它基本上是一门艺术。有时——尤其是事后回顾——人们至少能对正在做的事情看到一丝“科学解释”的微光。但大多数情况下，事物是通过试错发现的，添加的想法和技巧逐渐积累起关于如何使用神经网络的重要经验。

Particularly over the past decade, there’ve been many advances in the art of training neural nets. And, yes, it is basically an art. Sometimes—especially in retrospect—one can see at least a glimmer of a “scientific explanation” for something that’s being done. But mostly things have been discovered by trial and error, adding ideas and tricks that have progressively built a significant lore about how to work with neural nets.

有几个关键部分。首先，是针对特定任务应该使用什么样的神经网络架构的问题。然后是关键问题：如何获取用于训练神经网络的数据。而且，越来越多的情况不是从头开始训练网络：相反，一个新网络可以直接整合另一个已经训练好的网络，或者至少可以利用该网络为自己生成更多的训练示例。

There are several key parts. First, there’s the matter of what architecture of neural net one should use for a particular task. Then there’s the critical issue of how one’s going to get the data on which to train the neural net. And increasingly one isn’t dealing with training a net from scratch: instead a new net can either directly incorporate another already-trained net, or at least can use that net to generate more training examples for itself.

人们可能认为，对于每种特定类型的任务，都需要不同的神经网络架构。但发现的是，相同的架构通常甚至对明显不同的任务也有效。在某种程度上，这让人想起通用计算的思想（以及我的计算等价原理），但正如我稍后将讨论的，我认为这更多地反映了这样一个事实：我们通常试图让神经网络完成的任务是“类人的”——而神经网络可以捕捉相当通用的“类人过程”。

One might have thought that for every particular kind of task one would need a different architecture of neural net. But what’s been found is that the same architecture often seems to work even for apparently quite different tasks. At some level this reminds one of the idea of universal computation (and my Principle of Computational Equivalence), but, as I’ll discuss later, I think it’s more a reflection of the fact that the tasks we’re typically trying to get neural nets to do are “human-like” ones—and neural nets can capture quite general “human-like processes”.

在神经网络的早期，人们倾向于认为应该“让神经网络尽可能少做”。例如，在将语音转换为文本时，人们认为应该首先分析语音的音频，将其分解为音素等。但发现的是——至少对于“类人任务”——通常最好尝试在“端到端问题”上训练神经网络，让它自己“发现”必要的中间特征、编码等。

In earlier days of neural nets, there tended to be the idea that one should “make the neural net do as little as possible”. For example, in converting speech to text it was thought that one should first analyze the audio of the speech, break it into phonemes, etc. But what was found is that—at least for “human-like tasks”—it’s usually better just to try to train the neural net on the “end-to-end problem”, letting it “discover” the necessary intermediate features, encodings, etc. for itself.

还有一种想法是，应该在神经网络中引入复杂的单个组件，让它实际上“显式实现特定的算法思想”。但同样，这大多被证明是不值得的；相反，最好只处理非常简单的组件，让它们“自我组织”（尽管通常以我们无法理解的方式）来实现（大概）那些算法思想的等价物。

There was also the idea that one should introduce complicated individual components into the neural net, to let it in effect “explicitly implement particular algorithmic ideas”. But once again, this has mostly turned out not to be worthwhile; instead, it’s better just to deal with very simple components and let them “organize themselves” (albeit usually in ways we can’t understand) to achieve (presumably) the equivalent of those algorithmic ideas.

这并不是说没有与神经网络相关的“结构化思想”。例如，具有局部连接的二维神经元阵列至少在图像处理的早期阶段似乎非常有用。而专注于“回顾序列”的连接模式似乎有用——正如我们稍后将看到的——在处理人类语言之类的事情时，例如在 ChatGPT 中。

That’s not to say that there are no “structuring ideas” that are relevant for neural nets. Thus, for example, having 2D arrays of neurons with local connections seems at least very useful in the early stages of processing images. And having patterns of connectivity that concentrate on “looking back in sequences” seems useful—as we’ll see later—in dealing with things like human language, for example in ChatGPT.

但神经网络的一个重要特征是——像一般的计算机一样——它们最终只处理数据。而当前的神经网络——采用当前的神经网络训练方法——专门处理数字数组。但在处理过程中，这些数组可以完全重新排列和重塑。例如，我们上面用于识别数字的网络从一个二维的“类图像”数组开始，迅速“加厚”到多个通道，然后“浓缩”成一个一维数组，最终包含代表不同可能输出数字的元素：

But an important feature of neural nets is that—like computers in general—they’re ultimately just dealing with data. And current neural nets—with current approaches to neural net training—specifically deal with arrays of numbers. But in the course of processing, those arrays can be completely rearranged and reshaped. And as an example, the network we used for identifying digits above starts with a 2D “image-like” array, quickly “thickening” to many channels, but then “concentrating down” into a 1D array that will ultimately contain elements representing the different possible output digits:

但是，好吧，如何判断一个特定任务需要多大的神经网络呢？这有点像一门艺术。在某种程度上，关键是要知道“任务有多难”。但对于类人任务，这通常很难估计。是的，可能有系统的方法通过计算机非常“机械地”完成任务。但很难知道是否存在可以视为技巧或捷径的东西，使得能够以“类人水平”更轻松地完成任务。可能需要枚举一个巨大的博弈树来“机械地”玩某个游戏；但可能存在一种更简单（“启发式”）的方法来实现“人类水平的游戏”。

But, OK, how can one tell how big a neural net one will need for a particular task? It’s something of an art. At some level the key thing is to know “how hard the task is”. But for human-like tasks that’s typically very hard to estimate. Yes, there may be a systematic way to do the task very “mechanically” by computer. But it’s hard to know if there are what one might think of as tricks or shortcuts that allow one to do the task at least at a “human-like level” vastly more easily. It might take enumerating a giant game tree to “mechanically” play a certain game; but there might be a much easier (“heuristic”) way to achieve “human-level play”.

当处理微小的神经网络和简单的任务时，有时可以明确地看到“从这里无法到达那里”。例如，以下是用几个小神经网络在前一节的任务上能做到的最佳结果：

When one’s dealing with tiny neural nets and simple tasks one can sometimes explicitly see that one “can’t get there from here”. For example, here’s the best one seems to be able to do on the task from the previous section with a few small neural nets:

我们看到的是，如果网络太小，它就无法重现我们想要的函数。但超过某个大小，它就没有问题——至少如果训练时间足够长，示例足够多的话。顺便说一句，这些图片说明了神经网络的一条经验：如果中间有一个“挤压”迫使所有信息通过一个较小的中间神经元数量，通常可以使用更小的网络。（还值得一提的是，“无中间层”网络——或所谓的“感知器”——只能学习本质上线性的函数——但只要有一个中间层，原则上总是可以任意好地逼近任何函数，至少如果有足够多的神经元，尽管为了使其可训练，通常需要某种正则化或归一化。）

And what we see is that if the net is too small, it just can’t reproduce the function we want. But above some size, it has no problem—at least if one trains it for long enough, with enough examples. And, by the way, these pictures illustrate a piece of neural net lore: that one can often get away with a smaller network if there’s a “squeeze” in the middle that forces everything to go through a smaller intermediate number of neurons. (It’s also worth mentioning that “no-intermediate-layer”—or so-called “perceptron”—networks can only learn essentially linear functions—but as soon as there’s even one intermediate layer it’s always in principle possible to approximate any function arbitrarily well, at least if one has enough neurons, though to make it feasibly trainable one typically has some kind of regularization or normalization.)

好了，假设我们已经确定了某种神经网络架构。现在的问题是获取数据来训练网络。而围绕神经网络——以及一般机器学习——的许多实际挑战都集中在获取或准备必要的训练数据上。在许多情况下（“监督学习”），人们希望获得输入和期望输出的明确示例。例如，可能需要根据图像中的内容或其他属性进行标记的图像。也许需要明确地——通常付出巨大努力——进行标记。但通常发现可以借助已经完成的事情，或将其用作某种代理。例如，可以使用为网页图像提供的 alt 标签。或者，在不同领域，可以使用为视频创建的字幕。或者——对于语言翻译训练——可以使用不同语言存在的网页或其他文档的并行版本。

OK, so let’s say one’s settled on a certain neural net architecture. Now there’s the issue of getting data to train the network with. And many of the practical challenges around neural nets—and machine learning in general—center on acquiring or preparing the necessary training data. In many cases (“supervised learning”) one wants to get explicit examples of inputs and the outputs one is expecting from them. Thus, for example, one might want images tagged by what’s in them, or some other attribute. And maybe one will have to explicitly go through—usually with great effort—and do the tagging. But very often it turns out to be possible to piggyback on something that’s already been done, or use it as some kind of proxy. And so, for example, one might use alt tags that have been provided for images on the web. Or, in a different domain, one might use closed captions that have been created for videos. Or—for language translation training—one might use parallel versions of webpages or other documents that exist in different languages.

需要向神经网络展示多少数据才能针对特定任务进行训练？同样，很难从基本原理估计。当然，通过使用“迁移学习”来“迁移”另一个网络中已学习的重要特征列表，可以显著减少需求。但通常神经网络需要“看到大量示例”才能训练好。至少对于某些任务，神经网络的一条重要经验是示例可以极其重复。实际上，一种标准策略是反复向神经网络展示所有示例。在每个这样的“训练轮次”（或“时期”）中，神经网络至少会处于略有不同的状态，而“提醒它”某个特定示例有助于让它“记住那个示例”。（是的，也许这类似于重复对人类记忆的有用性。）

How much data do you need to show a neural net to train it for a particular task? Again, it’s hard to estimate from first principles. Certainly the requirements can be dramatically reduced by using “transfer learning” to “transfer in” things like lists of important features that have already been learned in another network. But generally neural nets need to “see a lot of examples” to train well. And at least for some tasks it’s an important piece of neural net lore that the examples can be incredibly repetitive. And indeed it’s a standard strategy to just show a neural net all the examples one has, over and over again. In each of these “training rounds” (or “epochs”) the neural net will be in at least a slightly different state, and somehow “reminding it” of a particular example is useful in getting it to “remember that example”. (And, yes, perhaps this is analogous to the usefulness of repetition in human memorization.)

但通常仅仅重复相同的示例是不够的。还需要向神经网络展示示例的变体。神经网络的一条经验是，这些“数据增强”变体不必复杂就能有用。只需用基本图像处理稍微修改图像，就可以使它们对于神经网络训练来说“几乎像新的一样”。同样，当用于训练自动驾驶汽车的实际视频等用完后，可以继续从类似视频游戏的模拟环境中获取数据，而无需真实世界场景的所有细节。

But often just repeating the same example over and over again isn’t enough. It’s also necessary to show the neural net variations of the example. And it’s a feature of neural net lore that those “data augmentation” variations don’t have to be sophisticated to be useful. Just slightly modifying images with basic image processing can make them essentially “as good as new” for neural net training. And, similarly, when one’s run out of actual video, etc. for training self-driving cars, one can go on and just get data from running simulations in a model videogame-like environment without all the detail of actual real-world scenes.

像 ChatGPT 这样的东西呢？嗯，它有一个很好的特性，即可以进行“无监督学习”，这使得获取训练示例更加容易。回想一下，ChatGPT 的基本任务是找出如何继续给定的文本片段。因此，要获得“训练示例”，只需获取一段文本，屏蔽其结尾，然后将其用作“训练输入”——“输出”是完整的、未屏蔽的文本片段。我们稍后将详细讨论这一点，但主要点是——与学习图像中的内容不同——不需要“显式标记”；ChatGPT 实际上可以直接从给定的任何文本示例中学习。

How about something like ChatGPT? Well, it has the nice feature that it can do “unsupervised learning”, making it much easier to get it examples to train from. Recall that the basic task for ChatGPT is to figure out how to continue a piece of text that it’s been given. So to get it “training examples” all one has to do is get a piece of text, and mask out the end of it, and then use this as the “input to train from”—with the “output” being the complete, unmasked piece of text. We’ll discuss this more later, but the main point is that—unlike, say, for learning what’s in images—there’s no “explicit tagging” needed; ChatGPT can in effect just learn directly from whatever examples of text it’s given.

好了，那么神经网络中的实际学习过程呢？最终，这一切都关乎确定哪些权重能最好地捕捉给定的训练示例。并且有各种详细的选择和“超参数设置”（之所以这样称呼，是因为权重可以被视为“参数”），可以用来调整这个过程。有不同的损失函数选择（平方和、绝对值之和等）。有不同的损失最小化方法（每次在权重空间中移动多远等）。然后还有诸如每次展示多大“批次”的示例以获得要最小化的损失的连续估计等问题。是的，可以应用机器学习（例如，我们在 Wolfram 语言中就是这样做的）来自动化机器学习——并自动设置超参数等。

OK, so what about the actual learning process in a neural net? In the end it’s all about determining what weights will best capture the training examples that have been given. And there are all sorts of detailed choices and “hyperparameter settings” (so called because the weights can be thought of as “parameters”) that can be used to tweak how this is done. There are different choices of loss function (sum of squares, sum of absolute values, etc.). There are different ways to do loss minimization (how far in weight space to move at each step, etc.). And then there are questions like how big a “batch” of examples to show to get each successive estimate of the loss one’s trying to minimize. And, yes, one can apply machine learning (as we do, for example, in Wolfram Language) to automate machine learning—and to automatically set things like hyperparameters.

但最终，整个训练过程可以通过观察损失如何逐渐减少来表征（如下面的 Wolfram 语言小训练进度监视器所示）：

But in the end the whole process of training can be characterized by seeing how the loss progressively decreases (as in this Wolfram Language progress monitor for a small training):

通常看到的是损失先减少一段时间，但最终在某个常数值处趋于平稳。如果该值足够小，则训练可以认为是成功的；否则，可能表明应该尝试改变网络架构。

And what one typically sees is that the loss decreases for a while, but eventually flattens out at some constant value. If that value is sufficiently small, then the training can be considered successful; otherwise it’s probably a sign one should try changing the network architecture.

能否判断“学习曲线”需要多长时间才能趋于平稳？就像许多其他事情一样，似乎存在近似幂律的缩放关系，取决于神经网络的大小和使用的数据量。但一般结论是，训练神经网络很难——并且需要大量的计算工作。实际上，这些工作绝大部分花在对数字数组的操作上，而这正是 GPU 擅长的——这就是为什么神经网络训练通常受限于 GPU 的可用性。

Can one tell how long it should take for the “learning curve” to flatten out? Like for so many other things, there seem to be approximate power-law scaling relationships that depend on the size of neural net and amount of data one’s using. But the general conclusion is that training a neural net is hard—and takes a lot of computational effort. And as a practical matter, the vast majority of that effort is spent doing operations on arrays of numbers, which is what GPUs are good at—which is why neural net training is typically limited by the availability of GPUs.

未来，是否会有根本更好的方法来训练神经网络——或者一般地做神经网络所做的事情？我认为几乎可以肯定。神经网络的基本思想是用大量简单的（基本相同的）组件创建一个灵活的“计算结构”——并使这个“结构”能够逐步修改以从示例中学习。在当前的神经网络中，基本上使用微积分的思想——应用于实数——来进行这种逐步修改。但越来越清楚的是，高精度数字并不重要；即使使用当前方法，8 位或更少可能就足够了。

In the future, will there be fundamentally better ways to train neural nets—or generally do what neural nets do? Almost certainly, I think. The fundamental idea of neural nets is to create a flexible “computing fabric” out of a large number of simple (essentially identical) components—and to have this “fabric” be one that can be incrementally modified to learn from examples. In current neural nets, one’s essentially using the ideas of calculus—applied to real numbers—to do that incremental modification. But it’s increasingly clear that having high-precision numbers doesn’t matter; 8 bits or less might be enough even with current methods.

对于像元胞自动机这样基本上并行操作在许多单个位上的计算系统，如何做这种逐步修改一直不清楚，但没有理由认为这是不可能的。实际上，就像“2012 年的深度学习突破”一样，这种逐步修改在更复杂的情况下可能比在简单情况下更容易。

With computational systems like cellular automata that basically operate in parallel on many individual bits it’s never been clear how to do this kind of incremental modification, but there’s no reason to think it isn’t possible. And in fact, much like with the “deep-learning breakthrough of 2012” it may be that such incremental modification will effectively be easier in more complicated cases than in simple ones.

神经网络——也许有点像大脑——被设置为具有基本固定的神经元网络，修改的是它们之间连接强度（“权重”）。（也许至少在年轻的大脑中，大量全新的连接也可以生长。）但虽然这对生物学来说可能是一个方便的设置，但完全不清楚它是否接近实现我们所需功能的最佳方式。而涉及渐进式网络重写（也许让人想起我们的物理项目）的东西可能最终会更好。

Neural nets—perhaps a bit like brains—are set up to have an essentially fixed network of neurons, with what’s modified being the strength (“weight”) of connections between them. (Perhaps in at least young brains significant numbers of wholly new connections can also grow.) But while this might be a convenient setup for biology, it’s not at all clear that it’s even close to the best way to achieve the functionality we need. And something that involves the equivalent of progressive network rewriting (perhaps reminiscent of our Physics Project) might well ultimately be better.

但即使在现有神经网络的框架内，目前也存在一个关键限制：当前进行的神经网络训练本质上是顺序的，每批示例的效果被反向传播以更新权重。实际上，在当前的计算机硬件上——即使考虑到 GPU——神经网络的大部分在训练期间大部分时间都处于“空闲”状态，一次只更新一部分。在某种意义上，这是因为我们当前的计算机往往具有与 CPU（或 GPU）分离的内存。但在大脑中，情况可能不同——每个“记忆元素”（即神经元）也是一个潜在活跃的计算元素。如果我们能以这种方式设置未来的计算机硬件，那么可能可以更高效地进行训练。

But even within the framework of existing neural nets there’s currently a crucial limitation: neural net training as it’s now done is fundamentally sequential, with the effects of each batch of examples being propagated back to update the weights. And indeed with current computer hardware—even taking into account GPUs—most of a neural net is “idle” most of the time during training, with just one part at a time being updated. And in a sense this is because our current computers tend to have memory that is separate from their CPUs (or GPUs). But in brains it’s presumably different—with every “memory element” (i.e. neuron) also being a potentially active computational element. And if we could set up our future computer hardware this way it might become possible to do training much more efficiently.

“一个足够大的网络当然能做任何事！” “Surely a Network That’s Big Enough Can Do Anything!”

像 ChatGPT 这样的东西的能力令人印象深刻，以至于人们可能会想象，如果只要“继续下去”并训练越来越大的神经网络，它们最终就能“做任何事情”。如果关注的是人类直接思考容易触及的事物，那么情况很可能确实如此。但过去几百年科学的教训是，有些东西可以通过形式化过程推导出来，但并非人类直接思考容易触及的。

The capabilities of something like ChatGPT seem so impressive that one might imagine that if one could just “keep going” and train larger and larger neural networks, then they’d eventually be able to “do everything”. And if one’s concerned with things that are readily accessible to immediate human thinking, it’s quite possible that this is the case. But the lesson of the past several hundred years of science is that there are things that can be figured out by formal processes, but aren’t readily accessible to immediate human thinking.

非平凡数学就是一个重要例子。但一般情况实际上是计算。而最终的问题在于计算不可约性这一现象。有些计算人们可能认为需要很多步骤才能完成，但实际上可以“约化”为相当直接的东西。但计算不可约性的发现表明，这并非总是成立。相反，存在一些过程——可能如下所示——要弄清楚发生了什么，不可避免地需要逐步骤追踪计算：

Nontrivial mathematics is one big example. But the general case is really computation. And ultimately the issue is the phenomenon of computational irreducibility. There are some computations which one might think would take many steps to do, but which can in fact be “reduced” to something quite immediate. But the discovery of computational irreducibility implies that this doesn’t always work. And instead there are processes—probably like the one below—where to work out what happens inevitably requires essentially tracing each computational step:

我们通常用大脑做的事情大概是特意选择来避免计算不可约性的。在大脑中进行数学需要特殊的努力。而实际上，仅凭大脑“思考”任何非平凡程序的操作步骤在很大程度上是不可能的。

The kinds of things that we normally do with our brains are presumably specifically chosen to avoid computational irreducibility. It takes special effort to do math in one’s brain. And it’s in practice largely impossible to “think through” the steps in the operation of any nontrivial program just in one’s brain.

但当然，我们有计算机来做这件事。有了计算机，我们可以轻松地进行长的、计算上不可约的事情。关键点在于，这些通常没有捷径。

But of course for that we have computers. And with computers we can readily do long, computationally irreducible things. And the key point is that there’s in general no shortcut for these.

是的，我们可以记住某个特定计算系统中发生的大量具体例子。也许我们甚至能看到一些（“计算可约的”）模式，从而能够进行一点泛化。但关键在于，计算不可约性意味着我们永远无法保证意外不会发生——只有通过显式执行计算，你才能知道在特定情况下实际发生了什么。

Yes, we could memorize lots of specific examples of what happens in some particular computational system. And maybe we could even see some (“computationally reducible”) patterns that would allow us to do a little generalization. But the point is that computational irreducibility means that we can never guarantee that the unexpected won’t happen—and it’s only by explicitly doing the computation that you can tell what actually happens in any particular case.

最终，可学习性与计算不可约性之间存在根本性的张力。学习实际上是通过利用规律性来压缩数据。但计算不可约性意味着，最终规律性可能存在一个极限。

And in the end there’s just a fundamental tension between learnability and computational irreducibility. Learning involves in effect compressing data by leveraging regularities. But computational irreducibility implies that ultimately there’s a limit to what regularities there may be.

实际上，可以想象将小型计算设备——如元胞自动机或图灵机——构建到像神经网络这样的可训练系统中。事实上，这样的设备可以作为神经网络的好“工具”——就像 Wolfram|Alpha 可以是 ChatGPT 的好工具一样。但计算不可约性意味着，你不能期望“进入”这些设备并让它们学习。

As a practical matter, one can imagine building little computational devices—like cellular automata or Turing machines—into trainable systems like neural nets. And indeed such devices can serve as good “tools” for the neural net—like Wolfram|Alpha can be a good tool for ChatGPT. But computational irreducibility implies that one can’t expect to “get inside” those devices and have them learn.

换句话说，能力与可训练性之间存在终极权衡：你越希望系统“真正利用”其计算能力，它就越会表现出计算不可约性，也就越不可训练。而它越从根本上可训练，它就越不能进行复杂的计算。

Or put another way, there’s an ultimate tradeoff between capability and trainability: the more you want a system to make “true use” of its computational capabilities, the more it’s going to show computational irreducibility, and the less it’s going to be trainable. And the more it’s fundamentally trainable, the less it’s going to be able to do sophisticated computation.

（对于目前的 ChatGPT，情况实际上更为极端，因为用于生成每个输出 token 的神经网络是纯“前馈”网络，没有循环，因此没有任何能力进行具有非平凡“控制流”的计算。）

(For ChatGPT as it currently is, the situation is actually much more extreme, because the neural net used to generate each token of output is a pure “feed-forward” network, without loops, and therefore has no ability to do any kind of computation with nontrivial “control flow”.)

当然，人们可能会怀疑，能够进行不可约计算是否真的重要。确实，在人类历史的大部分时间里，这并不特别重要。但我们的现代技术世界建立在至少利用数学计算——并且越来越多地利用更一般计算——的工程之上。如果我们观察自然世界，它充满了不可约计算——我们正在慢慢理解如何模拟它们并将其用于我们的技术目的。

Of course, one might wonder whether it’s actually important to be able to do irreducible computations. And indeed for much of human history it wasn’t particularly important. But our modern technological world has been built on engineering that makes use of at least mathematical computations—and increasingly also more general computations. And if we look at the natural world, it’s full of irreducible computation—that we’re slowly understanding how to emulate and use for our technological purposes.

是的，神经网络当然能够注意到自然世界中那些我们可能用“未经辅助的人类思维”也能轻易注意到的规律性。但如果我们想要解决属于数学或计算科学范畴的问题，神经网络将无法做到——除非它有效地“使用一个工具”，即一个“普通的”计算系统。

Yes, a neural net can certainly notice the kinds of regularities in the natural world that we might also readily notice with “unaided human thinking”. But if we want to work out things that are in the purview of mathematical or computational science the neural net isn’t going to be able to do it—unless it effectively “uses as a tool” an “ordinary” computational system.

但这一切中有些可能令人困惑。过去有很多任务——包括写文章——我们曾认为对计算机来说“从根本上太难了”。现在看到它们被 ChatGPT 之类的东西完成，我们往往突然认为计算机一定变得极其强大——特别是超越了它们已经基本能够做的事情（比如逐步计算元胞自动机等计算系统的行为）。

But there’s something potentially confusing about all of this. In the past there were plenty of tasks—including writing essays—that we’ve assumed were somehow “fundamentally too hard” for computers. And now that we see them done by the likes of ChatGPT we tend to suddenly think that computers must have become vastly more powerful—in particular surpassing things they were already basically able to do (like progressively computing the behavior of computational systems like cellular automata).

但这并不是正确的结论。计算不可约过程仍然是计算不可约的，并且对计算机来说仍然从根本上困难——即使计算机可以轻松计算它们的单个步骤。相反，我们应该得出的结论是，像写文章这样的任务——我们人类能做，但曾认为计算机不能做——实际上在某种意义上比我们想象的计算上更浅。

But this isn’t the right conclusion to draw. Computationally irreducible processes are still computationally irreducible, and are still fundamentally hard for computers—even if computers can readily compute their individual steps. And instead what we should conclude is that tasks—like writing essays—that we humans could do, but we didn’t think computers could do, are actually in some sense computationally easier than we thought.

换句话说，神经网络能够成功写文章的原因在于，写文章原来是一个比我们想象中“计算上更浅”的问题。在某种意义上，这让我们更接近“拥有一个理论”来解释我们人类如何做到写文章之类的事情，或者一般地处理语言。

In other words, the reason a neural net can be successful in writing an essay is because writing an essay turns out to be a “computationally shallower” problem than we thought. And in a sense this takes us closer to “having a theory” of how we humans manage to do things like writing essays, or in general deal with language.

如果你有一个足够大的神经网络，那么是的，你可能能够做到人类能轻易做的任何事情。但你无法捕捉自然世界一般能做的事情——或者我们用自然世界打造的工具能做的事情。正是这些工具的使用——无论是实践性的还是概念性的——让我们在近几个世纪超越了“纯粹未经辅助的人类思维”所能触及的边界，并为人类目的捕捉了物理和计算宇宙中更多的东西。

If you had a big enough neural net then, yes, you might be able to do whatever humans can readily do. But you wouldn’t capture what the natural world in general can do—or that the tools that we’ve fashioned from the natural world can do. And it’s the use of those tools—both practical and conceptual—that have allowed us in recent centuries to transcend the boundaries of what’s accessible to “pure unaided human thought”, and capture for human purposes more of what’s out there in the physical and computational universe.

嵌入的概念 The Concept of Embeddings

神经网络——至少按照目前的设置——从根本上说是基于数字的。因此，如果我们要用它们来处理文本之类的东西，就需要一种用数字表示文本的方法。当然，我们可以（基本上就像 ChatGPT 所做的那样）首先为字典中的每个单词分配一个数字。但有一个重要的想法——例如，它是 ChatGPT 的核心——超越了这一点。这就是“嵌入”的概念。我们可以将嵌入视为一种尝试用数字数组来表示某物的“本质”的方法——其特性是“附近的事物”由附近的数字表示。

Neural nets—at least as they’re currently set up—are fundamentally based on numbers. So if we’re going to to use them to work on something like text we’ll need a way to represent our text with numbers. And certainly we could start (essentially as ChatGPT does) by just assigning a number to every word in the dictionary. But there’s an important idea—that’s for example central to ChatGPT—that goes beyond that. And it’s the idea of “embeddings”. One can think of an embedding as a way to try to represent the “essence” of something by an array of numbers—with the property that “nearby things” are represented by nearby numbers.

因此，例如，我们可以将词嵌入视为试图将单词排列在一种“意义空间”中，其中在意义上“相近”的单词在嵌入中出现在附近。实际使用的嵌入——比如在 ChatGPT 中——往往涉及大量的数字列表。但如果我们投影到二维空间，我们可以展示嵌入如何排列单词的示例：

And so, for example, we can think of a word embedding as trying to lay out words in a kind of “meaning space” in which words that are somehow “nearby in meaning” appear nearby in the embedding. The actual embeddings that are used—say in ChatGPT—tend to involve large lists of numbers. But if we project down to 2D, we can show examples of how words are laid out by the embedding:

是的，我们所看到的在捕捉典型的日常印象方面表现得非常好。但是我们如何构建这样的嵌入呢？大致思路是查看大量文本（这里是从网络获取的 50 亿个单词），然后观察不同单词出现的“环境”有多“相似”。因此，例如，“alligator”和“crocodile”通常会在其他方面相似的句子中几乎互换地出现，这意味着它们会在嵌入中被放置在附近。但“turnip”和“eagle”往往不会出现在其他相似的句子中，因此它们会在嵌入中被放置得很远。

And, yes, what we see does remarkably well in capturing typical everyday impressions. But how can we construct such an embedding? Roughly the idea is to look at large amounts of text (here 5 billion words from the web) and then see “how similar” the “environments” are in which different words appear. So, for example, “alligator” and “crocodile” will often appear almost interchangeably in otherwise similar sentences, and that means they’ll be placed nearby in the embedding. But “turnip” and “eagle” won’t tend to appear in otherwise similar sentences, so they’ll be placed far apart in the embedding.

但是，如何使用神经网络实际实现这样的东西呢？让我们先讨论嵌入，不是针对单词，而是针对图像。我们希望找到某种方法，用数字列表来表征图像，使得“我们认为相似的图像”被分配相似的数字列表。

But how does one actually implement something like this using neural nets? Let’s start by talking about embeddings not for words, but for images. We want to find some way to characterize images by lists of numbers in such a way that “images we consider similar” are assigned similar lists of numbers.

我们如何判断是否应该“认为图像相似”呢？好吧，如果我们的图像是手写数字，那么如果两个图像是同一个数字，我们可能会“认为它们相似”。之前我们讨论过一个经过训练可以识别手写数字的神经网络。我们可以将这个神经网络视为设置成在其最终输出中将图像放入 10 个不同的箱子中，每个数字一个。

How do we tell if we should “consider images similar”? Well, if our images are, say, of handwritten digits we might “consider two images similar” if they are of the same digit. Earlier we discussed a neural net that was trained to recognize handwritten digits. And we can think of this neural net as being set up so that in its final output it puts images into 10 different bins, one for each digit.

但是，如果我们在最终做出“它是‘4’”的决定之前“截取”神经网络内部发生的事情呢？我们可能期望在神经网络内部有一些数字将图像表征为“主要是 4 像但有点 2 像”之类的。想法是提取这些数字作为嵌入的元素。

But what if we “intercept” what’s going on inside the neural net before the final “it’s a ‘4’” decision is made? We might expect that inside the neural net there are numbers that characterize images as being “mostly 4-like but a bit 2-like” or some such. And the idea is to pick up such numbers to use as elements in an embedding.

所以这就是概念。我们不是直接尝试表征“哪个图像与哪个其他图像接近”，而是考虑一个明确定义的任务（在本例中是数字识别），我们可以为此获得明确的训练数据——然后利用这样一个事实：在执行此任务时，神经网络隐式地必须做出相当于“接近性决策”的事情。因此，我们不必显式地讨论“图像的接近性”，而只是讨论图像代表哪个数字的具体问题，然后“让神经网络”隐式地确定这对“图像的接近性”意味着什么。

So here’s the concept. Rather than directly trying to characterize “what image is near what other image”, we instead consider a well-defined task (in this case digit recognition) for which we can get explicit training data—then use the fact that in doing this task the neural net implicitly has to make what amount to “nearness decisions”. So instead of us ever explicitly having to talk about “nearness of images” we’re just talking about the concrete question of what digit an image represents, and then we’re “leaving it to the neural net” to implicitly determine what that implies about “nearness of images”.

那么，对于数字识别网络，这在更详细的层面上是如何工作的呢？我们可以将网络视为由 11 个连续的层组成，我们可以像这样用图标来概括（激活函数显示为单独的层）：

So how in more detail does this work for the digit recognition network? We can think of the network as consisting of 11 successive layers, that we might summarize iconically like this (with activation functions shown as separate layers):

一开始，我们将实际图像（由像素值的二维数组表示）输入到第一层。最后——从最后一层——我们得到一个包含 10 个值的数组，我们可以将其视为表示网络“有多确定”该图像对应于数字 0 到 9 中的每一个。

At the beginning we’re feeding into the first layer actual images, represented by 2D arrays of pixel values. And at the end—from the last layer—we’re getting out an array of 10 values, which we can think of saying “how certain” the network is that the image corresponds to each of the digits 0 through 9.

输入图像，最后一层神经元的值为：

Feed in the image and the values of the neurons in that last layer are:

换句话说，此时神经网络“极其确定”该图像是 4——要实际获得输出“4”，我们只需选出具有最大值的神经元的位置。

In other words, the neural net is by this point “incredibly certain” that this image is a 4—and to actually get the output “4” we just have to pick out the position of the neuron with the largest value.

但是，如果我们提前一步观察呢？网络中的最后一个操作是所谓的 softmax，它试图“强制确定性”。但在应用 softmax 之前，神经元的值为：

But what if we look one step earlier? The very last operation in the network is a so-called softmax which tries to “force certainty”. But before that’s been applied the values of the neurons are:

代表“4”的神经元仍然具有最高的数值。但其他神经元的值中也包含信息。我们可以预期，这个数字列表在某种意义上可以用来表征图像的“本质”——从而提供我们可以用作嵌入的东西。因此，例如，这里的每个 4 都有一个略有不同的“签名”（或“特征嵌入”）——都与 8 的签名非常不同：

The neuron representing “4” still has the highest numerical value. But there’s also information in the values of the other neurons. And we can expect that this list of numbers can in a sense be used to characterize the “essence” of the image—and thus to provide something we can use as an embedding. And so, for example, each of the 4’s here has a slightly different “signature” (or “feature embedding”)—all very different from the 8’s:

这里我们基本上使用 10 个数字来表征图像。但通常使用更多数字会更好。例如，在我们的数字识别网络中，我们可以通过接入前一层获得一个包含 500 个数字的数组。这很可能是一个合理的数组，可以用作“图像嵌入”。

Here we’re essentially using 10 numbers to characterize our images. But it’s often better to use much more than that. And for example in our digit recognition network we can get an array of 500 numbers by tapping into the preceding layer. And this is probably a reasonable array to use as an “image embedding”.

如果我们想对手写数字的“图像空间”进行显式可视化，我们需要“降低维度”，实际上是将我们得到的 500 维向量投影到，比如说，三维空间：

If we want to make an explicit visualization of “image space” for handwritten digits we need to “reduce the dimension”, effectively by projecting the 500-dimensional vector we’ve got into, say, 3D space:

我们刚刚讨论了如何基于通过确定图像是否（根据我们的训练集）对应于相同的手写数字来识别图像的相似性，从而为图像创建表征（以及因此的嵌入）。如果我们有一个训练集，可以识别每个图像属于 5000 种常见物体类型（猫、狗、椅子……）中的哪一种，那么我们可以对图像做同样的事情，而且更加通用。通过这种方式，我们可以制作一个图像嵌入，它由我们对常见物体的识别“锚定”，然后根据神经网络的行为“围绕它进行泛化”。关键是，只要这种行为与我们人类感知和解释图像的方式一致，这最终将成为一个“对我们来说看起来正确”的嵌入，并且在执行“类似人类判断”的任务时在实践中很有用。

We’ve just talked about creating a characterization (and thus embedding) for images based effectively on identifying the similarity of images by determining whether (according to our training set) they correspond to the same handwritten digit. And we can do the same thing much more generally for images if we have a training set that identifies, say, which of 5000 common types of object (cat, dog, chair, …) each image is of. And in this way we can make an image embedding that’s “anchored” by our identification of common objects, but then “generalizes around that” according to the behavior of the neural net. And the point is that insofar as that behavior aligns with how we humans perceive and interpret images, this will end up being an embedding that “seems right to us”, and is useful in practice in doing “human-judgement-like” tasks.

好的，那么我们如何遵循相同的方法来找到单词的嵌入呢？关键是从一个我们可以轻松进行训练的关于单词的任务开始。标准的此类任务是“单词预测”。想象我们得到“the _ cat”。基于一个大型文本语料库（例如，网络的文本内容），不同单词“填空”的概率是多少？或者，给定“_ black _”，不同“侧翼词”的概率是多少？

OK, so how do we follow the same kind of approach to find embeddings for words? The key is to start from a task about words for which we can readily do training. And the standard such task is “word prediction”. Imagine we’re given “the _ cat”. Based on a large corpus of text (say, the text content of the web), what are the probabilities for different words that might “fill in the blank”? Or, alternatively, given “_ black _” what are the probabilities for different “flanking words”?

我们如何为神经网络设置这个问题？最终，我们必须用数字来表述一切。一种方法是简单地为英语中大约 50,000 个常用单词中的每一个分配一个唯一的数字。例如，“the”可能是 914，“ cat”（前面有一个空格）可能是 3542。（这些是 GPT-2 使用的实际数字。）因此，对于“the _ cat”问题，我们的输入可能是 {914, 3542}。输出应该是什么样的？嗯，它应该是一个大约 50,000 个数字的列表，有效地给出每个可能的“填空”单词的概率。再一次，为了找到嵌入，我们想要在神经网络“得出结论”之前“截取”其“内部”——然后提取那里出现的数字列表，我们可以将其视为“表征每个单词”。

How do we set this problem up for a neural net? Ultimately we have to formulate everything in terms of numbers. And one way to do this is just to assign a unique number to each of the 50,000 or so common words in English. So, for example, “the” might be 914, and “ cat” (with a space before it) might be 3542. (And these are the actual numbers used by GPT-2.) So for the “the _ cat” problem, our input might be {914, 3542}. What should the output be like? Well, it should be a list of 50,000 or so numbers that effectively give the probabilities for each of the possible “fill-in” words. And once again, to find an embedding, we want to “intercept” the “insides” of the neural net just before it “reaches its conclusion”—and then pick up the list of numbers that occur there, and that we can think of as “characterizing each word”.

好的，那么这些表征看起来像什么？在过去的 10 年里，已经开发了一系列不同的系统（word2vec、GloVe、BERT、GPT……），每个都基于不同的神经网络方法。但最终，所有这些系统都处理单词，并用数百到数千个数字的列表来表征它们。

OK, so what do those characterizations look like? Over the past 10 years there’ve been a sequence of different systems developed (word2vec, GloVe, BERT, GPT, …), each based on a different neural net approach. But ultimately all of them take words and characterize them by lists of hundreds to thousands of numbers.

在原始形式中，这些“嵌入向量”相当不具信息量。例如，以下是 GPT-2 为三个特定单词生成的原始嵌入向量：

In their raw form, these “embedding vectors” are quite uninformative. For example, here’s what GPT-2 produces as the raw embedding vectors for three specific words:

如果我们做诸如测量这些向量之间的距离之类的事情，那么我们可以找到诸如单词的“接近度”之类的东西。稍后我们将更详细地讨论我们可能认为这些嵌入的“认知”意义。但现在，主要的一点是我们有一种方法可以将单词有用且有效地转化为“神经网络友好”的数字集合。

If we do things like measure distances between these vectors, then we can find things like “nearnesses” of words. Later we’ll discuss in more detail what we might consider the “cognitive” significance of such embeddings. But for now the main point is that we have a way to usefully turn words into “neural-net-friendly” collections of numbers.

但实际上，我们可以更进一步，不仅用数字集合来表征单词；我们还可以对单词序列，甚至整个文本块这样做。在 ChatGPT 内部，它就是这样处理事物的。它获取到目前为止的文本，并生成一个嵌入向量来表示它。然后它的目标是找到可能出现的下一个单词的概率。它将其答案表示为一个数字列表，该列表基本上给出了大约 50,000 个可能单词中每一个的概率。

But actually we can go further than just characterizing words by collections of numbers; we can also do this for sequences of words, or indeed whole blocks of text. And inside ChatGPT that’s how it’s dealing with things. It takes the text it’s got so far, and generates an embedding vector to represent it. Then its goal is to find the probabilities for different words that might occur next. And it represents its answer for this as a list of numbers that essentially give the probabilities for each of the 50,000 or so possible words.

（严格来说，ChatGPT 处理的不是单词，而是“词元”——方便的语言单元，可能是完整的单词，也可能只是像“pre”或“ing”或“ized”这样的片段。使用词元使 ChatGPT 更容易处理罕见的、复合的和非英语的单词，并且有时，无论好坏，还能创造新词。）

(Strictly, ChatGPT does not deal with words, but rather with “tokens”—convenient linguistic units that might be whole words, or might just be pieces like “pre” or “ing” or “ized”. Working with tokens makes it easier for ChatGPT to handle rare, compound and non-English words, and, sometimes, for better or worse, to invent new words.)

ChatGPT 内部机制 Inside ChatGPT

好了，我们终于准备好讨论 ChatGPT 内部是什么了。是的，它本质上是一个巨大的神经网络——目前是所谓的 GPT-3 网络的一个版本，拥有 1750 亿个权重。在很多方面，这个神经网络与我们讨论过的其他神经网络非常相似。但它是一个特别为处理语言而设置的神经网络。其最显著的特征是一种称为“Transformer”的神经网络架构。

OK, so we’re finally ready to discuss what’s inside ChatGPT. And, yes, ultimately, it’s a giant neural net—currently a version of the so-called GPT-3 network with 175 billion weights. In many ways this is a neural net very much like the other ones we’ve discussed. But it’s a neural net that’s particularly set up for dealing with language. And its most notable feature is a piece of neural net architecture called a “transformer”.

在我们上面讨论的第一个神经网络中，任何给定层的每个神经元基本上都与前一层的每个神经元相连（至少有一定的权重）。但是，如果处理的数据具有特定的已知结构，这种全连接网络（可能）就过于冗余了。因此，例如，在处理图像的早期阶段，通常使用所谓的卷积神经网络（“卷积网络”），其中神经元实际上被排列在与图像像素类似的网格上，并且只与网格上附近的神经元相连。

In the first neural nets we discussed above, every neuron at any given layer was basically connected (at least with some weight) to every neuron on the layer before. But this kind of fully connected network is (presumably) overkill if one’s working with data that has particular, known structure. And thus, for example, in the early stages of dealing with images, it’s typical to use so-called convolutional neural nets (“convnets”) in which neurons are effectively laid out on a grid analogous to the pixels in the image—and connected only to neurons nearby on the grid.

Transformer 的思想是对构成文本的标记序列做至少类似的事情。但是，Transformer 并没有仅仅定义序列中一个固定的连接区域，而是引入了“注意力”的概念——以及“更关注”序列某些部分而非其他部分的想法。也许有一天，直接启动一个通用神经网络并通过训练进行所有定制会变得合理。但至少目前来看，在实践中“模块化”似乎是至关重要的——正如 Transformer 所做的那样，可能我们的大脑也是如此。

The idea of transformers is to do something at least somewhat similar for sequences of tokens that make up a piece of text. But instead of just defining a fixed region in the sequence over which there can be connections, transformers instead introduce the notion of “attention”—and the idea of “paying attention” more to some parts of the sequence than others. Maybe one day it’ll make sense to just start a generic neural net and do all customization through training. But at least as of now it seems to be critical in practice to “modularize” things—as transformers do, and probably as our brains also do.

好了，那么 ChatGPT（或者更确切地说，它所基于的 GPT-3 网络）实际上做了什么呢？回想一下，它的总体目标是根据其训练（包括查看来自网络等数十亿页文本）所看到的内容，以“合理”的方式继续文本。因此，在任何给定时刻，它都有一定量的文本——其目标是为要添加的下一个标记提出合适的选择。

OK, so what does ChatGPT (or, rather, the GPT-3 network on which it’s based) actually do? Recall that its overall goal is to continue text in a “reasonable” way, based on what it’s seen from the training it’s had (which consists in looking at billions of pages of text from the web, etc.) So at any given point, it’s got a certain amount of text—and its goal is to come up with an appropriate choice for the next token to add.

它分三个基本阶段运行。首先，它获取与到目前为止的文本相对应的标记序列，并找到一个嵌入（即一个数字数组）来表示这些标记。然后，它对这个嵌入进行操作——以“标准神经网络方式”，值在网络的连续层中“传播”——以产生一个新的嵌入（即一个新的数字数组）。然后，它取这个数组的最后一部分，并从中生成一个大约 50,000 个值的数组，这些值转化为不同可能的下一个标记的概率。（是的，碰巧使用的标记数量与英语中常见单词的数量大致相同，尽管只有大约 3000 个标记是完整的单词，其余的是片段。）

It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embedding—in a “standard neural net way”, with values “rippling through” successive layers in a network—to produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.)

一个关键点是，这个流程的每个部分都由一个神经网络实现，其权重由网络的端到端训练决定。换句话说，实际上除了整体架构之外，没有任何东西是“显式设计的”；一切都是从训练数据中“学习”而来的。

A critical point is that every part of this pipeline is implemented by a neural network, whose weights are determined by end-to-end training of the network. In other words, in effect nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data.

然而，在架构的设置方式中有很多细节——反映了各种经验和神经网络的知识。而且——尽管这肯定会深入细节——我认为讨论其中一些细节是有用的，尤其是为了了解构建像 ChatGPT 这样的东西需要什么。

There are, however, plenty of details in the way the architecture is set up—reflecting all sorts of experience and neural net lore. And—even though this is definitely going into the weeds—I think it’s useful to talk about some of those details, not least to get a sense of just what goes into building something like ChatGPT.

首先是嵌入模块。以下是 GPT-2 的 Wolfram 语言示意图：

First comes the embedding module. Here’s a schematic Wolfram Language representation for it for GPT-2:

输入是一个包含 n 个标记的向量（如上节所述，用从 1 到大约 50,000 的整数表示）。每个标记都被转换（通过一个单层神经网络）成一个嵌入向量（GPT-2 的长度为 768，ChatGPT 的 GPT-3 的长度为 12,288）。同时，有一条“辅助路径”获取标记的（整数）位置序列，并从这些整数创建另一个嵌入向量。最后，来自标记值和标记位置的嵌入向量被相加——以从嵌入模块产生最终的嵌入向量序列。

The input is a vector of _n_ tokens (represented as in the previous section by integers from 1 to about 50,000). Each of these tokens is converted (by a single-layer neural net) into an embedding vector (of length 768 for GPT-2 and 12,288 for ChatGPT’s GPT-3). Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens, and from these integers creates another embedding vector. And finally the embedding vectors from the token value and the token position are added together—to produce the final sequence of embedding vectors from the embedding module.

为什么只是将标记值和标记位置的嵌入向量相加？我认为这没有什么特别的科学道理。只是尝试了各种不同的方法，而这是似乎有效的一种。并且这是神经网络知识的一部分——在某种意义上——只要我们的设置“大致正确”，通常可以通过进行足够的训练来微调细节，而无需真正“在工程层面理解”神经网络最终是如何配置自身的。

Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.

以下是嵌入模块对字符串 _hello hello hello hello hello hello hello hello hello hello bye bye bye bye bye bye bye bye bye bye_ 的操作：

Here’s what the embedding module does, operating on the string _hello hello hello hello hello hello hello hello hello hello bye bye bye bye bye bye bye bye bye bye_:

每个标记的嵌入向量元素沿页面向下显示，横向我们看到先是“_hello_”嵌入的序列，然后是“_bye_”嵌入的序列。上面的第二个数组是位置嵌入——其看似随机的结构只是“碰巧被学习到的”（这里是在 GPT-2 中）。

The elements of the embedding vector for each token are shown down the page, and across the page we see first a run of “_hello_” embeddings, followed by a run of “_bye_” ones. The second array above is the positional embedding—with its somewhat-random-looking structure being just what “happened to be learned” (in this case in GPT-2).

好了，在嵌入模块之后是 Transformer 的“主要事件”：一系列所谓的“注意力块”（GPT-2 有 12 个，ChatGPT 的 GPT-3 有 96 个）。这一切都相当复杂——让人想起典型的大型难以理解的工程系统，或者，就此而言，生物系统。但无论如何，这里是一个单个“注意力块”（对于 GPT-2）的示意图：

OK, so after the embedding module comes the “main event” of the transformer: a sequence of so-called “attention blocks” (12 for GPT-2, 96 for ChatGPT’s GPT-3). It’s all pretty complicated—and reminiscent of typical large hard-to-understand engineering systems, or, for that matter, biological systems. But anyway, here’s a schematic representation of a single “attention block” (for GPT-2):

在每个这样的注意力块内，有一组“注意力头”（GPT-2 有 12 个，ChatGPT 的 GPT-3 有 96 个）——每个注意力头独立地对嵌入向量中不同块的值进行操作。（是的，我们不知道为什么拆分嵌入向量是个好主意，或者它的不同部分“意味着”什么；这只是那些“被发现有效”的事情之一。）

Within each such attention block there are a collection of “attention heads” (12 for GPT-2, 96 for ChatGPT’s GPT-3)—each of which operates independently on different chunks of values in the embedding vector. (And, yes, we don’t know any particular reason why it’s a good idea to split up the embedding vector, or what the different parts of it “mean”; this is just one of those things that’s been “found to work”.)

好了，那么注意力头做什么呢？基本上，它们是一种“回顾”标记序列（即到目前为止生成的文本）的方式，并以对寻找下一个标记有用的形式“打包过去”。在上面的第一节中，我们讨论了使用 2-gram 概率根据前一个词来选择词。Transformer 中的“注意力”机制所做的是允许“关注”甚至更早的词——从而可能捕捉到，例如，动词可以指代句子中出现在它们前面多个词的名词的方式。

OK, so what do the attention heads do? Basically they’re a way of “looking back” in the sequence of tokens (i.e. in the text produced so far), and “packaging up the past” in a form that’s useful for finding the next token. In the first section above we talked about using 2-gram probabilities to pick words based on their immediate predecessors. What the “attention” mechanism in transformers does is to allow “attention to” even much earlier words—thus potentially capturing the way, say, verbs can refer to nouns that appear many words before them in a sentence.

在更详细的层面上，注意力头所做的是以一定的权重重新组合与不同标记相关的嵌入向量中的块。因此，例如，第一个注意力块（在 GPT-2 中）的 12 个注意力头对于上面的“_hello_, _bye_”字符串具有以下（“一直回溯到标记序列开头”）的“重组权重”模式：

At a more detailed level, what an attention head does is to recombine chunks in the embedding vectors associated with different tokens, with certain weights. And so, for example, the 12 attention heads in the first attention block (in GPT-2) have the following (“look-back-all-the-way-to-the-beginning-of-the-sequence-of-tokens”) patterns of “recombination weights” for the “_hello_, _bye_” string above:

经过注意力头处理后，得到的“重新加权的嵌入向量”（GPT-2 长度为 768，ChatGPT 的 GPT-3 长度为 12,288）通过一个标准的“全连接”神经网络层。很难理解这一层在做什么。但这里是它使用的 768×768 权重矩阵的图（这里是 GPT-2）：

After being processed by the attention heads, the resulting “re-weighted embedding vector” (of length 768 for GPT-2 and length 12,288 for ChatGPT’s GPT-3) is passed through a standard “fully connected” neural net layer. It’s hard to get a handle on what this layer is doing. But here’s a plot of the 768×768 matrix of weights it’s using (here for GPT-2):

取 64×64 移动平均，一些（类似随机游走的）结构开始出现：

Taking 64×64 moving averages, some (random-walk-ish) structure begins to emerge:

是什么决定了这种结构？最终，它可能是人类语言特征的某种“神经网络编码”。但截至目前，这些特征可能是什么还不得而知。实际上，我们正在“打开 ChatGPT（或至少 GPT-2）的大脑”，并发现，是的，里面很复杂，我们也不理解它——尽管最终它产生了可识别的人类语言。

What determines this structure? Ultimately it’s presumably some “neural net encoding” of features of human language. But as of now, what those features might be is quite unknown. In effect, we’re “opening up the brain of ChatGPT” (or at least GPT-2) and discovering, yes, it’s complicated in there, and we don’t understand it—even though in the end it’s producing recognizable human language.

好了，经过一个注意力块后，我们得到了一个新的嵌入向量——然后它被依次传递到更多的注意力块（GPT-2 总共 12 个；GPT-3 总共 96 个）。每个注意力块都有自己特定的“注意力”和“全连接”权重模式。以下是 GPT-2 对于“hello, bye”输入，第一个注意力头的注意力权重序列：

OK, so after going through one attention block, we’ve got a new embedding vector—which is then successively passed through additional attention blocks (a total of 12 for GPT-2; 96 for GPT-3). Each attention block has its own particular pattern of “attention” and “fully connected” weights. Here for GPT-2 are the sequence of attention weights for the “hello, bye” input, for the first attention head:

以下是全连接层的（移动平均）“矩阵”：

And here are the (moving-averaged) “matrices” for the fully connected layers:

奇怪的是，尽管不同注意力块中的这些“权重矩阵”看起来非常相似，但权重大小的分布可能有些不同（并且不总是高斯分布）：

Curiously, even though these “matrices of weights” in different attention blocks look quite similar, the distributions of the sizes of weights can be somewhat different (and are not always Gaussian):

那么，经过所有这些注意力块后，Transformer 的净效果是什么？本质上，它是将原始的一组标记序列的嵌入转换为最终的集合。而 ChatGPT 工作的特定方式就是取这个集合中的最后一个嵌入，并“解码”它以产生下一个标记应该是什么的概率列表。

So after going through all these attention blocks what is the net effect of the transformer? Essentially it’s to transform the original collection of embeddings for the sequence of tokens to a final collection. And the particular way ChatGPT works is then to pick up the last embedding in this collection, and “decode” it to produce a list of probabilities for what token should come next.

这就是 ChatGPT 内部的大致情况。它可能看起来很复杂（尤其是因为其许多不可避免的有点随意的“工程选择”），但实际上所涉及的最终元素非常简单。因为最终我们处理的只是一个由“人工神经元”组成的神经网络，每个神经元执行简单的操作：获取一组数值输入，然后以一定的权重将它们组合起来。

So that’s in outline what’s inside ChatGPT. It may seem complicated (not least because of its many inevitably somewhat arbitrary “engineering choices”), but actually the ultimate elements involved are remarkably simple. Because in the end what we’re dealing with is just a neural net made of “artificial neurons”, each doing the simple operation of taking a collection of numerical inputs, and then combining them with certain weights.

ChatGPT 的原始输入是一个数字数组（到目前为止标记的嵌入向量），当 ChatGPT“运行”以产生一个新标记时，发生的只是这些数字在神经网络的层中“传播”，每个神经元“做自己的事”并将结果传递给下一层的神经元。没有循环或“回溯”。一切都通过网络“前馈”。

The original input to ChatGPT is an array of numbers (the embedding vectors for the tokens so far), and what happens when ChatGPT “runs” to produce a new token is just that these numbers “ripple through” the layers of the neural net, with each neuron “doing its thing” and passing the result to neurons on the next layer. There’s no looping or “going back”. Everything just “feeds forward” through the network.

这与典型的计算系统（如图灵机）非常不同，在图灵机中，结果被相同的计算元素反复“重新处理”。在这里——至少在生成给定输出标记时——每个计算元素（即神经元）只使用一次。

It’s a very different setup from a typical computational system—like a Turing machine—in which results are repeatedly “reprocessed” by the same computational elements. Here—at least in generating a given token of output—each computational element (i.e. neuron) is used only once.

但即使在 ChatGPT 中，在某种意义上仍然存在一个重用计算元素的“外循环”。因为当 ChatGPT 要生成一个新标记时，它总是“读取”（即作为输入）它之前的所有标记序列，包括 ChatGPT 自己之前“写入”的标记。我们可以认为这种设置意味着 ChatGPT——至少在其最外层——涉及一个“反馈循环”，尽管每次迭代都明确地作为出现在它生成的文本中的一个标记而可见。

But there is in a sense still an “outer loop” that reuses computational elements even in ChatGPT. Because when ChatGPT is going to generate a new token, it always “reads” (i.e. takes as input) the whole sequence of tokens that come before it, including tokens that ChatGPT itself has “written” previously. And we can think of this setup as meaning that ChatGPT does—at least at its outermost level—involve a “feedback loop”, albeit one in which every iteration is explicitly visible as a token that appears in the text that it generates.

但让我们回到 ChatGPT 的核心：被反复用于生成每个标记的神经网络。在某种程度上，它非常简单：一整套相同的人工神经元。网络的某些部分仅由（“全连接”）神经元层组成，其中给定层上的每个神经元都与前一层的每个神经元相连（具有一定的权重）。但特别是由于其 Transformer 架构，ChatGPT 具有结构更复杂的部分，其中只有不同层上的特定神经元相连。（当然，仍然可以说“所有神经元都相连”——但有些权重为零。）

But let’s come back to the core of ChatGPT: the neural net that’s being repeatedly used to generate each token. At some level it’s very simple: a whole collection of identical artificial neurons. And some parts of the network just consist of (“fully connected”) layers of neurons in which every neuron on a given layer is connected (with some weight) to every neuron on the layer before. But particularly with its transformer architecture, ChatGPT has parts with more structure, in which only specific neurons on different layers are connected. (Of course, one could still say that “all neurons are connected”—but some just have zero weight.)

此外，ChatGPT 中神经网络的某些方面并不最自然地被认为仅由“同质”层组成。例如——正如上面的标志性总结所示——在注意力块内部，有些地方会制作输入数据的“多个副本”，每个副本随后经过不同的“处理路径”，可能涉及不同数量的层，并且只在稍后重新组合。但是，虽然这可能是对正在发生的事情的一种方便表示，但至少在原则上，总是可以认为“密集填充”层，只是让一些权重为零。

In addition, there are aspects of the neural net in ChatGPT that aren’t most naturally thought of as just consisting of “homogeneous” layers. And for example—as the iconic summary above indicates—inside an attention block there are places where “multiple copies are made” of incoming data, each then going through a different “processing path”, potentially involving a different number of layers, and only later recombining. But while this may be a convenient representation of what’s going on, it’s always at least in principle possible to think of “densely filling in” layers, but just having some weights be zero.

如果查看通过 ChatGPT 的最长路径，大约涉及 400 个（核心）层——在某种程度上数量并不大。但是有数百万个神经元——总共有 1750 亿个连接，因此有 1750 亿个权重。需要认识到的一点是，每次 ChatGPT 生成一个新标记时，它都必须进行涉及每一个这些权重的计算。在实现上，这些计算可以按层组织成高度并行的数组操作，可以方便地在 GPU 上完成。但是对于生成的每个标记，仍然必须进行 1750 亿次计算（最终还要多一点）——所以，是的，ChatGPT 生成一段长文本需要一段时间也就不足为奇了。

If one looks at the longest path through ChatGPT, there are about 400 (core) layers involved—in some ways not a huge number. But there are millions of neurons—with a total of 175 billion connections and therefore 175 billion weights. And one thing to realize is that every time ChatGPT generates a new token, it has to do a calculation involving every single one of these weights. Implementationally these calculations can be somewhat organized “by layer” into highly parallel array operations that can conveniently be done on GPUs. But for each token that’s produced, there still have to be 175 billion calculations done (and in the end a bit more)—so that, yes, it’s not surprising that it can take a while to generate a long piece of text with ChatGPT.

但最终，令人惊奇的是，所有这些操作——尽管单独来看很简单——却能够共同出色地完成如此“类人”的文本生成任务。必须再次强调，（至少据我们所知）没有“最终的理论原因”可以解释为什么这样的事情应该有效。事实上，正如我们将要讨论的，我认为我们必须将其视为一项——可能令人惊讶的——科学发现：在像 ChatGPT 这样的神经网络中，不知何故能够捕捉到人类大脑在生成语言时所做事情的本质。

But in the end, the remarkable thing is that all these operations—individually as simple as they are—can somehow together manage to do such a good “human-like” job of generating text. It has to be emphasized again that (at least so far as we know) there’s no “ultimate theoretical reason” why anything like this should work. And in fact, as we’ll discuss, I think we have to view this as a—potentially surprising—scientific discovery: that somehow in a neural net like ChatGPT’s it’s possible to capture the essence of what human brains manage to do in generating language.

ChatGPT 的训练 The Training of ChatGPT

好了，我们现在已经概述了 ChatGPT 在建立后是如何工作的。但它是如何建立的呢？其神经网络中所有 1750 亿个权重是如何确定的？基本上，它们是基于人类编写的大量文本语料库（网络、书籍等）进行超大规模训练的结果。正如我们所说，即使有了所有这些训练数据，神经网络能否成功生成“类人”文本也远非显而易见。而且，再一次，似乎需要详细的工程细节才能实现这一点。但 ChatGPT 的巨大惊喜和发现是，这是完全可能的。而且，实际上，一个“仅”有 1750 亿个权重的神经网络可以生成人类所写文本的“合理模型”。

OK, so we’ve now given an outline of how ChatGPT works once it’s set up. But how did it get set up? How were all those 175 billion weights in its neural net determined? Basically they’re the result of very large-scale training, based on a huge corpus of text—on the web, in books, etc.—written by humans. As we’ve said, even given all that training data, it’s certainly not obvious that a neural net would be able to successfully produce “human-like” text. And, once again, there seem to be detailed pieces of engineering needed to make that happen. But the big surprise—and discovery—of ChatGPT is that it’s possible at all. And that—in effect—a neural net with “just” 175 billion weights can make a “reasonable model” of text humans write.

在现代，有大量人类编写的文本以数字形式存在。公共网络至少有数十亿个人类编写的页面，总共可能有约一万亿词的文本。如果包括非公开网页，这个数字可能至少大 100 倍。到目前为止，已有超过 500 万本数字化书籍可供使用（在已出版的约 1 亿本书中），这又提供了约 1000 亿词的文本。这还不包括视频中语音转录的文本等。（就我个人而言，我一生发表的出版物总字数略低于 300 万词，过去 30 年我写了约 1500 万词的电子邮件，总共可能打了 5000 万词——而在过去几年里，我在直播中说了超过 1000 万词。是的，我将用所有这些来训练一个机器人。）

In modern times, there’s lots of text written by humans that’s out there in digital form. The public web has at least several billion human-written pages, with altogether perhaps a trillion words of text. And if one includes non-public webpages, the numbers might be at least 100 times larger. So far, more than 5 million digitized books have been made available (out of 100 million or so that have ever been published), giving another 100 billion or so words of text. And that’s not even mentioning text derived from speech in videos, etc. (As a personal comparison, my total lifetime output of published material has been a bit under 3 million words, and over the past 30 years I’ve written about 15 million words of email, and altogether typed perhaps 50 million words—and in just the past couple of years I’ve spoken more than 10 million words on livestreams. And, yes, I’ll train a bot from all of that.)

但是，好吧，有了所有这些数据，如何从中训练一个神经网络呢？基本过程与我们上面讨论的简单示例非常相似。你提供一批示例，然后调整网络中的权重，以最小化网络在这些示例上的误差（“损失”）。从误差进行“反向传播”的主要昂贵之处在于，每次这样做时，网络中的每个权重通常都会发生至少微小的变化，而需要处理的权重非常多。（实际的“反向计算”通常只比前向计算难一个小的常数因子。）

But, OK, given all this data, how does one train a neural net from it? The basic process is very much as we discussed it in the simple examples above. You present a batch of examples, and then you adjust the weights in the network to minimize the error (“loss”) that the network makes on those examples. The main thing that’s expensive about “back propagating” from the error is that each time you do this, every weight in the network will typically change at least a tiny bit, and there are just a lot of weights to deal with. (The actual “back computation” is typically only a small constant factor harder than the forward one.)

使用现代 GPU 硬件，可以轻松并行计算数千个示例批次的结果。但在实际更新神经网络中的权重时，当前的方法要求我们基本上逐批次进行。（是的，这可能是实际大脑——其结合了计算和存储元素——目前至少在架构上具有优势的地方。）

With modern GPU hardware, it’s straightforward to compute the results from batches of thousands of examples in parallel. But when it comes to actually updating the weights in the neural net, current methods require one to do this basically batch by batch. (And, yes, this is probably where actual brains—with their combined computation and memory elements—have, for now, at least an architectural advantage.)

即使在我们之前讨论的学习数值函数这种看似简单的情况下，我们也发现通常需要数百万个示例才能成功训练一个网络，至少是从头开始训练。那么，这意味着我们需要多少个示例来训练一个“类人语言”模型呢？似乎没有基本的“理论”方法可以知道。但实际上，ChatGPT 是在几千亿词的文本上成功训练的。

Even in the seemingly simple cases of learning numerical functions that we discussed earlier, we found we often had to use millions of examples to successfully train a network, at least from scratch. So how many examples does this mean we’ll need in order to train a “human-like language” model? There doesn’t seem to be any fundamental “theoretical” way to know. But in practice ChatGPT was successfully trained on a few hundred billion words of text.

它看到的文本有些被多次输入，有些只输入一次。但不知何故，它从看到的文本中“得到了所需”。但考虑到要学习的文本量，需要多大的网络才能“学好”呢？同样，我们还没有基本的理论方法来说明。最终——正如我们将在下面进一步讨论的那样——人类语言以及人们通常用它所说的话，可能具有一定的“总算法内容”。但接下来的问题是，神经网络在实现基于该算法内容的模型时效率如何。我们同样不知道——尽管 ChatGPT 的成功表明它相当高效。

Some of the text it was fed several times, some of it only once. But somehow it “got what it needed” from the text it saw. But given this volume of text to learn from, how large a network should it require to “learn it well”? Again, we don’t yet have a fundamental theoretical way to say. Ultimately—as we’ll discuss further below—there’s presumably a certain “total algorithmic content” to human language and what humans typically say with it. But the next question is how efficient a neural net will be at implementing a model based on that algorithmic content. And again we don’t know—although the success of ChatGPT suggests it’s reasonably efficient.

最后，我们可以注意到，ChatGPT 使用几千亿个权重来完成其工作——这个数量与它被给予的训练数据的总词数（或 token 数）相当。在某种程度上，也许令人惊讶（尽管在 ChatGPT 的较小类似物中也经验性地观察到），“网络大小”与“训练数据大小”如此接近是有效的。毕竟，这当然不是说“在 ChatGPT 内部”所有来自网络和书籍等的文本都被“直接存储”了。因为 ChatGPT 内部实际上是一堆数字——精度略低于 10 位——这些数字是所有这些文本的聚合结构的某种分布式编码。

And in the end we can just note that ChatGPT does what it does using a couple hundred billion weights—comparable in number to the total number of words (or tokens) of training data it’s been given. In some ways it’s perhaps surprising (though empirically observed also in smaller analogs of ChatGPT) that the “size of the network” that seems to work well is so comparable to the “size of the training data”. After all, it’s certainly not that somehow “inside ChatGPT” all that text from the web and books and so on is “directly stored”. Because what’s actually inside ChatGPT are a bunch of numbers—with a bit less than 10 digits of precision—that are some kind of distributed encoding of the aggregate structure of all that text.

换句话说，我们可能会问人类语言以及通常用它所说的话的“有效信息内容”是什么。有原始的语言示例语料库。还有 ChatGPT 神经网络中的表示。这种表示很可能远非“算法最小”表示（我们将在下面讨论）。但它是一种神经网络易于使用的表示。在这种表示中，最终对训练数据的“压缩”相当少；平均而言，基本上只需要略少于一个神经网络权重来承载一个训练数据词的“信息内容”。

Put another way, we might ask what the “effective information content” is of human language and what’s typically said with it. There’s the raw corpus of examples of language. And then there’s the representation in the neural net of ChatGPT. That representation is very likely far from the “algorithmically minimal” representation (as we’ll discuss below). But it’s a representation that’s readily usable by the neural net. And in this representation it seems there’s in the end rather little “compression” of the training data; it seems on average to basically take only a bit less than one neural net weight to carry the “information content” of a word of training data.

当我们运行 ChatGPT 生成文本时，我们基本上必须使用每个权重一次。因此，如果有 n 个权重，我们就需要大约 n 个计算步骤——尽管在实践中，许多步骤通常可以在 GPU 中并行完成。但如果我们需要大约 n 个词的训练数据来设置这些权重，那么根据我们上面所说的，我们可以得出结论，我们需要大约 n² 个计算步骤来训练网络——这就是为什么在当前方法下，最终需要谈论数十亿美元的训练工作。

When we run ChatGPT to generate text, we’re basically having to use each weight once. So if there are _n_ weights, we’ve got of order _n_ computational steps to do—though in practice many of them can typically be done in parallel in GPUs. But if we need about _n_ words of training data to set up those weights, then from what we’ve said above we can conclude that we’ll need about _n_ 2 computational steps to do the training of the network—which is why, with current methods, one ends up needing to talk about billion-dollar training efforts.

超越基础训练 Beyond Basic Training

训练 ChatGPT 的大部分工作都花在“展示”来自网络、书籍等的大量现有文本上。但事实证明，还有另一个——显然相当重要的——部分。

The majority of the effort in training ChatGPT is spent “showing it” large amounts of existing text from the web, books, etc. But it turns out there’s another—apparently rather important—part too.

一旦它完成了对原始文本语料库的“原始训练”，ChatGPT 内部的神经网络就准备好开始生成自己的文本，根据提示继续生成等。但是，尽管这些结果通常看起来合理，但它们往往——特别是对于较长的文本——以相当非人类的方式“偏离”。这不是通过传统文本统计就能轻易检测到的，但实际阅读文本的人类很容易注意到。

As soon as it’s finished its “raw training” from the original corpus of text it’s been shown, the neural net inside ChatGPT is ready to start generating its own text, continuing from prompts, etc. But while the results from this may often seem reasonable, they tend—particularly for longer pieces of text—to “wander off” in often rather non-human-like ways. It’s not something one can readily detect, say, by doing traditional statistics on the text. But it’s something that actual humans reading the text easily notice.

构建 ChatGPT 的一个关键思想是在“被动阅读”网络等内容之后增加一个步骤：让真实的人类主动与 ChatGPT 互动，查看其生成的内容，并有效地给予它关于“如何成为一个好的聊天机器人”的反馈。但是神经网络如何利用这些反馈呢？第一步是让人类对神经网络的结果进行评分。然后构建另一个神经网络模型来尝试预测这些评分。现在，这个预测模型可以像损失函数一样在原始网络上运行，从而有效地让网络根据所给出的人类反馈进行“调优”。实际上，这对系统生成“类人”输出的成功有显著影响。

And a key idea in the construction of ChatGPT was to have another step after “passively reading” things like the web: to have actual humans actively interact with ChatGPT, see what it produces, and in effect give it feedback on “how to be a good chatbot”. But how can the neural net use that feedback? The first step is just to have humans rate results from the neural net. But then another neural net model is built that attempts to predict those ratings. But now this prediction model can be run—essentially like a loss function—on the original network, in effect allowing that network to be “tuned up” by the human feedback that’s been given. And the results in practice seem to have a big effect on the success of the system in producing “human-like” output.

总的来说，有趣的是，“最初训练”的网络似乎只需要很少的“引导”就能有效地朝着特定方向前进。人们可能会认为，要让网络表现得好像“学到了新东西”，就必须运行训练算法，调整权重等。

In general, it’s interesting how little “poking” the “originally trained” network seems to need to get it to usefully go in particular directions. One might have thought that to have the network behave as if it’s “learned something new” one would have to go in and run a training algorithm, adjusting weights, and so on.

但事实并非如此。相反，基本上只需在提示中一次性告诉 ChatGPT 某些内容，它就能在生成文本时成功利用你告诉它的信息。我认为，这一事实再次为理解 ChatGPT“真正在做什么”以及它如何与人类语言和思维结构相关联提供了重要线索。

But that’s not the case. Instead, it seems to be sufficient to basically tell ChatGPT something one time—as part of the prompt you give—and then it can successfully make use of what you told it when it generates text. And once again, the fact that this works is, I think, an important clue in understanding what ChatGPT is “really doing” and how it relates to the structure of human language and thinking.

这确实有些类人之处：至少一旦它完成了所有预训练，你可以只告诉它一次，它就能“记住”——至少“足够长”以利用它生成一段文本。那么在这种情况下发生了什么？可能是“你可能告诉它的所有内容都已经存在于某处”——你只是引导它到正确的位置。但这似乎不太可能。相反，更可能的是，元素确实已经存在，但具体细节是由类似“这些元素之间的轨迹”定义的，而当你告诉它某些内容时，你正是在引入这个轨迹。

There’s certainly something rather human-like about it: that at least once it’s had all that pre-training you can tell it something just once and it can “remember it”—at least “long enough” to generate a piece of text using it. So what’s going on in a case like this? It could be that “everything you might tell it is already in there somewhere”—and you’re just leading it to the right spot. But that doesn’t seem plausible. Instead, what seems more likely is that, yes, the elements are already in there, but the specifics are defined by something like a “trajectory between those elements” and that’s what you’re introducing when you tell it something.

事实上，与人类非常相似，如果你告诉它一些完全不符合其已知框架的离奇和意外的事情，它似乎无法成功“整合”这些信息。只有当它基本上以相当简单的方式依赖于已有框架时，它才能“整合”。

And indeed, much like for humans, if you tell it something bizarre and unexpected that completely doesn’t fit into the framework it knows, it doesn’t seem like it’ll successfully be able to “integrate” this. It can “integrate” it only if it’s basically riding in a fairly simple way on top of the framework it already has.

还需要再次指出的是，神经网络能够“掌握”的内容不可避免地存在“算法限制”。告诉它像“这个对应那个”之类的“浅层”规则，神经网络很可能能够很好地表示和复现这些规则——事实上，它从语言中“已经知道”的内容会立即给它一个遵循的模式。但试图给它一个涉及许多潜在计算不可约步骤的实际“深层”计算的规则，它就行不通了。（请记住，每一步它都只是在网络中“前馈数据”，从不循环，除非通过生成新标记。）

It’s also worth pointing out again that there are inevitably “algorithmic limits” to what the neural net can “pick up”. Tell it “shallow” rules of the form “this goes to that”, etc., and the neural net will most likely be able to represent and reproduce these just fine—and indeed what it “already knows” from language will give it an immediate pattern to follow. But try to give it rules for an actual “deep” computation that involves many potentially computationally irreducible steps and it just won’t work. (Remember that at each step it’s always just “feeding data forward” in its network, never looping except by virtue of generating new tokens.)

当然，网络可以学习特定“不可约”计算的答案。但只要存在组合数量的可能性，这种“查表式”方法就行不通。因此，是的，就像人类一样，神经网络是时候“伸出援手”并使用实际的计算工具了。（而且，是的，Wolfram|Alpha 和 Wolfram Language 特别适合，因为它们被构建为“谈论世界上的事物”，就像语言模型神经网络一样。）

Of course, the network can learn the answer to specific “irreducible” computations. But as soon as there are combinatorial numbers of possibilities, no such “table-lookup-style” approach will work. And so, yes, just like humans, it’s time then for neural nets to “reach out” and use actual computational tools. (And, yes, Wolfram|Alpha and Wolfram Language are uniquely suitable, because they’ve been built to “talk about things in the world”, just like the language-model neural nets.)

是什么真正让 ChatGPT 发挥作用？ What Really Lets ChatGPT Work?

人类语言——以及生成语言所涉及的思维过程——似乎一直代表着复杂性的顶峰。事实上，人类大脑——拥有“仅仅”约 1000 亿个神经元（以及可能 100 万亿个连接）的网络——能够负责语言，这似乎相当引人注目。也许有人会想象，大脑除了神经元网络之外还有更多东西——比如某种尚未发现的物理学新层面。但现在，借助 ChatGPT，我们获得了一条重要的新信息：我们知道，一个纯粹的、连接数量与大脑神经元数量相当的人工神经网络，能够出人意料地出色地生成人类语言。

Human language—and the processes of thinking involved in generating it—have always seemed to represent a kind of pinnacle of complexity. And indeed it’s seemed somewhat remarkable that human brains—with their network of a “mere” 100 billion or so neurons (and maybe 100 trillion connections) could be responsible for it. Perhaps, one might have imagined, there’s something more to brains than their networks of neurons—like some new layer of undiscovered physics. But now with ChatGPT we’ve got an important new piece of information: we know that a pure, artificial neural network with about as many connections as brains have neurons is capable of doing a surprisingly good job of generating human language.

是的，这仍然是一个庞大而复杂的系统——其神经网络权重的数量大约相当于当前世界上可用的文本单词数量。但在某种程度上，仍然难以相信语言及其所能谈论的一切丰富性都可以封装在这样一个有限的系统中。部分原因无疑反映了一种普遍现象（最初在规则 30 的例子中变得明显），即计算过程实际上可以极大地放大系统的表观复杂性，即使其底层规则很简单。但实际上，正如我们上面讨论的，ChatGPT 中使用的那种神经网络往往被专门构建以限制这种现象及其相关的计算不可约性的影响，以便使其训练更容易进行。

And, yes, that’s still a big and complicated system—with about as many neural net weights as there are words of text currently available out there in the world. But at some level it still seems difficult to believe that all the richness of language and the things it can talk about can be encapsulated in such a finite system. Part of what’s going on is no doubt a reflection of the ubiquitous phenomenon (that first became evident in the example of rule 30) that computational processes can in effect greatly amplify the apparent complexity of systems even when their underlying rules are simple. But, actually, as we discussed above, neural nets of the kind used in ChatGPT tend to be specifically constructed to restrict the effect of this phenomenon—and the computational irreducibility associated with it—in the interest of making their training more accessible.

那么，像 ChatGPT 这样的东西是如何在语言方面取得如此成就的呢？我认为，基本答案是，语言在根本层面上比它看起来要简单。这意味着 ChatGPT——即使其神经网络结构最终很简单——也能成功地“捕捉人类语言及其背后思维的本质”。而且，在其训练过程中，ChatGPT 以某种方式“隐式地发现了”语言（和思维）中使这成为可能的任何规律。

So how is it, then, that something like ChatGPT can get as far as it does with language? The basic answer, I think, is that language is at a fundamental level somehow simpler than it seems. And this means that ChatGPT—even with its ultimately straightforward neural net structure—is successfully able to “capture the essence” of human language and the thinking behind it. And moreover, in its training, ChatGPT has somehow “implicitly discovered” whatever regularities in language (and thinking) make this possible.

我认为，ChatGPT 的成功为我们提供了一项基础且重要的科学证据：它表明，我们可以期待发现重要的新“语言定律”——实际上也是“思维定律”。在作为神经网络构建的 ChatGPT 中，这些定律充其量是隐式的。但如果我们能以某种方式使这些定律显式化，就有可能以更直接、更高效且更透明的方式完成 ChatGPT 所做的事情。

The success of ChatGPT is, I think, giving us evidence of a fundamental and important piece of science: it’s suggesting that we can expect there to be major new “laws of language”—and effectively “laws of thought”—out there to discover. In ChatGPT—built as it is as a neural net—those laws are at best implicit. But if we could somehow make the laws explicit, there’s the potential to do the kinds of things ChatGPT does in vastly more direct, efficient—and transparent—ways.

但是，好吧，那么这些定律可能是什么样的呢？最终，它们必须为我们提供某种关于语言——以及我们用语言所说的事物——是如何组合在一起的处方。稍后我们将讨论“查看 ChatGPT 内部”如何可能为我们提供一些关于此的提示，以及我们从构建计算语言中所了解的内容如何指明前进的道路。但首先，让我们讨论两个长期已知的、相当于“语言定律”的例子——以及它们如何与 ChatGPT 的操作相关联。

But, OK, so what might these laws be like? Ultimately they must give us some kind of prescription for how language—and the things we say with it—are put together. Later we’ll discuss how “looking inside ChatGPT” may be able to give us some hints about this, and how what we know from building computational language suggests a path forward. But first let’s discuss two long-known examples of what amount to “laws of language”—and how they relate to the operation of ChatGPT.

第一个是语言的句法。语言不仅仅是单词的随机堆砌。相反，有（相当）明确的语法规则规定不同种类的单词如何组合在一起：例如，在英语中，名词前面可以有形容词，后面可以有动词，但通常两个名词不能紧挨着。这种语法结构可以（至少近似地）由一组规则来捕获，这些规则定义了如何组合出所谓的“解析树”：

The first is the syntax of language. Language is not just a random jumble of words. Instead, there are (fairly) definite grammatical rules for how words of different kinds can be put together: in English, for example, nouns can be preceded by adjectives and followed by verbs, but typically two nouns can’t be right next to each other. Such grammatical structure can (at least approximately) be captured by a set of rules that define how what amount to “parse trees” can be put together:

ChatGPT 对这些规则没有任何显式的“知识”。但不知何故，在其训练过程中，它隐式地“发现”了它们——然后似乎很擅长遵循它们。那么这是如何工作的呢？在“宏观”层面上，这并不清楚。但为了获得一些见解，也许值得看一个更简单的例子。

ChatGPT doesn’t have any explicit “knowledge” of such rules. But somehow in its training it implicitly “discovers” them—and then seems to be good at following them. So how does this work? At a “big picture” level it’s not clear. But to get some insight it’s perhaps instructive to look at a much simpler example.

考虑一种由 '(' 和 ')' 序列组成的“语言”，其语法规定括号必须始终平衡，如解析树所示：

Consider a “language” formed from sequences of (’s and )’s, with a grammar that specifies that parentheses should always be balanced, as represented by a parse tree like:

我们能否训练一个神经网络来生成“语法正确”的括号序列？在神经网络中有多种处理序列的方法，但让我们像 ChatGPT 一样使用 Transformer 网络。给定一个简单的 Transformer 网络，我们可以开始向其提供语法正确的括号序列作为训练示例。一个微妙之处（实际上也出现在 ChatGPT 生成人类语言的过程中）是，除了我们的“内容标记”（这里是 '(' 和 ')'）之外，我们还必须包含一个“结束”标记，该标记的生成表示输出不应再继续（即对于 ChatGPT，表示已经到达“故事的结尾”）。

Can we train a neural net to produce “grammatically correct” parenthesis sequences? There are various ways to handle sequences in neural nets, but let’s use transformer nets, as ChatGPT does. And given a simple transformer net, we can start feeding it grammatically correct parenthesis sequences as training examples. A subtlety (which actually also appears in ChatGPT’s generation of human language) is that in addition to our “content tokens” (here “(” and “)”) we have to include an “End” token, that’s generated to indicate that the output shouldn’t continue any further (i.e. for ChatGPT, that one’s reached the “end of the story”).

如果我们设置一个 Transformer 网络，只有一个注意力块，8 个头，特征向量长度为 128（ChatGPT 也使用长度为 128 的特征向量，但有 96 个注意力块，每个有 96 个头），那么似乎无法让它学到很多关于括号语言的知识。但是使用 2 个注意力块，学习过程似乎收敛了——至少在提供了大约 1000 万个示例之后（并且，正如 Transformer 网络常见的那样，展示更多示例似乎只会降低其性能）。

If we set up a transformer net with just one attention block with 8 heads and feature vectors of length 128 (ChatGPT also uses feature vectors of length 128, but has 96 attention blocks, each with 96 heads) then it doesn’t seem possible to get it to learn much about parenthesis language. But with 2 attention blocks, the learning process seems to converge—at least after 10 million or so examples have been given (and, as is common with transformer nets, showing yet more examples just seems to degrade its performance).

因此，使用这个网络，我们可以做类似于 ChatGPT 所做的事情，并询问下一个标记应该是什么的概率——在括号序列中：

So with this network, we can do the analog of what ChatGPT does, and ask for probabilities for what the next token should be—in a parenthesis sequence:

在第一种情况下，网络“相当确定”序列不能在这里结束——这很好，因为如果结束了，括号将保持不平衡。然而，在第二种情况下，它“正确地识别出”序列可以在这里结束，尽管它也“指出”可以“重新开始”，放下一个 '('，大概后面跟着一个 ')'。但是，哎呀，即使经过大约 40 万个辛苦训练的权重，它说下一个标记有 15% 的概率是 ')'——这是不对的，因为这必然会导致括号不平衡。

And in the first case, the network is “pretty sure” that the sequence can’t end here—which is good, because if it did, the parentheses would be left unbalanced. In the second case, however, it “correctly recognizes” that the sequence can end here, though it also “points out” that it’s possible to “start again”, putting down a “(”, presumably with a “)” to follow. But, oops, even with its 400,000 or so laboriously trained weights, it says there’s a 15% probability to have “)” as the next token—which isn’t right, because that would necessarily lead to an unbalanced parenthesis.

如果我们要求网络为逐渐变长的 '(' 序列提供最高概率的补全，我们会得到以下结果：

Here’s what we get if we ask the network for the highest-probability completions for progressively longer sequences of (’s:

是的，直到一定长度，网络都表现得很好。但随后它开始失败。在像这样“精确”的情况下，神经网络（或一般的机器学习）出现这种情况是相当典型的。人类“一眼就能解决”的情况，神经网络也能解决。但需要做“更多算法性”事情的情况（例如，显式地计算括号以查看它们是否闭合），神经网络往往在某种程度上“计算上太浅”而无法可靠地完成。（顺便说一句，即使是完整的当前 ChatGPT 也很难正确匹配长序列中的括号。）

And, yes, up to a certain length the network does just fine. But then it starts failing. It’s a pretty typical kind of thing to see in a “precise” situation like this with a neural net (or with machine learning in general). Cases that a human “can solve in a glance” the neural net can solve too. But cases that require doing something “more algorithmic” (e.g. explicitly counting parentheses to see if they’re closed) the neural net tends to somehow be “too computationally shallow” to reliably do. (By the way, even the full current ChatGPT has a hard time correctly matching parentheses in long sequences.)

那么，这对于像 ChatGPT 和英语这样的语言的句法意味着什么呢？括号语言是“简朴的”——并且更像是一个“算法故事”。但在英语中，基于局部单词选择和其他提示来“猜测”语法上合适的内容要现实得多。而且，是的，神经网络在这方面要好得多——尽管它可能会错过一些“形式上正确”的情况，而人类也可能会错过。但主要的一点是，语言存在整体句法结构——以及由此带来的所有规律性——在某种意义上限制了神经网络需要学习的“量”。一个关键的“类似自然科学”的观察是，像 ChatGPT 中的神经网络那样的 Transformer 架构似乎能够成功地学习那种嵌套树状句法结构，这种结构似乎（至少在某种近似上）存在于所有人类语言中。

So what does this mean for things like ChatGPT and the syntax of a language like English? The parenthesis language is “austere”—and much more of an “algorithmic story”. But in English it’s much more realistic to be able to “guess” what’s grammatically going to fit on the basis of local choices of words and other hints. And, yes, the neural net is much better at this—even though perhaps it might miss some “formally correct” case that, well, humans might miss as well. But the main point is that the fact that there’s an overall syntactic structure to the language—with all the regularity that implies—in a sense limits “how much” the neural net has to learn. And a key “natural-science-like” observation is that the transformer architecture of neural nets like the one in ChatGPT seems to successfully be able to learn the kind of nested-tree-like syntactic structure that seems to exist (at least in some approximation) in all human languages.

句法为语言提供了一种约束。但显然还有更多。像“好奇的电子吃蓝色的理论当鱼”这样的句子在语法上是正确的，但通常不会有人这么说，如果 ChatGPT 生成了它，也不会被认为是成功的——因为，嗯，按照其中单词的正常含义，它基本上是无意义的。

Syntax provides one kind of constraint on language. But there are clearly more. A sentence like “Inquisitive electrons eat blue theories for fish” is grammatically correct but isn’t something one would normally expect to say, and wouldn’t be considered a success if ChatGPT generated it—because, well, with the normal meanings for the words in it, it’s basically meaningless.

但是否有一种通用的方法来判断一个句子是否有意义？对此没有传统的整体理论。但可以认为，ChatGPT 在接受了来自网络等的数十亿个（大概有意义的）句子的训练后，已经隐式地“发展出了一个理论”。

But is there a general way to tell if a sentence is meaningful? There’s no traditional overall theory for that. But it’s something that one can think of ChatGPT as having implicitly “developed a theory for” after being trained with billions of (presumably meaningful) sentences from the web, etc.

这个理论可能是什么样的呢？嗯，有一个小角落基本上已经为人所知两千年了，那就是逻辑。当然，在亚里士多德发现的三段论形式中，逻辑基本上是一种说法，即遵循某些模式的句子是合理的，而其他则不然。因此，例如，说“所有 X 都是 Y。这不是 Y，所以它不是 X”是合理的（如“所有的鱼都是蓝色的。这不是蓝色的，所以它不是鱼。”）。正如人们可以有点异想天开地想象亚里士多德通过（“机器学习风格地”）浏览大量修辞例子发现了三段论逻辑一样，人们也可以想象，在 ChatGPT 的训练过程中，它能够通过查看网络上的大量文本等来“发现三段论逻辑”。（而且，是的，虽然因此可以期望 ChatGPT 生成的文本包含基于三段论逻辑等“正确推理”的内容，但当涉及到更复杂的形式逻辑时，情况就完全不同了——我认为可以预期它会因为与括号匹配失败相同的原因而在此失败。）

What might this theory be like? Well, there’s one tiny corner that’s basically been known for two millennia, and that’s logic. And certainly in the syllogistic form in which Aristotle discovered it, logic is basically a way of saying that sentences that follow certain patterns are reasonable, while others are not. Thus, for example, it’s reasonable to say “All X are Y. This is not Y, so it’s not an X” (as in “All fishes are blue. This is not blue, so it’s not a fish.”). And just as one can somewhat whimsically imagine that Aristotle discovered syllogistic logic by going (“machine-learning-style”) through lots of examples of rhetoric, so too one can imagine that in the training of ChatGPT it will have been able to “discover syllogistic logic” by looking at lots of text on the web, etc. (And, yes, while one can therefore expect ChatGPT to produce text that contains “correct inferences” based on things like syllogistic logic, it’s a quite different story when it comes to more sophisticated formal logic—and I think one can expect it to fail here for the same kind of reasons it fails in parenthesis matching.)

但除了逻辑这个狭窄的例子之外，关于如何系统地构建（或识别）甚至看似有意义的文本，还能说些什么呢？是的，有像 Mad Libs 这样使用非常具体的“短语模板”的东西。但不知何故，ChatGPT 隐式地有一种更通用的方法。也许除了“当你拥有 1750 亿个神经网络权重时，它就会以某种方式发生”之外，没有什么可说的了。但我强烈怀疑有一个更简单、更有力的故事。

But beyond the narrow example of logic, what can be said about how to systematically construct (or recognize) even plausibly meaningful text? Yes, there are things like Mad Libs that use very specific “phrasal templates”. But somehow ChatGPT implicitly has a much more general way to do it. And perhaps there’s nothing to be said about how it can be done beyond “somehow it happens when you have 175 billion neural net weights”. But I strongly suspect that there’s a much simpler and stronger story.

意义空间与语义运动定律 Meaning Space and Semantic Laws of Motion

我们之前讨论过，在 ChatGPT 内部，任何文本实际上都由一个数字数组表示，我们可以将其视为某种“语言特征空间”中点的坐标。因此，当 ChatGPT 延续一段文本时，这对应于在语言特征空间中描绘出一条轨迹。但现在我们可以问，是什么使得这条轨迹对应于我们认为有意义的文本？是否可能存在某种“语义运动定律”，它定义——或至少约束——语言特征空间中的点如何在保持“有意义”的同时移动？

We discussed above that inside ChatGPT any piece of text is effectively represented by an array of numbers that we can think of as coordinates of a point in some kind of “linguistic feature space”. So when ChatGPT continues a piece of text this corresponds to tracing out a trajectory in linguistic feature space. But now we can ask what makes this trajectory correspond to text we consider meaningful. And might there perhaps be some kind of “semantic laws of motion” that define—or at least constrain—how points in linguistic feature space can move around while preserving “meaningfulness”?

那么，这个语言特征空间是什么样的？下面是一个例子，展示了如果我们把这个特征空间投影到二维，单个单词（这里是常见名词）可能会如何排列：

So what is this linguistic feature space like? Here’s an example of how single words (here, common nouns) might get laid out if we project such a feature space down to 2D:

我们之前看到了另一个基于代表植物和动物的单词的例子。但这两个例子的关键点是，“语义相似的单词”被放置在附近。

We saw another example above based on words representing plants and animals. But the point in both cases is that “semantically similar words” are placed nearby.

另一个例子，这里是不同词性的单词如何排列：

As another example, here’s how words corresponding to different parts of speech get laid out:

当然，一个给定的单词通常不仅仅有“一个意思”（或者不一定只对应一种词性）。通过观察包含某个单词的句子在特征空间中的布局，我们通常可以“区分”不同的含义——就像这里“crane”（鸟还是机器？）的例子：

Of course, a given word doesn’t in general just have “one meaning” (or necessarily correspond to just one part of speech). And by looking at how sentences containing a word lay out in feature space, one can often “tease apart” different meanings—as in the example here for the word “crane” (bird or machine?):

好的，所以至少可以合理地认为，我们可以把这个特征空间看作是“意义相近的单词”在空间中靠得很近。但我们能在这个空间中识别出什么样的额外结构呢？例如，是否存在某种“平行移动”的概念，以反映空间的“平坦性”？处理这个问题的一种方法是看类比：

OK, so it’s at least plausible that we can think of this feature space as placing “words nearby in meaning” close in this space. But what kind of additional structure can we identify in this space? Is there for example some kind of notion of “parallel transport” that would reflect “flatness” in the space? One way to get a handle on that is to look at analogies:

是的，即使当我们投影到二维时，通常也至少存在“平坦性的暗示”，尽管这当然不是普遍可见的。

And, yes, even when we project down to 2D, there’s often at least a “hint of flatness”, though it’s certainly not universally seen.

那么轨迹呢？我们可以观察 ChatGPT 的提示在特征空间中遵循的轨迹——然后看看 ChatGPT 如何延续它：

So what about trajectories? We can look at the trajectory that a prompt for ChatGPT follows in feature space—and then we can see how ChatGPT continues that:

这里当然没有“几何上明显的”运动定律。这并不令人惊讶；我们完全预料到这会是一个相当复杂的故事。例如，即使存在一个“语义运动定律”有待发现，它最自然地用哪种嵌入（或者实际上，用哪种“变量”）来表述也远非显而易见。

There’s certainly no “geometrically obvious” law of motion here. And that’s not at all surprising; we fully expect this to be a considerably more complicated story. And, for example, it’s far from obvious that even if there is a “semantic law of motion” to be found, what kind of embedding (or, in effect, what “variables”) it’ll most naturally be stated in.

在上图中，我们展示了“轨迹”中的几个步骤——在每个步骤中，我们选择 ChatGPT 认为最可能的单词（“零温度”情况）。但我们也可以问，在给定点上，哪些单词可以“接下来出现”以及以什么概率出现：

In the picture above, we’re showing several steps in the “trajectory”—where at each step we’re picking the word that ChatGPT considers the most probable (the “zero temperature” case). But we can also ask what words can “come next” with what probabilities at a given point:

在这种情况下我们看到的是，有一个高概率单词的“扇形”，似乎在特征空间中朝着或多或少确定的方向延伸。如果我们继续前进会发生什么？以下是当我们“沿着轨迹移动”时出现的连续“扇形”：

And what we see in this case is that there’s a “fan” of high-probability words that seems to go in a more or less definite direction in feature space. What happens if we go further? Here are the successive “fans” that appear as we “move along” the trajectory:

这是一个 3D 表示，总共走了 40 步：

Here’s a 3D representation, going for a total of 40 steps:

是的，这看起来一团糟——并且丝毫不会鼓励人们认为可以通过经验研究“ChatGPT 内部在做什么”来识别出“数学物理式”的“语义运动定律”。但也许我们只是看了“错误的变量”（或错误的坐标系），如果我们看了正确的变量，就会立即看到 ChatGPT 在做一些“数学物理简单”的事情，比如沿着测地线运动。但截至目前，我们还没有准备好从它的“内部行为”中“经验性地解码”出 ChatGPT 关于人类语言是如何“组织起来”的“发现”。

And, yes, this seems like a mess—and doesn’t do anything to particularly encourage the idea that one can expect to identify “mathematical-physics-like” “semantic laws of motion” by empirically studying “what ChatGPT is doing inside”. But perhaps we’re just looking at the “wrong variables” (or wrong coordinate system) and if only we looked at the right one, we’d immediately see that ChatGPT is doing something “mathematical-physics-simple” like following geodesics. But as of now, we’re not ready to “empirically decode” from its “internal behavior” what ChatGPT has “discovered” about how human language is “put together”.

语义语法与计算语言的力量 Semantic Grammar and the Power of Computational Language

要产生“有意义的人类语言”需要什么？过去，我们可能认为这非人脑莫属。但现在我们知道，ChatGPT 的神经网络已经能相当出色地做到这一点。不过，也许这就是极限了，不会有更简单或更易于人类理解的方法能奏效。但我强烈怀疑，ChatGPT 的成功隐含地揭示了一个重要的“科学”事实：有意义的人类语言实际上比我们以往所知的更具结构和简洁性——最终甚至可能存在相当简单的规则来描述这种语言是如何组合起来的。

What does it take to produce “meaningful human language”? In the past, we might have assumed it could be nothing short of a human brain. But now we know it can be done quite respectably by the neural net of ChatGPT. Still, maybe that’s as far as we can go, and there’ll be nothing simpler—or more human understandable—that will work. But my strong suspicion is that the success of ChatGPT implicitly reveals an important “scientific” fact: that there’s actually a lot more structure and simplicity to meaningful human language than we ever knew—and that in the end there may be even fairly simple rules that describe how such language can be put together.

如上所述，句法语法给出了如何将对应不同词性（如名词、动词）的词语组合成人类语言的规则。但要处理意义，我们需要更进一步。而做到这一点的一种方式是不仅考虑语言的句法语法，还要考虑语义语法。

As we mentioned above, syntactic grammar gives rules for how words corresponding to things like different parts of speech can be put together in human language. But to deal with meaning, we need to go further. And one version of how to do this is to think about not just a syntactic grammar for language, but also a semantic one.

就句法而言，我们识别名词和动词等。但就语义而言，我们需要“更细的粒度”。例如，我们可能识别“移动”的概念，以及“保持身份独立于位置”的“物体”概念。这些“语义概念”各有无数具体实例。但为了我们的语义语法，我们只需一些通用规则，基本上说明“物体”可以“移动”。关于这一切如何运作有很多可说的（其中一些我以前说过）。但我在此只满足于指出一些潜在的前进方向。

For purposes of syntax, we identify things like nouns and verbs. But for purposes of semantics, we need “finer gradations”. So, for example, we might identify the concept of “moving”, and the concept of an “object” that “maintains its identity independent of location”. There are endless specific examples of each of these “semantic concepts”. But for the purposes of our semantic grammar, we’ll just have some general kind of rule that basically says that “objects” can “move”. There’s a lot to say about how all this might work (some of which I’ve said before). But I’ll content myself here with just a few remarks that indicate some of the potential path forward.

值得一提的是，即使一个句子在语义语法上完全正确，也不意味着它在实践中已被实现（甚至可能实现）。“大象去了月球”无疑会“通过”我们的语义语法，但它肯定没有在我们的现实世界中实现（至少目前没有）——尽管在虚构世界中这完全合理。

It’s worth mentioning that even if a sentence is perfectly OK according to the semantic grammar, that doesn’t mean it’s been realized (or even could be realized) in practice. “The elephant traveled to the Moon” would doubtless “pass” our semantic grammar, but it certainly hasn’t been realized (at least yet) in our actual world—though it’s absolutely fair game for a fictional world.

当我们开始谈论“语义语法”时，我们很快会问“它下面是什么？”它假设了什么样的“世界模型”？句法语法实际上只是关于从词语构建语言。但语义语法必然涉及某种“世界模型”——它作为一个“骨架”，由实际词语构成的语言可以层叠其上。

When we start talking about “semantic grammar” we’re soon led to ask “What’s underneath it?” What “model of the world” is it assuming? A syntactic grammar is really just about the construction of language from words. But a semantic grammar necessarily engages with some kind of “model of the world”—something that serves as a “skeleton” on top of which language made from actual words can be layered.

直到最近，我们可能还认为（人类）语言将是描述我们“世界模型”的唯一通用方式。几个世纪前，开始出现基于数学的特定事物的形式化。但现在有一种更通用的形式化方法：计算语言。

Until recent times, we might have imagined that (human) language would be the only general way to describe our “model of the world”. Already a few centuries ago there started to be formalizations of specific kinds of things, based particularly on mathematics. But now there’s a much more general approach to formalization: computational language.

是的，这是我四十多年来的大项目（现已体现在 Wolfram 语言中）：开发一种精确的符号表示，能够尽可能广泛地谈论世界上的事物以及我们关心的抽象事物。例如，我们有城市、分子、图像和神经网络的符号表示，并且我们有关于如何计算这些事物的内置知识。

And, yes, that’s been my big project over the course of more than four decades (as now embodied in the Wolfram Language): to develop a precise symbolic representation that can talk as broadly as possible about things in the world, as well as abstract things that we care about. And so, for example, we have symbolic representations for cities and molecules and images and neural networks, and we have built-in knowledge about how to compute about those things.

经过几十年的工作，我们已经以这种方式覆盖了许多领域。但在过去，我们并没有特别处理“日常话语”。在“我买了两磅苹果”中，我们可以轻松表示（并进行营养和其他计算）“两磅苹果”。但我们（还）没有“我买了”的符号表示。

And, after decades of work, we’ve covered a lot of areas in this way. But in the past, we haven’t particularly dealt with “everyday discourse”. In “I bought two pounds of apples” we can readily represent (and do nutrition and other computations on) the “two pounds of apples”. But we don’t (quite yet) have a symbolic representation for “I bought”.

这一切都与语义语法的思想相关——目标是拥有一个通用的概念“符号构建工具包”，为我们提供什么能与什么组合的规则，从而提供我们可能转化为人类语言的“流程”。

It’s all connected to the idea of semantic grammar—and the goal of having a generic symbolic “construction kit” for concepts, that would give us rules for what could fit together with what, and thus for the “flow” of what we might turn into human language.

但假设我们有了这种“符号话语语言”。我们会用它做什么？我们可以开始做诸如生成“局部有意义的文本”之类的事情。但最终我们可能想要更“全局有意义”的结果——这意味着“计算”更多关于世界上实际存在或可能发生的事情（或者在某些一致的虚构世界中）。

But let’s say we had this “symbolic discourse language”. What would we do with it? We could start off doing things like generating “locally meaningful text”. But ultimately we’re likely to want more “globally meaningful” results—which means “computing” more about what can actually exist or happen in the world (or perhaps in some consistent fictional world).

现在在 Wolfram 语言中，我们有大量关于许多事物的内置计算知识。但对于一个完整的符号话语语言，我们必须构建关于世界中一般事物的额外“演算”：如果一个物体从 A 移动到 B，再从 B 移动到 C，那么它就从 A 移动到了 C，等等。

Right now in Wolfram Language we have a huge amount of built-in computational knowledge about lots of kinds of things. But for a complete symbolic discourse language we’d have to build in additional “calculi” about general things in the world: if an object moves from A to B and from B to C, then it’s moved from A to C, etc.

有了符号话语语言，我们可以用它来做出“独立陈述”。但我们也可以用它来提问关于世界的问题，“Wolfram|Alpha 风格”。或者我们可以用它来陈述我们“想要实现”的事情，可能通过某种外部驱动机制。或者我们可以用它来做出断言——可能是关于现实世界，也可能是关于我们考虑的某个特定世界，虚构或其他。

Given a symbolic discourse language we might use it to make “standalone statements”. But we can also use it to ask questions about the world, “Wolfram|Alpha style”. Or we can use it to state things that we “want to make so”, presumably with some external actuation mechanism. Or we can use it to make assertions—perhaps about the actual world, or perhaps about some specific world we’re considering, fictional or otherwise.

人类语言本质上是不精确的，尤其是因为它没有“绑定”到特定的计算实现，其意义基本上由用户之间的“社会契约”定义。但计算语言本质上具有某种基本精确性——因为它指定的内容最终总是可以“在计算机上明确执行”。人类语言通常可以容忍某种模糊性。（当我们说“行星”时，它是否包括系外行星等？）但在计算语言中，我们必须精确和清晰地说明我们做出的所有区分。

Human language is fundamentally imprecise, not least because it isn’t “tethered” to a specific computational implementation, and its meaning is basically defined just by a “social contract” between its users. But computational language, by its nature, has a certain fundamental precision—because in the end what it specifies can always be “unambiguously executed on a computer”. Human language can usually get away with a certain vagueness. (When we say “planet” does it include exoplanets or not, etc.?) But in computational language we have to be precise and clear about all the distinctions we’re making.

在计算语言中命名时，利用普通人类语言通常是方便的。但它们在计算语言中的含义必然是精确的——可能涵盖也可能不涵盖典型人类语言用法中的某些特定含义。

It’s often convenient to leverage ordinary human language in making up names in computational language. But the meanings they have in computational language are necessarily precise—and might or might not cover some particular connotation in typical human language usage.

如何找出适合通用符号话语语言的基本“本体论”？嗯，这并不容易。这也许就是为什么自两千多年前亚里士多德的初步开端以来，进展甚微。但今天我们对如何以计算方式思考世界了解得如此之多（并且拥有来自我们的物理项目和 ruliad 思想的“基本形而上学”也无妨），这确实很有帮助。

How should one figure out the fundamental “ontology” suitable for a general symbolic discourse language? Well, it’s not easy. Which is perhaps why little has been done since the primitive beginnings Aristotle made more than two millennia ago. But it really helps that today we now know so much about how to think about the world computationally (and it doesn’t hurt to have a “fundamental metaphysics” from our Physics Project and the idea of the ruliad).

但这一切在 ChatGPT 的背景下意味着什么？通过训练，ChatGPT 有效地“拼凑”了相当数量的语义语法。但它的成功本身给了我们一个理由，认为以计算语言形式构建更完整的东西是可行的。而且，与我们目前对 ChatGPT 内部机制的了解不同，我们可以期望设计出易于人类理解的计算语言。

But what does all this mean in the context of ChatGPT? From its training ChatGPT has effectively “pieced together” a certain (rather impressive) quantity of what amounts to semantic grammar. But its very success gives us a reason to think that it’s going to be feasible to construct something more complete in computational language form. And, unlike what we’ve so far figured out about the innards of ChatGPT, we can expect to design the computational language so that it’s readily understandable to humans.

当我们谈论语义语法时，可以类比三段论逻辑。起初，三段论逻辑本质上是一套关于用人类语言表达的陈述的规则。但（是的，两千年后）当形式逻辑发展起来时，三段论逻辑的原始基本构造被用来构建巨大的“形式塔”，包括例如现代数字电路的操作。因此，我们可以预期，更通用的语义语法也会如此。起初，它可能只能处理简单的模式，例如以文本形式表达。但一旦其整个计算语言框架建立起来，我们可以预期它将被用来建立“广义语义逻辑”的高塔，使我们能够以精确和形式化的方式处理各种事物，这些事物以前除了通过人类语言及其所有模糊性在“底层水平”上，从未对我们可及。

When we talk about semantic grammar, we can draw an analogy to syllogistic logic. At first, syllogistic logic was essentially a collection of rules about statements expressed in human language. But (yes, two millennia later) when formal logic was developed, the original basic constructs of syllogistic logic could now be used to build huge “formal towers” that include, for example, the operation of modern digital circuitry. And so, we can expect, it will be with more general semantic grammar. At first, it may just be able to deal with simple patterns, expressed, say, as text. But once its whole computational language framework is built, we can expect that it will be able to be used to erect tall towers of “generalized semantic logic”, that allow us to work in a precise and formal way with all sorts of things that have never been accessible to us before, except just at a “ground-floor level” through human language, with all its vagueness.

我们可以将计算语言和语义语法的构建视为一种终极的表示压缩。因为它允许我们谈论可能性的本质，而不必处理普通人类语言中存在的所有“措辞”。我们可以将 ChatGPT 的巨大优势视为类似的东西：因为它也在某种意义上“钻透”到了能够“以语义有意义的方式组合语言”而无需考虑不同可能措辞的程度。

We can think of the construction of computational language—and semantic grammar—as representing a kind of ultimate compression in representing things. Because it allows us to talk about the essence of what’s possible, without, for example, dealing with all the “turns of phrase” that exist in ordinary human language. And we can view the great strength of ChatGPT as being something a bit similar: because it too has in a sense “drilled through” to the point where it can “put language together in a semantically meaningful way” without concern for different possible turns of phrase.

那么，如果我们把 ChatGPT 应用于底层的计算语言会怎样？计算语言可以描述什么是可能的。但还可以添加的是“什么是流行的”感觉——例如基于阅读网络上的所有内容。但随后——在底层——使用计算语言意味着像 ChatGPT 这样的系统能够立即且基本地访问那些用于利用潜在不可约计算的终极工具。这使得它成为一个不仅能“生成合理文本”，而且能期望计算出关于该文本是否对世界或它应该谈论的内容做出“正确”陈述的任何可计算结果的系统。

So what would happen if we applied ChatGPT to underlying computational language? The computational language can describe what’s possible. But what can still be added is a sense of “what’s popular”—based for example on reading all that content on the web. But then—underneath—operating with computational language means that something like ChatGPT has immediate and fundamental access to what amount to ultimate tools for making use of potentially irreducible computations. And that makes it a system that can not only “generate reasonable text”, but can expect to work out whatever can be worked out about whether that text actually makes “correct” statements about the world—or whatever it’s supposed to be talking about.

那么……ChatGPT 在做什么，为什么有效？ So … What Is ChatGPT Doing, and Why Does It Work?

ChatGPT 的基本概念在某种程度上相当简单。从网络、书籍等人类创作的文本的巨大样本开始。然后训练一个神经网络来生成“像这样”的文本。特别是，使其能够从“提示”开始，然后继续生成“像它训练过的内容”的文本。

The basic concept of ChatGPT is at some level rather simple. Start from a huge sample of human-created text from the web, books, etc. Then train a neural net to generate text that’s “like this”. And in particular, make it able to start from a “prompt” and then continue with text that’s “like what it’s been trained with”.

正如我们所看到的，ChatGPT 中的实际神经网络由非常简单的元素组成——尽管有数十亿个。神经网络的基本操作也非常简单，本质上包括将迄今为止生成的文本中提取的输入“一次性通过其元素”（没有任何循环等）来生成每个新词（或词的一部分）。

As we’ve seen, the actual neural net in ChatGPT is made up of very simple elements—though billions of them. And the basic operation of the neural net is also very simple, consisting essentially of passing input derived from the text it’s generated so far “once through its elements” (without any loops, etc.) for every new word (or part of a word) that it generates.

但引人注目且出乎意料的是，这个过程可以产生成功“像”网络、书籍等中存在的文本。它不仅连贯地使用人类语言，还能“说出”遵循其提示的“内容”，利用它“读过”的内容。它并不总是说出“全局有意义”（或对应于正确计算）的内容——因为（例如，没有访问 Wolfram|Alpha 的“计算超能力”）它只是根据训练材料中事物“听起来的样子”说出“听起来正确”的内容。

But the remarkable—and unexpected—thing is that this process can produce text that’s successfully “like” what’s out there on the web, in books, etc. And not only is it coherent human language, it also “says things” that “follow its prompt” making use of content it’s “read”. It doesn’t always say things that “globally make sense” (or correspond to correct computations)—because (without, for example, accessing the “computational superpowers” of Wolfram|Alpha) it’s just saying things that “sound right” based on what things “sounded like” in its training material.

ChatGPT 的具体工程使其相当引人注目。但最终（至少在其能够使用外部工具之前），ChatGPT 只是从其积累的“传统智慧统计”中提取出一些“连贯的文本线索”。但结果如此像人类，令人惊叹。正如我所讨论的，这暗示了至少在科学上非常重要的一点：人类语言（及其背后的思维模式）在结构上比我们想象的更简单、更“有规律”。ChatGPT 已经隐式地发现了这一点。但我们有可能通过语义语法、计算语言等显式地揭示它。

The specific engineering of ChatGPT has made it quite compelling. But ultimately (at least until it can use outside tools) ChatGPT is “merely” pulling out some “coherent thread of text” from the “statistics of conventional wisdom” that it’s accumulated. But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of thinking behind it) are somehow simpler and more “law like” in their structure than we thought. ChatGPT has implicitly discovered it. But we can potentially explicitly expose it, with semantic grammar, computational language, etc.

ChatGPT 在生成文本方面所做的非常令人印象深刻——结果通常非常像我们人类会产生的。那么这是否意味着 ChatGPT 像大脑一样工作？其底层的人工神经网络结构最终是基于对大脑的理想化建模。而且，当我们人类生成语言时，所发生事情的许多方面很可能非常相似。

What ChatGPT does in generating text is very impressive—and the results are usually very much like what we humans would produce. So does this mean ChatGPT is working like a brain? Its underlying artificial-neural-net structure was ultimately modeled on an idealization of the brain. And it seems quite likely that when we humans generate language many aspects of what’s going on are quite similar.

在训练（又称学习）方面，大脑和当前计算机的不同“硬件”（以及可能一些未开发的算法思想）迫使 ChatGPT 使用一种可能与大脑相当不同（并且在某些方面效率低得多）的策略。还有另外一点：与典型的算法计算不同，ChatGPT 内部没有“循环”或“对数据重新计算”。这不可避免地限制了它的计算能力——即使相对于当前计算机，也绝对相对于大脑。

When it comes to training (AKA learning) the different “hardware” of the brain and of current computers (as well as, perhaps, some undeveloped algorithmic ideas) forces ChatGPT to use a strategy that’s probably rather different (and in some ways much less efficient) than the brain. And there’s something else as well: unlike even in typical algorithmic computation, ChatGPT doesn’t internally “have loops” or “recompute on data”. And that inevitably limits its computational capability—even with respect to current computers, but definitely with respect to the brain.

目前尚不清楚如何“修复这个问题”同时保持以合理效率训练系统的能力。但这样做可能会让未来的 ChatGPT 做更多“像大脑一样的事情”。当然，有很多事情大脑做得并不好——特别是涉及不可约计算的事情。对于这些，大脑和像 ChatGPT 这样的东西都必须寻求“外部工具”——比如 Wolfram 语言。

It’s not clear how to “fix that” and still maintain the ability to train the system with reasonable efficiency. But to do so will presumably allow a future ChatGPT to do even more “brain-like things”. Of course, there are plenty of things that brains don’t do so well—particularly involving what amount to irreducible computations. And for these both brains and things like ChatGPT have to seek “outside tools”—like Wolfram Language.

但就目前而言，看到 ChatGPT 已经能够做到的事情令人兴奋。在某种程度上，它是基本科学事实的一个极好例子，即大量简单的计算元素可以完成非凡且出乎意料的事情。但它也可能提供了两千年来我们最好的动力，去更好地理解人类语言及其背后的思维过程这一人类状况核心特征的基本特征和原则可能是什么。

But for now it’s exciting to see what ChatGPT has already been able to do. At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. But it also provides perhaps the best impetus we’ve had in two thousand years to understand better just what the fundamental character and principles might be of that central feature of the human condition that is human language and the processes of thinking behind it.

致谢 Thanks

我关注神经网络的发展已有大约 43 年，在此期间与许多人就此进行过交流。其中——有些是多年前的，有些是近期的，还有些是跨越多年保持联系的——包括：Giulio Alessandrini、Dario Amodei、Etienne Bernard、Taliesin Beynon、Sebastian Bodenstein、Greg Brockman、Jack Cowan、Pedro Domingos、Jesse Galef、Roger Germundsson、Robert Hecht-Nielsen、Geoff Hinton、John Hopfield、Yann LeCun、Jerry Lettvin、Jerome Louradour、Marvin Minsky、Eric Mjolsness、Cayden Pierce、Tomaso Poggio、Matteo Salvarezza、Terry Sejnowski、Oliver Selfridge、Gordon Shaw、Jonas Sjöberg、Ilya Sutskever、Gerry Tesauro 和 Timothee Verdier。在撰写本文时，我特别感谢 Giulio Alessandrini 和 Brad Klee 的帮助。

I’ve been following the development of neural nets now for about 43 years, and during that time I’ve interacted with many people about them. Among them—some from long ago, some from recently, and some across many years—have been: Giulio Alessandrini, Dario Amodei, Etienne Bernard, Taliesin Beynon, Sebastian Bodenstein, Greg Brockman, Jack Cowan, Pedro Domingos, Jesse Galef, Roger Germundsson, Robert Hecht-Nielsen, Geoff Hinton, John Hopfield, Yann LeCun, Jerry Lettvin, Jerome Louradour, Marvin Minsky, Eric Mjolsness, Cayden Pierce, Tomaso Poggio, Matteo Salvarezza, Terry Sejnowski, Oliver Selfridge, Gordon Shaw, Jonas Sjöberg, Ilya Sutskever, Gerry Tesauro and Timothee Verdier. For help with this piece, I’d particularly like to thank Giulio Alessandrini and Brad Klee.

其他资源 Additional Resources

引用本文：Stephen Wolfram (2023), "What Is ChatGPT Doing ... and Why Does It Work?," Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work。

Cite this as Stephen Wolfram (2023), "What Is ChatGPT Doing ... and Why Does It Work?," Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work.

Stephen Wolfram (2023), "What Is ChatGPT Doing ... and Why Does It Work?," Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work。

Stephen Wolfram (2023), "What Is ChatGPT Doing ... and Why Does It Work?," Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work.

Wolfram, Stephen. "What Is ChatGPT Doing ... and Why Does It Work?." Stephen Wolfram Writings. February 14, 2023. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work。

Wolfram, Stephen. "What Is ChatGPT Doing ... and Why Does It Work?." Stephen Wolfram Writings. February 14, 2023. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work.

Wolfram, S. (2023, February 14). What is ChatGPT doing ... and why does it work?. Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work。

Wolfram, S. (2023, February 14). What is ChatGPT doing ... and why does it work?. Stephen Wolfram Writings. writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work.

发布于：人工智能、语言与交流、新技术

Posted in: Artificial Intelligence, Language & Communication, New Technology

请输入您的评论（至少 5 个字符）。

Please enter your comment (at least 5 characters).

60 条评论 60 comments

1. 非常令人印象深刻，简洁而富有表现力的叙述，否则可能会被归为炼金术或巫术科学。感谢 Steve 和团队！！非常感谢您的启迪！Nicos Kekchidis 2023 年 6 月 2 日 下午 7:55

1. Very impressive, succinct yet eloquent narrative to what otherwise could be relegated to alchemy or voodoo science. Thank you Steve and the Team!! Much obliged to your enlightenment! Nicos Kekchidis June 2, 2023 at 7:55 pm

2. 任何想了解神经网络/LLM 的人都应该读一读这篇文章。对一个复杂主题的清晰解释。感谢 Stephen 和团队让我们变得更聪明。Yuri 2023 年 6 月 6 日 上午 9:26

2. This should be a must read for anyone who wants to understand Neural Networks \ LLMs. Clear explanation of a complicated subject. Thank you Stephen and the team for making us a bit smarter. Yuri June 6, 2023 at 9:26 am

3. 非常感谢 Stephen Wolfram 和团队这篇有意义、有启发性且相当长*的文章。作为“外行”（普通的后端软件工程师），我终于理解了 GPT/LLM 模型以及 AI/ML 的基本概念。我非常感激您省略了 AI/ML 社区常见的那些流行词。*我理解，主题本身可能使得无法进一步压缩这里提供的丰富信息，同时仍能让读者理解所呈现的主题。来自德国的问候 Paul Schmidt 2023 年 6 月 15 日 上午 5:27

3. Thank you very much Stephen Wolfram & team for this meaningful, enlightening and rather long* 🙂 article.

4. 这是有史以来最好的解释，充满了令人惊叹的反思！非常感谢。Daniele 2024 年 5 月 10 日 下午 9:31

As a “laymen” (run of the mill backend software engineer) I finally got the basic concepts behind GPT/LLM models and AI/ML in general.

5. Stephen，我们所有人都感谢您和团队发表这篇文章。Rickson 2024 年 5 月 22 日 上午 6:11

I very much appreciate that you left out the usual buzzword bingo of the AI/ML community.

6. 发现它引人入胜，因为 Stephan 通过例子剖析模型来讲述故事！尽管篇幅很长，但文章读起来非常流畅。任何试图了解 GenAI 巨兽内部工作原理的人都必读！Vinayaka Hunasekoppa 2024 年 5 月 26 日 上午 2:21

* I appreciate that the subject matter probably makes it impossible to condense the wealth of information provided here even more than you already did and still provide the reader with the ability to understand the topics presented.

7. 这里有很多乐观情绪，但我觉得有两个地方，这类模型的承诺建立在对语言的误解之上——所以这是一位语言学家的几点小批评。1) 自 1970 年以来，我们一直使用一种并不特别复杂的语言模型。ChatGPT 并没有教会我们这一点，逻辑已经教会了我们。计算机不需要学习和简单性的效率，但人类学习语言非常快，语言对于交流是高效的，因此它们本质上不可能具有巨大的计算复杂性。它们总是关于创建模式和重复，以便我们能够被理解。考虑到这些模型使用的处理和调整量，它们能够产生相当可理解和相关的语言输出并不令人惊讶，只是与以往经验不同。（“chicken chicken chocolate chicken”神经网络也曾令人惊讶，但原因不同。）2) 基于语言训练的 LLM 有一个主要弱点，即它们只受二阶信息的影响。人类是感知和处理机器，我们产生的语言是这一过程的副产品，而不是结果。因此，如果你期望 LLM 产生关于世界的良好数据，我们必须超越语言作为输入。它太混乱了。它可以产生一篇论文的表象，但它从未写过一篇值得阅读的论文，因为它无法将其对产生的语义内容的理解与世界进行比较，因为它缺乏两样东西——递归和理解。当人类以同样的方式运作时，重复他们听到的内容，而不将输出与世界进行比较，也会犯很多错误。我们需要训练数据是世界本身，而不是语言。Marid 2024 年 5 月 29 日 上午 9:12

Greetings from Germany Paul Schmidt June 15, 2023 at 5:27 am

8. 关于语义空间中的轨迹。我们在二维投影中看不到任何结构和逻辑，但即使对于三维空间，也有无限多的二维投影，而我们拥有 12000 维空间。也许某些投影会显示一些几何和逻辑。我经常看到提到“统计”，比如“它积累的传统智慧的统计”，但我认为这是误导。统计在训练结束后就结束了。在训练过程中，神经网络的深层创建关于输入数据的高级抽象。就像在向神经网络输入数千个 Pi 数字并对其错误的下一个数字输出施加惩罚后，神经网络的记忆被填满，它必须发明一个公式/定律来生成任何新数字。我们能称之为统计吗？我怀疑。我认为在深层中应该存在接近通用计算语言的东西。内层存储了太多关于语言的技术信息来处理标记、识别语法等，外层准备输出并以相反的顺序做同样的事情。但在深层中，神经网络“理解”了所发生事情的逻辑，它较少依赖于文本和特定语言，它应该存储高级特征和关于引力、颜色等的知识。它们不是以模糊文本的形式存储，而是以语义空间区域的形式存储。Andrey 2024 年 6 月 23 日 下午 1:34

4. This is the best explanation ever, full of amazing reflections!

9. 一篇关于 ChatGPT 背后原理的好文章！我会把它传给我对这个话题感兴趣的学生。Hong Xiang 2024 年 6 月 23 日 下午 9:24

Thank you very much. Daniele May 10, 2024 at 9:31 pm

10. 引人入胜的解释，Stephen！“只是一次加一个词”的想法确实使过程去神秘化，同时仍然突出了令人难以置信的涌现智能。这让我想到针对非常具体、创造性的任务微调这类模型的潜力，在这些任务中，微妙的词汇选择至关重要。感谢您如此清晰地解释！AvatarCraftAI 2026 年 6 月 25 日 下午 11:33

5. Stephen, all of us are grateful to you and your team for putting up this article. Rickson May 22, 2024 at 6:11 am

启动 Wolfram 语言和 Mathematica 版本 15：内置（有用的）AI 和大量新核心功能 AI 和大量新核心功能

6. Found it fascinating because of how Stephan dissects the model with examples to tell a story! The article reads very well despite being lengthy. A must read for anybody trying to look under the hood of the GenAI juggernaut! Vinayaka Hunasekoppa May 26, 2024 at 2:21 am

程序之间的游戏：竞争的规则学

7. There’s a lot of optimism in here, but I feel like there are two places where the promise of this kind of model is built on misconceptions about language–so here is one linguist’s minor critiques.

使 Wolfram 技术作为 LLM 系统的基础工具可用

1) Since 1970 we’ve been working with a model of language that is NOT particularly complex. ChatGPT hasn’t taught us this, logic has. Computers have no need for efficiency and simplicity in learning, but humans learn language very fast and languages are efficient for communication, so they inherently cannot be hugely computationally complex. They are always about creating patterns and repetition so that we can be understood. With the amount of processing and twiddling that these models use, it’s not a logical surprise that they can produce reasonably comprehensible and relevant linguistic outputs, merely a surprise of previous experience. (The “chicken chicken chocolate chicken” neural nets were surprising too, but for different reasons.)

即时超级计算：启动 Wolfram 计算服务

2) LLMs trained on language have a major weakness, which is that they are informed by only second order information. Humans are perceiving and processing machines, and what we produce as language is a byproduct of that, not a result. So if you expect LLMs to produce good data about the world, we’ve got to move past language as an input. It’s too messy. It can produce a semblance of an essay, but it has never written an essay worth reading, because it cannot compare its understanding of its produced semantic content to the world, because it lacks two things–recursion, and comprehension. Humans make plenty of mistakes when they function in the same way, repeating what they have heard, and not comparing their output to the world. We need training data to be the world itself, not language. Marid May 29, 2024 at 9:12 am

启动 Wolfram 语言和 Mathematica 版本 15：内置（有用的）AI 和大量新核心功能 AI 和大量新核心功能

8. About trajectories in semantic space. We cannot see any structure and logic in 2d projection but even for 3d space there are infinite number of 2d projections and we have 12000d space. Maybe some projections show some geometry and logic.

程序之间的游戏：竞争的规则学

I often see mentioning “statistics ” like “statistics of conventional wisdom that it’s accumulated” but I think it is misleading. Statistic ends after training. During training deep layers of neural network create high-level abstractions about input data. It is like after feeding thousands of Pi digits to neural network and adding penalties for wrong next digit it outputs, neural net’s memory is filled in and it has to invent a formula/low that allows it to generate any new digit. Can we call it statistic? I doubt it.

使 Wolfram 技术作为 LLM 系统的基础工具可用

I think that something close to universal computational language should exist in deep layers. Inner layers store too much technical information about language to process tokens, recognize syntax and so, outer layers prepare output and do the same in reverse order. But in deep layers neural net “understands” the logic of what happens, it is less dependent in text and specific language, it should store high-level features and knowledge of gravitation, colors etc. They are stored not in form of vague texts but in form of regions of semantic space. Andrey June 23, 2024 at 1:34 pm

最终存在什么？形而上学与 Ruliad

9. A nice article about what’s behind ChatGPT! I will circulate this to my students who are interested in this topic. Hong Xiang June 23, 2024 at 9:24 pm

P vs. NP 与计算的难度：一种规则学方法

10. Fascinating explanation, Stephen! The idea that it’s “just adding one word at a time” really demystifies the process while still highlighting the incredible emergent intelligence. It makes me think about the potential for fine-tuning such models for very specific, creative tasks where nuanced word choice is paramount. Thanks for breaking this down so clearly! AvatarCraftAI June 25, 2026 at 11:33 pm

创意制造者：对著名人物的个人视角

[](https://writings.stephenwolfram.com/2026/06/launching-version-15-of-wolfram-language-mathematica-built-in-useful-ai-lots-of-new-core-functionality/ "Permanent Link to Launching Version 15 of Wolfram Language & Mathematica: Built-in (Useful) AI & Lots of New Core Functionality")

© Stephen Wolfram, LLC | 开放内容：(代码：) | 条款 | RSS

Launching Version 15 of Wolfram Language & Mathematica: Built-in (Useful) AI & Lots of New Core Functionality AI & Lots of New Core Functionality")

[](https://writings.stephenwolfram.com/2026/06/games-between-programs-the-ruliology-of-competition/ "Permanent Link to Games between Programs: The Ruliology of Competition")

[](https://writings.stephenwolfram.com/2026/02/making-wolfram-tech-available-as-a-foundation-tool-for-llm-systems/ "Permanent Link to Making Wolfram Tech Available as a Foundation Tool for LLM Systems")

[](https://writings.stephenwolfram.com/2025/12/instant-supercompute-launching-wolfram-compute-services/ "Permanent Link to Instant Supercompute: Launching Wolfram Compute Services")

[](https://writings.stephenwolfram.com/2026/06/launching-version-15-of-wolfram-language-mathematica-built-in-useful-ai-lots-of-new-core-functionality/ "Permanent Link to Launching Version 15 of Wolfram Language & Mathematica: Built-in (Useful) AI & Lots of New Core Functionality")

Launching Version 15 of Wolfram Language & Mathematica: Built-in (Useful) AI & Lots of New Core Functionality AI & Lots of New Core Functionality")

[](https://writings.stephenwolfram.com/2026/06/games-between-programs-the-ruliology-of-competition/ "Permanent Link to Games between Programs: The Ruliology of Competition")

[](https://writings.stephenwolfram.com/2026/02/making-wolfram-tech-available-as-a-foundation-tool-for-llm-systems/ "Permanent Link to Making Wolfram Tech Available as a Foundation Tool for LLM Systems")

[](https://writings.stephenwolfram.com/2026/02/what-ultimately-is-there-metaphysics-and-the-ruliad/ "Permanent Link to What Ultimately Is There? Metaphysics and the Ruliad")

[](https://writings.stephenwolfram.com/2026/01/p-vs-np-and-the-difficulty-of-computation-a-ruliological-approach/ "Permanent Link to P vs. NP and the Difficulty of Computation: A Ruliological Approach")

[](https://www.amazon.com/Idea-Makers-Personal-Perspectives-Notable/dp/1579550037)