玻尔兹曼机的学习算法

概述 Overview

DAVID H. ACKLEY GEOFFREY E. HINTON 卡内基梅隆大学计算机科学系 TERRENCE J. SEJNOWSKI 约翰霍普金斯大学生物物理学系

DAVID H. ACKLEY GEOFFREY E. HINTON Computer Science Department Carnegie-Mellon University TERRENCE J. SEJNOWSKI Biophysics Department The Johns Hopkins University

由简单处理单元组成的大规模并行网络的计算能力，在于元件间硬件连接所提供的通信带宽。这些连接能够使系统知识的很大一部分在极短时间内应用于问题实例。大规模并行网络似乎特别适合的一种计算是大型约束满足搜索，但要高效利用这些连接，必须满足两个条件：首先，必须找到一种适合并行网络的搜索技术。其次，必须有一种选择内部表示的方法，使得预先存在的硬件连接能够被高效地用于编码搜索领域中的约束。我们描述了一种基于统计力学的一般并行搜索方法，并展示了它如何导出一个一般学习规则，用于修改连接强度，从而以高效的方式融入关于任务领域的知识。我们描述了一些简单例子，其中学习算法创建的内部表示被证明是使用预先存在的连接结构的最有效方式。

The computational power of massively parallel networks of simple processing elements resides in the communication bandwidth provided by the hardware connections between elements. These connections can allow a significant fraction of the knowledge of the system to be applied to an instance of a problem in a very short time. One kind of computation for which massively parallel networks appear to be well suited is large constraint satisfaction searches, but to use the connections efficiently two conditions must be met: First, a search technique that is suitable for parallel networks must be found. Second, there must be some way of choosing internal representations which allow the preexisting hardware connections to be used efficiently for encoding the constraints in the domain being searched. We describe a general parallel search method, based on statistical mechanics, and we show how it leads to a general learning rule for modifying the connection strengths so as to incorporate knowledge about a task domain in an efficient way. We describe some simple examples in which the learning algorithm creates internal representations that are demonstrably the most efficient way of using the preexisting connectivity structure.

关于大脑结构和新 VLSI 技术潜力的证据，引发了对“连接主义”系统兴趣的复兴。

Evidence about the architecture of the brain and the potential of the new VLSI technology have led to a resurgence of interest in “connectionist” systems,

本报告的研究工作得到了系统开发基金会的资助。感谢 Peter Brown、Francis Crick、Mark Derthick、Scott Fahlman、Jerry Feldman、Stuart Geman、Gail Gong、John Hopfield、Jay McClelland、Barak Pearlmutter、Harry Printz、Dave Rumelhart、Tim Shallice、Paul Smolensky、Rick Szeliski 和 Venkatraman Venkatasubramanian 的有益讨论。重印请求请寄至 David Ackley，卡内基梅隆大学计算机科学系，匹兹堡，PA 15213。147 148 ACKLEY. HINTON. AND SEJNOWSKI

The research reported here was supported by grants from the System Development Foundation. We thank Peter Brown, Francis Crick, Mark Derthick, Scott Fahlman, Jerry Feldman, Stuart Geman, Gail Gong, John Hopfield, Jay McClelland, Barak Pearlmutter, Harry Printz, Dave Rumelhart, Tim Shallice, Paul Smolensky, Rick Szeliski, and Venkatraman Venkatasubramanian for helpful discussions. Reprint requests should be addressed to David Ackley, Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA 15213. 147 148 ACKLEY. HINTON. AND SEJNOWSKI

系统（Feldman & Ballard, 1982; Hinton & Anderson, 1981）将其长期知识存储为简单类神经处理单元之间连接的强度。这些网络显然适用于视觉等任务，这些任务可以在具有物理连接的并行网络中高效执行，而连接恰好位于进程需要通信的地方。对于像从稀疏深度数据进行表面插值这样的问题（Crimson, 1981; Terzopoulos, 1984），其中必要的决策单元和通信路径可以预先确定，相对容易看出如何充分利用大规模并行性。更困难的问题是发现不需要将如此多的问题依赖信息内建到网络架构中的并行组织。理想情况下，这样的系统会将其处理单元和通信路径的给定结构适应于它所面临的任何问题。本文提出了一种并行约束满足网络，我们称之为“Boltzmann 机”，它只需通过展示领域中的示例就能学习表征该领域的基本约束。网络修改其连接强度，以构建一个内部生成模型，该模型以与展示示例相同的概率分布产生示例。然后，当展示任何特定示例时，网络可以通过找到……来“解释”它。

systems (Feldman & Ballard, 1982; Hinton & Anderson, 1981) that store their long-term knowledge as the strengths of the connections between simple neuron-like processing elements. These networks are clearly suited to tasks like vision that can be performed efficiently in parallel networks which have physical connections in just the places where processes need to communicate. For problems like surface interpolation from sparse depth data (Crimson, 1981; Terzopoulos, 1984) where the necessary decision units and communication paths can be determined in advance, it is relatively easy to see how to make good use of massive parallelism. The more difficult problem is to discover parallel organizations that do not require so much problem-dependent information to be built into the architecture of the network. Ideally, such a system would adapt a given structure of processors and communication paths to whatever problem it was faced with. This paper presents a type of parallel constraint satisfaction network which we call a “Boltzmann Machine” that is capable of learning the underlying constraints that characterize a domain simply by being shown examples from the domain. The network modifies the strengths of its connections so as to construct an internal generative model that produces examples with the same probability distribution as the examples it is shown. Then, when shown any particular example, the network can “interpret” it by finding

生成示例的内部模型中的变量。当给出部分示例时，网络可以通过找到生成该部分示例的内部变量值，然后利用这些值生成剩余部分，从而补全该示例。目前，我们有一个有趣的数学结果，保证某种学习过程能够构建内部表示，使得连接强度能够捕获隐含在从某个领域提取的大量示例中的约束。我们还有仿真表明，该理论对某些简单情况有效，但当前版本的学习算法非常缓慢。寻找允许并行网络学习其环境结构的通用原则往往始于网络随机连线的假设。在我们看来，这种观点与认为所有知识都是天生的错误一样。如果存在对网络要执行的特定任务有利的连接结构，那么在开始时将其构建进去会更高效。然而，并非所有任务都可以预见，即使是可预见的任务，微调可能仍然有帮助。另一个常见的信念是，通用的连接主义学习规则将使顺序的“基于规则”模型变得不必要。我们认为，这种观点源于对大型系统多级描述需求的误解；大型系统可以根据分析粒度有效地被视为并行或串行。大多数在顺序模型背景下研究的关键问题和疑问并不会在连接主义模型中神奇消失。仍然有必要执行玻尔兹曼机器学习 149。

The variables in the internal model that would generate the example. When shown a partial example, the network can complete it by finding internal variable values that generate the partial example and using them to generate the remainder. At present, we have an interesting mathematical result that guarantees that a certain learning procedure will build internal representations which allow the connection strengths to capture the underlying constraints that are implicit in a large ensemble of examples taken from a domain. We also have simulations which show that the theory works for some simple cases, but the current version of the learning algorithm is very slow. The search for general principles that allow parallel networks to learn the structure of their environment has often begun with the assumption that networks are randomly wired. This seems to us to be just as wrong as the view that all knowledge is innate. If there are connectivity structures that are good for particular tasks that the network will have to perform, it is much more efficient to build these in at the start. However, not all tasks can be foreseen, and even for ones that can, fine-tuning may still be helpful. Another common belief is that a general connectionist learning rule would make sequential "rule-based" models unnecessary. We believe that this view stems from a misunderstanding of the need for multiple levels of description of large systems, which can be usefully viewed as either parallel or serial depending on the grain of the analysis. Most of the key issues and questions that have been studied in the context of sequential models do not magically disappear in connectionist models. It is still necessary to perform Boltzmann Machine Learning 149.

搜索问题的良好解或对感知输入的良好解释，并创建复杂的内部表示。最终，有必要弥合面向硬件的连接主义描述与更为抽象的符号操作模型之间的鸿沟，后者已被证明是描述人类信息处理的一种极其强大且普遍的方式（Newell & Simon, 1972）。2. 玻尔兹曼机 玻尔兹曼机是一种并行计算组织，非常适合涉及大量“弱”约束的约束满足任务。约束满足搜索（例如 Waltz, 1975; Winston, 1984）通常使用任何解都必须满足的“强”约束。在诸如游戏和谜题等问题领域中，目标标准往往具有这种特征，因此强约束是常态。而在某些问题领域，例如寻找图像最合理的解释，许多标准并非全有或全无，即使是最佳解也经常违反某些约束（Hinton, 1977）。一种更适合此类领域的变体使用弱约束，违反时会付出代价。解的质量则由其违反的所有约束的总代价决定。例如，在感知解释任务中，这个总代价应反映该解释的不可信程度。该机器由称为单元的基本计算元素组成，这些单元通过双向链接相互连接。一个单元始终处于两种状态之一：开或关，它根据相邻单元的状态及其与它们之间链接上的权重，以概率函数的方式采用这些状态。权重可以是任意符号的实数。单元的开或关被视为系统当前接受或拒绝关于该领域的某个基本假设。链接上的权重表示两个假设之间的一对弱约束。正权重表明两个假设倾向于相互支持；如果当前接受其中一个，则接受另一个的可能性应该更大。相反，负权重表明，在其他条件相同的情况下，两个假设不应同时被接受。链接权重是对称的，在两个方向上具有相同的强度（Hinton & Sejnowski, 1983）。

searches for good solutions to problems or good interpretations of perceptual input, and to create complex internal representations. Ultimately it will be necessary to bridge the gap between hardware-oriented connectionist descriptions and the more abstract symbol manipulation models that have proved to be an extremely powerful and pervasive way of describing human information processing (Newell & Simon, 1972). 2. THE BOLTZMANN MACHINE The Boltzmann Machine is a parallel computational organization that is well suited to constraint satisfaction tasks involving large numbers of "weak" constraints. Constraint-satisfaction searches (e.g., Waltz, 1975; Winston, 1984) normally use "strong" constraints that must be satisfied by any solution. In problem domains such as games and puzzles, for example, the goal criteria often have this character, so strong constraints are the rule. In some problem domains, such as finding the most plausible interpretation of an image, many of the criteria are not all-or-none, and frequently even the best possible solution violates some constraints (Hinton, 1977). A variation that is more appropriate for such domains uses weak constraints that incur a cost when violated. The quality of a solution is then determined by the total cost of all the constraints that it violates. In a perceptual interpretation task, for example, this total cost should reflect the implausibility of the interpretation. The machine is composed of primitive computing elements called units that are connected to each other by bidirectional links. A unit is always in one of two states, on or off, and it adopts these states as a probabilistic function of the states of its neighboring units and the weights on its links to them. The weights can take on real values of either sign. A unit being on or off is taken to mean that the system currently accepts or rejects some elemental hypothesis about the domain. The weight on a link represents a weak pairwise constraint between two hypotheses. A positive weight indicates that the two hypotheses tend to support one another; if one is currently accepted, accepting the other should be more likely. Conversely, a negative weight suggests, other things being equal, that the two hypotheses should not both be accepted. Link weights are symmetric, having the same strength in both directions (Hinton & Sejnowski, 1983).

由此产生的结构与 Hopfield (1982) 描述的系统相关，并且与他的系统一样，网络的每个全局状态都可以被赋予一个称为该状态“能量”的单一数字。在正确的假设下，可以使得单个单元的行为最小化全局能量。如果某些单元被外部强制或“夹持”到特定状态以表示特定输入，那么系统将找到与该输入兼容的最小能量配置。配置的能量可以解释为该假设组合违反问题领域中隐含约束的程度，因此在最小化能量的过程中，系统会朝着越来越满足问题领域约束的该输入的“解释”演化。全局配置的能量定义为 \[ E = -\frac{1}{2} \sum_{i,j} w_{ij} S_i S_j + \sum_i \theta_i S_i \] 其中 \( w_{ij} \) 是单元 i 和 j 之间的连接强度，\( S_i \) 如果单元 i 为开则为 1，否则为 0，\( \theta_i \) 是阈值。2.1 能量最小化 一种寻找真值组合（局部最小值）的简单算法是：给定其他假设的当前状态，将每个假设切换到其两个状态中能产生较低总能量的那个状态。如果硬件单元异步做出决策，并且传输时间可忽略，那么系统总会稳定到一个局部能量最小值 (Hopfield, 1982)。由于连接是对称的，整个系统在拒绝第 k 个假设与接受第 k 个假设时的能量差可以由第 k 个单元局部确定，这个“能量间隙”正是 \( \Delta E_k = \sum_j w_{kj} S_j - \theta_k \)。因此，最小化单元贡献的能量的规则是：如果该单元来自其他单元和系统外部的总输入超过其阈值，则它采用开状态。这就是二元阈值单元的常见规则。阈值项可以通过以下观察从方程 (1) 和 (2) 中消除：\( \theta_i \) 对全局能量或单个单元能量间隙的影响，与单元 i 和一个被定义为始终处于开状态的特殊单元之间强度为 \( -\theta_i \) 的链接的效果相同。这个“真单元”不需要有物理实体，但它通过允许将单元的阈值与链接以相同方式处理来简化计算。值 \( -\theta_i \) 被称为单元 i 的偏置。如果永久玻尔兹曼机器学习 151。

The resulting structure is related to a system described by Hopfield (1982), and as in his system, each global state of the network can be assigned a single number called the "energy" of that state. With the right assumptions, the individual units can be made to act so as to minimize the global energy. If some of the units are externally forced or "clamped" into particular states to represent a particular input, the system will then find the minimum energy configuration that is compatible with that input. The energy of a configuration can be interpreted as the extent to which that combination of hypotheses violates the constraints implicit in the problem domain, so in minimizing energy the system evolves towards "interpretations" of that input that increasingly satisfy the constraints of the problem domain. The energy of a global configuration is defined as \[ E = -\frac{1}{2} \sum_{i,j} w_{ij} S_i S_j + \sum_i \theta_i S_i \] where \( w_{ij} \) is the strength of connection between units i and j, \( S_i \) is 1 if unit i is on and 0 otherwise, and \( \theta_i \) is a threshold. 2.1 Minimizing Energy A simple algorithm for finding a combination of truth values that is a local minimum is to switch each hypothesis into whichever of its two states yields the lower total energy given the current states of the other hypotheses. If hardware units make their decisions asynchronously, and if transmission times are negligible, then the system always settles into a local energy minimum (Hopfield, 1982). Because the connections are symmetric, the difference between the energy of the whole system with the kth hypothesis rejected and its energy with the kth hypothesis accepted can be determined locally by the kth unit, and this "energy gap" is just \( \Delta E_k = \sum_j w_{kj} S_j - \theta_k \). Therefore, the rule for minimizing the energy contributed by a unit is to adopt the on state if its total input from the other units and from outside the system exceeds its threshold. This is the familiar rule for binary threshold units. The threshold terms can be eliminated from Eqs. (1) and (2) by making the following observation: the effect of \( \theta_i \) on the global energy or on the energy gap of an individual unit is identical to the effect of a link with strength \( -\theta_i \) between unit i and a special unit that is by definition always held in the on state. This "true unit" need have no physical reality, but it simplifies the computations by allowing the threshold of a unit to be treated in the same manner as the links. The value \( -\theta_i \) is called the bias of unit i. If a permanent Boltzmann Machine Learning 151.

the variables in the internal model that would generate the exam-ple. When shown a partial example, the network can complete it by finding internal variable values that generate the partial example and using them to generate the remainder. At present, we have an interesting mathematical result that guarantees that a certain learning procedure will build internal representations which allow the connection strengths to capture the under-lying constraints that are implicit in a large ensemble of examples taken from a domain. We also have simulations which show that the theory works for some simple cases, but the current version of the learning algorithm is very slow. The search for general principles that allow parallel networks to learn the structure of their environment has often begun with the assumption that networks are randomly wired. This seemsto us to be just as wrong as the view that all knowledge is innate. If there are connectivity structures that are good for particular tasks that the network will have to perform, it is much more efficient to build these in at the start. However, not all tasks can be foreseen, and even for ones that can, fine-tuning may still be helpful. Another common belief is that a general connectionist learning rule would make sequential “rule-based” models unnecessary. We believe that this view stems from a misunderstanding of the need for multiple levels of description of large systems, which can be usefully viewed as either parallel or serial depending on the grain of the analysis. ,Most of the key issuesand questions that have been studied in the context of sequential models do not magically disappear in connectionist models. It is still necessaryto perform BOLTZMANN MACHINE LEARNING 149

searches for good solutions to problems or good interpretations of percep-tual input, and to create complex internal representations. Ultimately it will be necessary to bridge the gap between hardware-oriented connectionist descriptions and the more abstract symbol manipulation models that have proved to be an extremely powerful and pervasive way of describing human information processing (Newell & Simon, 1972). 2. THE BOLTZMANN MACHINE The Boltzmann Machine is a parallel computational organization that is well suited to constraint satisfaction tasks involving large numbers of “weak” constraints. Constraint-satisfaction searches (e.g., Waltz, 1975; Winston, 1984) normally use “strong” constraints that tnusl be satisfied by any solution. In problem domains such as gamesand puzzles, for example, the goal criteria often have this character, so strong constraints are the rule.’ In some problem domains, such as finding the most plausible interpretation of an image, many of the criteria are not all-or-none, and frequently even the best possible solution violates some constraints (Hinton, 1977). A varia-tion that is more appropriate for such domains usesweak constraints that incur a cost when violated. The quality of a solution is then determined by the total cost of all the constraints that it violates. In a perceptual interpre-tation task, for example, this total cost should reflect the implausibility of the interpretation. The machine is composed of primitive computing elementscalled unifs that are connected to each other by bidirectional links. A unit is always in one of two states, on or off, and it adopts these states as a probabilistic function of the states of its neighboring units and the weighfs on its links to them. The weights can take on real values of either sign. A unit being on or off is taken to mean that the system currently accepts or rejects some ele-mental hypothesis about the domain. The weight on a link representsa weak pairwise constraint between two hypotheses. A positive weight indicates that the two hypotheses tend to support one another; if one is currently ac-cepted, accepting the other should be more likely. Conversely, a negative weight suggests, other things being equal, that the two hypotheses should not both be accepted. Link weights are symmetric, having the samestrength in both directions (Hinton & Sejnowski, 1983).’

如果假设每个网络中都有一个永久活跃的“真实单元”，则方程(1)和(2)可写为：

If a permanently active “true unit” is assumed to be part of every network, then Eqs. (1) and (2) can be written as:

2.2 利用噪声逃离局部极小值

2.2 Using Noise to Escape from Local Minima

简单的确定性算法存在梯度下降法的标准弱点：它会陷入非全局最优的局部极小值。这在 Hopfield 系统中不是问题，因为他网络的局部能量极小值用于存储“项目”：如果系统从某个局部最小值附近开始，期望的行为是落入该最小值，而不是找到全局最小值。然而，对于约束满足任务，系统必须尝试逃离局部极小值，以找到当前输入下全局最小的配置。一个简单的离开局部极小值的方法是偶尔允许跳转到能量更高的构型。具有这种性质的算法由 Metropolis、Rosenbluth、Rosenbluth、Teller 和 Teller（1953）提出，用于研究热力学系统的平均性质（Binder，1978），最近已被应用于约束满足问题（Kirkpatrick、Gelatt 和 Vecchi，1983）。我们采用一种适合并行计算的 Metropolis 算法形式：如果第 k 个单元的开启和关闭状态之间的能量差为\( \Delta E \)，则无论先前状态如何，该单元以概率\( p = \frac{1}{1 + \exp(-\Delta E / T)} \)（5）设置其状态为 1，其中\( T \)是一个起温度作用的参数（见图 1）。

The simple, deterministic algorithm suffers from the standard weakness of gradient descent methods: It gets stuck in local minima that are not globally optimal. This is not a problem in Hopfield’s system because the local energy minima of his network are used to store “items”: If the system is started near some local minimum, the desired behavior is to fall into that minimum, not to find the global minimum. For constraint satisfaction tasks, however, the system must try to escape from local minima in order to find the configuration that is the global minimum given the current input. A simple way to get out of local minima is to occasionally allow jumps to configurations of higher energy. An algorithm with this property was introduced by Metropolis, Rosenbluth, Rosenbluth, Teller, & Teller (1953) to study average properties of thermodynamic systems (Binder, 1978) and has recently been applied to problems of constraint satisfaction (Kirkpatrick, Gelatt, & Vecchi, 1983). We adopt a form of the Metropolis algorithm that is suitable for parallel computation: If the energy gap between the on and off states of the k-th unit is \( \Delta E \), then the unit sets its state to 1 with probability \( p = \frac{1}{1 + \exp(-\Delta E / T)} \) (5) where \( T \) is a parameter that acts like temperature (see Figure 1).

式(5)中的决策规则与具有两个能量状态的粒子相同。与给定温度下的热浴接触的此类粒子系统最终将达到热平衡，并且找到系统处于任何全局状态的概率将服从玻尔兹曼分布。类似地，服从该决策规则的单元网络最终将达到“热平衡”，两个全局状态的相对概率将遵循玻尔兹曼分布：\( P_i = \frac{\exp(-E_i / T)}{\sum_j \exp(-E_j / T)} \) (6)，其中\( P_i \)是处于第\( i \)个全局状态的概率，\( E_i \)是该状态的能量。玻尔兹曼分布具有一些优美的数学性质，并且与信息论密切相关。特别地，两个全局状态的对数概率之差正好是它们的能量差（在温度 1 下）。这种关系的简单性以及平衡分布与到达平衡的路径无关的事实使得玻尔兹曼机引人入胜。在低温下，存在有利于低能量状态的强烈偏置，但达到平衡所需的时间可能很长。在较高温度下，偏置不那么有利，但平衡更快达到。克服这种权衡的一个好方法是从高温开始逐渐降低温度。这对应于物理系统的退火（Kirkpatrick、Gelatt 和 Vecchi，1983）。在高温下，网络将忽略小的能量差异并迅速趋近平衡。在此过程中，它将搜索全局状态空间的粗略整体结构，并在该粗略层次上找到一个好的最小值。随着温度降低，它开始响应更小的能量差异，并在高温发现的粗尺度最小值内找到一个更好的最小值。Kirkpatrick 等人已经证明，这种先搜索粗结构再搜索精细结构的方法对于图划分等组合问题非常有效，并且我们相信它对于试图满足多个弱约束也将是有用的，尽管在最优解对应于一个深、窄且孤立的最小值的情况下它会明显失效。3. 学习算法 玻尔兹曼机公式中最有趣的方面或许是它产生了一种与领域无关的学习算法，该算法修改玻尔兹曼机学习。

The decision rule in Eq. (5) is the same as that for a particle which has two energy states. A system of such particles in contact with a heat bath at a given temperature will eventually reach thermal equilibrium and the probability of finding the system in any global state will then obey a Boltzmann distribution. Similarly, a network of units obeying this decision rule will eventually reach “thermal equilibrium” and the relative probability of two global states will follow the Boltzmann distribution: \( P_i = \frac{\exp(-E_i / T)}{\sum_j \exp(-E_j / T)} \) (6) where \( P_i \) is the probability of being in the \( i \)-th global state, and \( E_i \) is the energy of that state. The Boltzmann distribution has some beautiful mathematical properties and it is intimately related to information theory. In particular, the difference in the log probabilities of two global states is just their energy difference (at a temperature of 1). The simplicity of this relationship and the fact that the equilibrium distribution is independent of the path followed in reaching equilibrium are what make Boltzmann machines interesting. At low temperatures there is a strong bias in favor of states with low energy, but the time required to reach equilibrium may be long. At higher temperatures the bias is not so favorable but equilibrium is reached faster. A good way to beat this trade-off is to start at a high temperature and gradually reduce it. This corresponds to annealing a physical system (Kirkpatrick, Gelatt, & Vecchi, 1983). At high temperatures, the network will ignore small energy differences and will rapidly approach equilibrium. In doing so, it will perform a search of the coarse overall structure of the space of global states, and will find a good minimum at that coarse level. As the temperature is lowered, it will begin to respond to smaller energy differences and will find one of the better minima within the coarse-scale minimum it discovered at high temperature. Kirkpatrick et al. have shown that this way of searching the coarse structure before the fine is very effective for combinatorial problems like graph partitioning, and we believe it will also prove useful when trying to satisfy multiple weak constraints, even though it will clearly fail in cases where the best solution corresponds to a minimum that is deep, narrow, and isolated. 3. A LEARNING ALGORITHM Perhaps the most interesting aspect of the Boltzmann Machine formulation is that it leads to a domain-independent learning algorithm that modifies the Boltzmann machine learning.

单元之间的连接强度以这样的方式修改，使得整个网络发展出一个内部模型，该模型捕捉其环境的底层结构。寻找此类算法的历史长期失败（Newell, 1982），许多人（尤其是人工智能领域）现在认为不存在这样的算法。阻止简单学习算法推广到更复杂网络的主要技术障碍是：要能够进行有趣的计算，网络必须包含不受输入直接约束的非线性元素，而当这样的网络做出错误行为时，似乎不可能确定众多连接强度中哪一个有错。这个“信用分配”问题导致了感知机的衰落（Minsky & Papert, 1968; Rosenblatt, 1961）。感知机收敛定理保证了单层决策单元的权重可以被训练，但当任务没有直接指定如何使用网络中的所有单元时，它无法推广到此类单元的网络。在玻尔兹曼机公式中可以解决这个版本的信用分配问题。通过使用正确的随机决策规则，并在有限温度下运行网络直至达到“热平衡”，我们得到了全局状态概率与其能量之间数学上简单的关系。对于没有环境输入而自由运行的网络，这种关系由式(6)给出。由于能量是权重的线性函数（式(1)），这导致全局状态的对数概率与单个连接强度之间存在非常简单的关联：

connection strengths between units in such a way that the whole network develops an internal model which captures the underlying structure of its environment. There has been a long history of failure in the search for such algorithms (Newell, 1982), and many people (particularly in Artificial Intelligence) now believe that no such algorithms exist. The major technical stumbling block which prevented the generalization of simple learning algorithms to more complex networks was this: To be capable of interesting computations, a network must contain nonlinear elements that are not directly constrained by the input, and when such a network does the wrong thing it appears to be impossible to decide which of the many connection strengths is at fault. This “credit-assignment” problem was what led to the demise of perceptrons (Minsky & Papert, 1968; Rosenblatt, 1961). The perceptron convergence theorem guarantees that the weights of a single layer of decision units can be trained, but it could not be generalized to networks of such units when the task did not directly specify how to use all the units in the network. This version of the credit-assignment problem can be solved within the Boltzmann Machine formulation. By using the right stochastic decision rule, and by running the network until it reaches “thermal equilibrium” at some finite temperature, we achieve a mathematically simple relationship between the probability of a global state and its energy. For a network that is running freely without any input from the environment, this relationship is given by Eq. (6). Because the energy is a linear function of the weights (Eq. 1) this leads to a remarkably simple relationship between the log probabilities of global states and the individual connection strengths:

其中 \( s_i^\alpha \) 是全局状态 \( \alpha \) 中第 \( i \) 个单元的状态（因此 \( s_i^\alpha s_j^\alpha \) 仅在单元 \( i \) 和 \( j \) 在状态 \( \alpha \) 中同时开启时为 1），而 \( p_{ij} \) 是系统处于平衡态时同时发现两个单元 \( i \) 和 \( j \) 开启的概率。给定式 (7)，可以操纵全局状态的对数概率。如果环境直接指定每个全局状态 \( \alpha \) 所需的概率 \( P_\alpha \)，那么存在一种直接的方法收敛到一组实现这些概率的权重，前提是存在任何这样的集合（详见 Hinton & Sejnowski, 1983a）。然而，这并不是一种特别有趣的学习方式，因为系统必须被给定完整全局状态所需的概率。这意味着内部表示应该使用什么这一核心问题已经由环境决定。有趣的问题出现在环境隐式包含高阶约束，并且网络必须选择允许这些约束有效表达的内部表示时。154 ACKLEY, HINTON, AND SEJNOWSKI

where \( s_i^\alpha \) is the state of the \( i \)-th unit in the global state \( \alpha \) (so \( s_i^\alpha s_j^\alpha \) is 1 only if units \( i \) and \( j \) are both on in state \( \alpha \)), and \( p_{ij} \) is just the probability of finding the two units \( i \) and \( j \) on at the same time when the system is at equilibrium. Given Eq. (7), it is possible to manipulate the log probabilities of global states. If the environment directly specifies the required probabilities \( P_\alpha \) for each global state \( \alpha \), there is a straightforward way of converging on a set of weights that achieve those probabilities, provided any such set exists (for details, see Hinton & Sejnowski, 1983a). However, this is not a particularly interesting kind of learning because the system has to be given the required probabilities of complete global states. This means that the central question of what internal representation should be used has already been decided by the environment. The interesting problem arises when the environment implicitly contains high-order constraints and the network must choose internal representations that allow these constraints to be expressed efficiently. 154 ACKLEY, HINTON, AND SEJNOWSKI

玻尔兹曼机的单元分为两个功能组：非空的可见单元集和可能为空隐藏单元集。可见单元是网络与环境之间的接口；训练期间，所有可见单元被环境钳位到特定状态；测试补全能力时，可见单元的任何子集可以被钳位。隐藏单元（如果有）永远不会被环境钳位，可用于“解释”输入向量整体中无法由可见单元间的成对约束表示的潜在约束。例如，如果环境要求三个可见单元的状态具有偶校验——这一规律无法仅通过成对相互作用实现，则需要一个隐藏单元。利用隐藏单元表示关于可见单元状态的更复杂假设，可见单元之间的此类高阶约束可以简化为整个单元集的一阶和二阶约束。我们假设每个环境输入向量持续足够长的时间，以使网络能够接近热平衡，并且我们忽略环境向量序列中可能存在的任何结构。然后，可以通过给出所有 \( 2^v \) 个可见单元状态的概率分布来指定环境的结构。如果在所有单元都未钳位（即无环境输入）的情况下，网络自由运行于热平衡时，在这些 \( 2^v \) 个状态上实现完全相同的概率分布，则称该网络具有环境的完美模型。除非隐藏单元的数量相对于可见单元数量呈指数级增长，否则不可能实现完美模型，因为即使网络全连接，\( v \) 个可见单元和 \( h \) 个隐藏单元之间的 \( (v+h-1)(v+h)/2 \) 个权重和 \( (v+h) \) 个偏置也不足以对环境指定的 \( 2^v \) 个可见单元状态的概率建模。然而，如果环境中存在规律性，并且网络利用其隐藏单元捕获这些规律性，则它可能与环境概率实现良好匹配。网络内部模型与环境之间差异的信息论度量是

The units of a Boltzmann Machine partition into two functional groups, a nonempty set of visible units and a possibly empty set of hidden units. The visible units are the interface between the network and the environment; during training all the visible units are clamped into specific states by the environment; when testing for completion ability, any subset of the visible units may be clamped. The hidden units, if any, are never clamped by the environment and can be used to “explain” underlying constraints in the ensemble of input vectors that cannot be represented by pairwise constraints among the visible units. A hidden unit would be needed, for example, if the environment demanded that the states of three visible units should have even parity—a regularity that cannot be enforced by pairwise interactions alone. Using hidden units to represent more complex hypotheses about the states of the visible units, such higher-order constraints among the visible units can be reduced to first and second-order constraints among the whole set of units. We assume that each of the environmental input vectors persists for long enough to allow the network to approach thermal equilibrium, and we ignore any structure that may exist in the sequence of environmental vectors. The structure of an environment can then be specified by giving the probability distribution over all \( 2^v \) states of the \( v \) visible units. The network will be said to have a perfect model of the environment if it achieves exactly the same probability distribution over these \( 2^v \) states when it is running freely at thermal equilibrium with all units unclamped so there is no environmental input. Unless the number of hidden units is exponentially large compared to the number of visible units, it will be impossible to achieve a perfect model because even if the network is totally connected the \( (v+h-1)(v+h)/2 \) weights and \( (v+h) \) biases among the \( v \) visible and \( h \) hidden units will be insufficient to model the \( 2^v \) probabilities of the states of the visible units specified by the environment. However, if there are regularities in the environment, and if the network uses its hidden units to capture these regularities, it may achieve a good match to the environmental probabilities. An information-theoretic measure of the discrepancy between the network’s internal model and the environment is

\[ G = \sum_{\alpha} P(V_\alpha) \log \frac{P(V_\alpha)}{P'(V_\alpha)} \] 其中 \( P(V_\alpha) \) 是可见单元状态由环境决定时第 \( \alpha \) 个状态的概率，\( P'(V_\alpha) \) 是网络自由运行且无环境输入时的相应概率。\( G \) 度量，有时称为非对称散度或信息增益（Kullback, 1959; Renyi, 1962），是衡量从 \( P'(V_\alpha) \) 分布到 \( P(V_\alpha) \) 分布距离的指标。

\[ G = \sum_{\alpha} P(V_\alpha) \log \frac{P(V_\alpha)}{P'(V_\alpha)} \] where \( P(V_\alpha) \) is the probability of the \( \alpha \)-th state of the visible units when their states are determined by the environment, and \( P'(V_\alpha) \) is the corresponding probability when the network is running freely with no environmental input. The \( G \) metric, sometimes called the asymmetric divergence or information gain (Kullback, 1959; Renyi, 1962), is a measure of the distance from the distribution given by the \( P'(V_\alpha) \) to the distribution given by the \( P(V_\alpha) \).

当且仅当分布相同时 G 为零；否则为正。项 \( P'(V_\alpha) \) 依赖于权重，因此可以通过改变权重来改变 G。要在 G 上执行梯度下降，需要知道 G 对每个单独权重的偏导数。在大多数交叉耦合非线性网络中，推导这一量非常困难，但由于热平衡时存在的简单关系，对于我们的网络，G 的偏导数推导起来很简单。全局状态的概率由其能量（式 6）决定，而能量由权重（式 1）决定。利用这些方程，G 的偏导数（见附录）为：

G is zero if and only if the distributions are identical; otherwise it is positive. The term \( P'(V_\alpha) \) depends on the weights, and so G can be altered by changing them. To perform gradient descent in G, it is necessary to know the partial derivative of G with respect to each individual weight. In most cross-coupled nonlinear networks it is very hard to derive this quantity, but because of the simple relationships that hold at thermal equilibrium, the partial derivative of G is straightforward to derive for our networks. The probabilities of global states are determined by their energies (Eq. 6) and the energies are determined by the weights (Eq. 1). Using these equations the partial derivative of G (see the appendix) is:

where s: is the state of the ifh unit in the & global state (so .c’:s;’ is 1 only if units i andj are both on in state CY), and p,; is just the probability of finding the two units i andj on at the same time when the system is at equilibrium. Given Eq. (7). it is possible to manipulate the log probabilities of global states. If the environment directly specifies the required probabilities P, for each global state (Y, there is a straightforward way of converging on a set of weights that achieve those probabilities, provided any such set exists (for details, see Hinton & Sejnowski, 1983a). However, this is not a particularly interesting kind of learning because the system has to be given the required probabilities of co/up/e/e global states. This means that the central question of what internal representation should be used has already been decided by the environment. The interesting problem arises when the environment im-plicitly contains high-order constraints and the network must choose inter-nal representations that allow these constraints to be expressed efficiently. 154 ACKLEY. HINTON, AND SEJNOWSKI

其中 p_{ij} 是当环境钳制可见单元状态时，两个单元同时处于开启状态的平均概率；而 p_{ij} 如公式 (7) 所示，是当环境输入不存在且网络自由运行时对应的概率。（这两个概率都必须在平衡状态下测量。）注意该方程与公式 (7) 的相似性，后者展示了改变权重如何影响单个状态的对数概率。因此，要最小化 G，只需在网络处于热平衡时观测 p_{ij} 和 p_{ij}，并根据这两个概率之差按比例改变每个权重：

where p_{ij} is the average probability of two units both being in the on state when the environment is clamping the states of the visible units, and p_{ij}, as in Eq. (7), is the corresponding probability when the environmental input is not present and the network is running freely. (Both these probabilities must be measured at equilibrium.) Note the similarity between this equation and Eq. (7), which shows how changing a weight affects the log probability of a single state. To minimize G, it is therefore sufficient to observe p_{ij} and p_{ij} when the network is at thermal equilibrium, and to change each weight by an amount proportional to the difference between these two probabilities:

其中 e 缩放每次权重变化的大小。该规则的一个令人惊讶的特点是它仅使用局部可用的信息。权重的改变仅取决于它所连接的两个单元的行为，尽管这种改变优化了一个全局度量，并且每个权重的最佳值取决于所有其他权重的值。如果没有隐藏单元，可以证明 G 空间是凹的（从上方看），因此简单的梯度下降不会陷入糟糕的局部最小值。然而，当存在隐藏单元时，可能会出现局部最小值，这些最小值对应于使用隐藏单元表示环境向量概率分布中隐含的高阶约束的不同方式。下一节将讨论处理这些更复杂 G 空间的一些技术。一旦 G 被最小化，网络将尽可能捕捉环境中的规律性，并且在执行补全时将强制这些规律性。另一种观点是，网络在最小化 G 时，是在寻找最有可能生成环境向量集合的权重集。可以证明，最大化这种似然在数学上等价于最小化 G（Peter Brown，个人通信，1983）。

where e scales the size of each weight change. A surprising feature of this rule is that it uses only locally available information. The change in a weight depends only on the behavior of the two units it connects, even though the change optimizes a global measure, and the best value for each weight depends on the values of all the other weights. If there are no hidden units, it can be shown that G-space is concave (when viewed from above) so that simple gradient descent will not get trapped at poor local minima. With hidden units, however, there can be local minima that correspond to different ways of using the hidden units to represent the higher-order constraints that are implicit in the probability distribution of environmental vectors. Some techniques for handling these more complex G-spaces are discussed in the next section. Once G has been minimized the network will have captured as well as possible the regularities in the environment, and these regularities will be enforced when performing completion. An alternative view is that the net-156 ACKLEY, HINTON, AND SEJNOWSKI

网络在最小化 G 时，是在寻找最有可能生成环境向量集合的权重集。可以证明，最大化这种似然在数学上等价于最小化 G（Peter Brown，个人通信，1983）。

work, in minimizing G, is finding the set of weights that is most likely to have generated the set of environmental vectors. It can be shown that maximizing this likelihood is mathematically equivalent to minimizing G (Peter Brown, personal communication, 1983).

上述学习算法中有许多自由参数和可能的变体。除了决定梯度下降每一步大小 e 的值之外，估计 p_{ij} 和 p_{ij} 的时间长度对学习过程也有显著影响。本文仿真中使用的值主要是基于经验观察选择的。一个估计 p_{ij} 和 p_{ij} 的实际系统必然会在估计中引入一些噪声，导致 G 值偶尔出现“上坡步”。由于网络中的隐藏单元会在 G 中产生局部最小值，这不一定是不利的。如果需要，可以通过使用较小的 e 值或更长时间地收集统计数据来减少估计中噪声的影响，因此实现 G 最小化的退火搜索相对容易。目标函数 G 是一个度量，用于指定两个概率分布匹配的程度。如果环境指定可见单元上可能出现的模式中只有一小部分实际出现，则会出现问题。默认情况下，未提到的模式必须以概率零出现，而在非零温度下运行的玻尔兹曼机保证某些配置永不出现的唯一方法是赋予这些配置无限高的能量，这需要无限大的权重。避免这种隐含的无限权重需求的一种方法是偶尔提供“嘈杂的”输入向量。这可以通过将“正确”的输入向量通过一个以小概率反转每个比特的过程进行滤波来实现。这些噪声向量随后被钳制在可见单元上。如果噪声很小，正确的向量将主导统计，但每个向量都有出现的可能性，因此不需要无限能量。本文展示的所有示例都使用了这种“噪声钳制”技术。它效果很好，但我们并不完全满意，并且正在研究其他方法，以防止当只有少数可能的输入向量出现时权重变得过大。下一节中的仿真采用了方程 (10) 所暗示的明显最速下降法的修改版本。我们不直接按差值 {ij} - p_{ij} 的比例改变每个权重，而是加入了一个动量项：每一步的权重变化是前一次变化的一部分与当前梯度的和。这平滑了轨迹，允许更大的步长，加快了收敛速度。此外，我们发现，在负相期间对称地钳制可见单元（即使用与正相相同的钳制状态）降低了梯度估计的方差。这些修改是随机优化中的标准做法，对于在具有隐藏单元的网络中获得稳定的学习至关重要。

There are a number of free parameters and possible variations in the learning algorithm presented above. As well as the size of e, which determines the size of each step taken for gradient descent, the lengths of time over which p_{ij} and p_{ij} are estimated have a significant impact on the learning process. The values employed for the simulations presented here were selected primarily on the basis of empirical observations. A practical system which estimates p_{ij} and p_{ij} will necessarily have some noise in the estimates, leading to occasional "uphill steps" in the value of G. Since hidden units in a network can create local minima in G, this is not necessarily a liability. The effect of the noise in the estimates can be reduced, if desired, by using a small value for e or by collecting statistics for a longer time, and so it is relatively easy to implement an annealing search for the minimum of G. The objective function G is a metric that specifies how well two probability distributions match. Problems arise if an environment specifies that only a small subset of the possible patterns over the visible units ever occur. By default, the unmentioned patterns must occur with probability zero, and the only way a Boltzmann Machine running at a non-zero temperature can guarantee that certain configurations never occur is to give those configurations infinitely high energy, which requires infinitely large weights. One way to avoid this implicit demand for infinite weights is to occasionally provide "noisy" input vectors. This can be done by filtering the "correct" input vectors through a process that has a small probability of reversing each of the bits. These noisy vectors are then clamped on the visible units. If the noise is small, the correct vectors will dominate the statistics, but every vector will have some chance of occurring and so infinite energies will not be needed. This "noisy clamping" technique was used for all the examples presented here. It works quite well, but we are not entirely satisfied with it and have been investigating other methods of preventing the weights from growing too large when only a few of the possible input vectors ever occur. The simulations presented in the next section employed a modification of the obvious steepest descent method implied by Eq. (10). Instead of chang-

而是使用动量项：每一步的权重变化是前一次变化的一部分与当前梯度的和。这平滑了轨迹，允许更大的步长，加快了收敛速度。此外，我们发现，在负相期间对称地钳制可见单元（即使用与正相相同的钳制状态）降低了梯度估计的方差。这些修改是随机优化中的标准做法，对于在具有隐藏单元的网络中获得稳定的学习至关重要。

ing each weight by an amount proportional to the difference {ij} - p_{ij}, we used a momentum term: the change in a weight at each step is the sum of a fraction of the previous change and the current gradient. This smoothed the trajectory and allowed a larger step size, speeding up convergence. Additionally, we found that symmetrically clamping the visible units during the negative phase (i.e., using the same clamped states as in the positive phase) reduced the variance of the gradient estimates. These modifications are standard in stochastic optimization and were crucial for obtaining stable learning in networks with hidden units.

如果 pij > pi，则数值按与 pij \- pi 成正比的量增加，否则按固定的“权重步长”递增；若 pij < pi，则按相同的量递减。

by an amount proportional to pij - pi; it is simply incremented by a fixed "weight-step" if pv > pi, and decremented by the same amount if pij < pi.

该方法相比最陡下降法的优势在于，它能应对 G 的一阶和二阶导数的巨大变化。它能在 G 平缓变化的维度上取得显著进展，同时避免在 G 快速下降后又快速上升的维度上产生大幅发散步长。在这种情况下，方程 (10) 中的 E 没有合适的取值。任何大到足以沿沟壑缓坡前进的值，都会导致沿沟壑陡峭侧壁上下发散振荡。

The advantage of this method over steepest descent is that it can cope Boltzmann Machine Learning 157 with wide variations in the first and second derivatives of G. It can make significant progress on dimensions where G changes gently without taking very large divergent steps on dimensions where G falls rapidly and then rises rapidly again. There is no suitable value for the E in Eq. (10) in such cases. Any value large enough to allow progress along the gently sloping floor of a ravine will cause divergent oscillations up and down the steep sides of the ravine.

"编码器问题"（由 Sanjaya Addanki 建议）是并行网络各组件间通信这一重复性任务的简单抽象。我们用该问题测试学习算法，因为其最优解明确且发现过程并不平凡。两组可见单元（记为 V1 和 V2）代表希望通信其状态的两个系统。每组有 v 个单元。在我们考虑的简单公式中，每组每次只有一个单元开启，因此每组只有 v 个不同状态。V1 和 V2 不直接相连，但都连接到一组 h 个隐藏单元 H，且 h < v，因此 H 可作为有限容量瓶颈，V1 和 V2 的状态信息必须通过该瓶颈压缩。由于所有仿真均从所有权重为零开始，求解该问题要求两组可见单元在没有任何先验通信约定的情况下，就一组编码的含义达成一致。为实现可见单元间的完美通信，必须有 h ≥ log2 v。我们研究了 h = log2 v 的最小情况，

The "encoder problem" (suggested to us by Sanjaya Addanki) is a simple abstraction of the recurring task of communicating information among various components of a parallel network. We have used this problem to test out the learning algorithm because it is clear what the optimal solution is like and it is nontrivial to discover it. Two groups of visible units, designated V1 and V2, represent two systems that wish to communicate their states. Each group has v units. In the simple formulation we consider here, each group has only one unit on at a time, so there are only v different states of each group. V1 and V2 are not connected directly but both are connected to a group of h hidden units H, with h < v so H may act as a limited capacity bottleneck through which information about the states of V1 and V2 must be squeezed. Since all simulations began with all weights set to zero, finding a solution to such a problem requires that the two visible groups come to agree upon the meanings of a set of codes without any a priori conventions for communication through H. To permit perfect communication between the visible groups, it must be the case that h ≥ log2 v. We investigated minimal cases in which h = log2 v,

以及 h 略大于 log2 v 的情况。在所有情况下，网络的环境由 v 个等概率的 2v 长度向量组成，这些向量指定 V1 中的一个单元和 V2 中对应的单元应同时开启，而所有其他单元关闭。每个可见组内部完全连接，每个可见组与 H 完全连接，但 H 中的单元彼此不连接。由于串行机器仿真的速度严重受限，且学习需要多次退火，我们主要对编码器问题的小规模版本进行了实验。例如，图 2 展示了一个 "4-2-4" 编码器问题的良好解。

and cases when h was somewhat larger than log2 v. In all cases, the environment for the network consisted of v equiprobable vectors of length 2v which specified that one unit in V1 and the corresponding unit in V2 should be on together with all other units off. Each visible group is completely connected internally and each is completely connected to H, but the units in H are not connected to each other. Because of the severe speed limitation of simulation on a sequential machine, and because the learning requires many annealings, we have primarily experimented with small versions of the encoder problem. For example, Figure 2 shows a good solution to a "4-2-4" encoder problem in

w,, by an amount proportional to pij -pi;, it is simply incremented by a fixed “weight-step” if pv>pi; and decremented by the same amount if pijc

阿克雷、辛顿和塞诺斯基图 2：编码器问题的解。连接权重使用递归符号表示。每个单元由一个阴影 L 形框表示；从上到下，框的行分别表示组 V₁、H 和 V₂。每个阴影框是整个网络的地图，显示该单元与其他单元连接的强度。在框的每个位置，白色（正）或黑色（负）矩形的大小表示权重的大小。在对应于单元与自身连接的位置（例如，顶部行第二个单元顶部行的第二个位置），显示偏置。所有单元之间的连接在图中出现两次，一次在每个被连接的两个单元的框中。例如，V₁最左侧单元右上角的黑色方块表示与 V₂最右侧单元左上角的黑色方块相同的连接。该连接的权重为-30，其中 v = 4，h = 2。可见组与 H 之间的互连已发展出二进制编码——每个可见单元在 H 的单元中引起不同的开/关状态模式，而 V₁和 V₂中对应的单元支持 H 中相同的模式。注意 V₁和 V₂中第二个单元的偏置为正，以补偿表示该单元的编码将所有 H 单元关闭的事实。4.1 4-2-4 编码器 对 v = 4 和 h = 2 的网络实验采用以下学习周期：1. 估计 p_{ij}：每个环境向量依次固定在可见单元上。对于每个环境向量，网络允许达到平衡两次。在平衡时收集关于单元对同时开启的频率的统计。为防止权重增长过大，我们使用了第 3.2 节所述的“噪声”固定技术。每个固定向量的开启位以 0.15 的概率被关闭，每个关闭位以 0.05 的概率被开启。2. 估计 p_{ij}'：网络完全解除固定，允许在温度为 10 时达到平衡。统计关于共现的数据，然后用于与估计 p_{ij}相同次数的退火。玻尔兹曼机学习 159

ACKLEY, HINTON, AND SEJNOWSKI Figure 2. A solution to an encoder problem. The link weights are displayed using a recursive notation. Each unit is represented by a shaded L-shaped box; from top to bottom the rows of boxes represent groups V₁, H, and V₂. Each shaded box is a map of the entire network, showing the strengths of that unit’s connections to other units. At each position in a box, the size of the white (positive) or black (negative) rectangle indicates the magnitude of the weight. In the position that would correspond to a unit connecting to itself (the second position in the top row of the second unit in the top row, for example), the bias is displayed. All connections between units appear twice in the diagram, once in the box for each of the two units being connected. For example, the black square in the top right corner of the left-most unit of V₁ represents the same connection as the black square in the top left corner of the rightmost unit of V₂. This connection has a weight of -30, which for v = 4 and h = 2. The interconnections between the visible groups and H have developed a binary coding—each visible unit causes a different pattern of on and off states in the units of H, and corresponding units in V₁ and V₂ support identical patterns in H. Note how the bias of the second unit of V₁ and V₂ is positive to compensate for the fact that the code which represents that unit has all the H units turned off. 4.1. The 4-2-4 Encoder The experiments on networks with v = 4 and h = 2 were performed using the following learning cycle: 1. Estimation of p_{ij}: Each environmental vector in turn was clamped over the visible units. For each environmental vector, the network was allowed to reach equilibrium twice. Statistics about how often pairs of units were both on together were gathered at equilibrium. To prevent the weights from growing too large we used the “noisy” clamping technique described in Section 3.2. Each on bit of a clamped vector was set to off with a probability of 0.15 and each off bit was set to on with a probability of 0.05. 2. Estimation of p_{ij}': The network was completely unclamped and allowed to reach equilibrium at a temperature of 10. Statistics about BOLTZMANN MACHINE LEARNING 159

共现数据随后用于与估计 p_{ij}相同次数的退火。

co-occurrences were then gathered for as many annealings as were used to estimate p_{ij}.

3. 更新权重：网络中所有权重以固定步长 2 递增或递减，增量的符号由 p_{ij} - p_{ij}'的符号决定。

3. Updating the weights: All weights in the network were incremented or decremented by a fixed weight-step of 2, with the sign of the increment being determined by the sign of p_{ij} - p_{ij}'.

当需要稳定到平衡时，所有未固定单元以相等概率随机置为开或关（对应于将温度升至无穷大），然后网络在以下温度下运行以下时间：[2@20, 2@15, 2@12, 4@10]。在此退火调度之后，假定网络已达到平衡，并在温度为 10 时收集 10 个单位时间的统计。我们观察到寻找 G 的全局最小值过程中有三个主要阶段，并发现这些阶段的发生对所使用参数的精确值相对不敏感。第一阶段从所有权重设为零开始，其特征是整个网络大部分权重发展为负，实现了两个赢家通吃网络，模拟环境结构的最简单方面——每个可见组中通常一次只有一个单元活动。例如，在 4-2-4 编码器中，可见单元上的可能模式数量为 28。通过在每组四个单元中实现赢家通吃网络，这可以减少到 4×4 个低能量模式。只有从 2⁸到 2⁴低能量模式的最后减少需要隐藏单元用于在两个可见组之间通信。图 3a 显示了经过四个学习周期后的 4-2-4 编码器网络。尽管在第二阶段中隐藏单元被用于抑制，但侧向抑制任务可以仅由可见组内的连接处理。在第二阶段，隐藏单元开始对可见组中的某些单元发展正权重，并且倾向于保持连接到 V₁中单元与连接到 V₂中对应单元的符号和近似大小的对称性。当每个隐藏单元对 V₁中每个单元具有显著连接权重，并且对 V₂中每个单元具有类似权重，且大多数不同编码已被使用时，第二阶段结束，但有些编码被多次使用，有些则未被使用。图 3b 显示了经过 60 个学习周期后的相同网络。偶尔，第二阶段结束时所有编码都被使用，此时问题已解决。但通常存在第三阶段，也是最长的阶段，学习算法解决剩余冲突并找到全局最小值。解决过程涉及两种基本机制。考虑图 3b 中第一个和第四个单元之间的冲突，它们都采用编码⟨–, +⟩。当系统没有环境输入运行时，这两个单元会频繁同时开启。因此，p_{ij}将高于 p_{ij}'，因为环境输入倾向于阻止这两个单元同时开启。因此，学习算法不断减小每组中第一个和第四个单元之间连接的权重，它们相互强烈抑制。（这个效应解释了图 2 中抑制权重的变化。具有相似编码的可见单元会相互强烈抑制。）可见单元因此在可能编码空间中竞争“领地”，这种排斥效应导致编码远离相似邻居。除了排斥效应，我们还观察到另一过程，它倾向于最终将未使用的编码（按汉明距离）置于涉及冲突的编码附近。该过程的机制有些微妙，我们在此不赘述。第三阶段结束于所有编码都被使用时，然后权重倾向于增加，使得解锁定并保持稳定，抵抗由共现统计随机变化引起的波动。（图 2 是经过 120 个学习周期后图 3 所示的同一网络。）在 250 次不同的 4-2-4 编码器测试中，它总是找到一个全局最小值，并且一旦到达就保持在那里。发现四个不同编码所需的中位时间为 110 个学习周期。最长时间为 1810 个学习周期。4.2 4-3-4 编码器 二进制编码器问题的一个变体是给 H 分配比编码 V₁和 V₂中模式绝对必要的单元更多的单元。一个简单的例子是 4-3-4 编码器，它使用与 4-2-4 编码器相同的参数运行。在这种情况下，学习算法快速找到四个不同编码。然后它总是继续修改编码，使其最优间隔，且没有一对仅相差一个比特，如图 4 所示。找到四个良好间隔编码的中位时间为 270 个学习周期，200 次试验中的最大时间为 1090。4.3 8-3-8 编码器 对于 v = 8 和 h = 3，找到所有 8 个三位编码需要更多学习周期。我们进行了 20 次模拟，每次运行 4000 个学习周期，使用与 4-2-4 情况相同的参数（但在噪声固定期间每个可见单元以 0.02 的概率翻转）。算法在 20 次模拟中的 16 次中找到了所有 8 个编码，其余找到了 7 个编码。找到 7 个编码的中位时间为 210 个学习周期，找到全部 8 个的中位时间为 1570 个周期。找到所有 8 个编码的困难并不令人惊讶，因为被视为解的权重空间比例远小于 4-2-4 情况。使用 8 个不同编码中 7 个的权重集相当常见，而包含全部 8 个的解则较少发现。

When a settling to equilibrium was required, all the unclamped units were randomized with equal probability on or off (corresponding to raising the temperature to infinity), and then the network was allowed to run for the following times at the following temperatures: [2@20, 2@15, 2@12, 4@10]. After this annealing schedule it was assumed that the network had reached equilibrium, and statistics were collected at a temperature of 10 for 10 units of time. We observed three main phases in the search for the global minimum of G, and found that the occurrence of these phases was relatively insensitive to the precise parameters used. The first phase begins with all the weights set to zero, and is characterized by the development of negative weights throughout most of the network, implementing two winner-take-all networks that model the simplest aspect of the environmental structure—only one unit in each visible group is normally active at a time. In a 4-2-4 encoder, for example, the number of possible patterns over the visible units is 28. By implementing a winner-take-all network among each group of four this can be reduced to 4 × 4 low energy patterns. Only the final reduction from 2⁸ to 2⁴ low energy patterns requires the hidden units to be used for communicating between the two visible groups. Figure 3a shows a 4-2-4 encoder network after four learning cycles. Although the hidden units are exploited for inhibition in the first phase, the lateral inhibition task can be handled by the connections within the visible groups alone. In the second phase, the hidden units begin to develop positive weights to some of the units in the visible groups, and they tend to maintain symmetry between the sign and approximate magnitude of a connection to a unit in V₁ and the corresponding unit in V₂. The second phase finishes when every hidden unit has significant connection weights to each unit in V₁ and analogous weights to each unit in V₂, and most of the different codes are being used, but there are some codes that are used more than once and some not at all. Figure 3b shows the same network after 60 learning cycles. Occasionally, all the codes are being used at the end of the second phase in which case the problem is solved. Usually, however, there is a third and longest phase during which the learning algorithm sorts out the remaining conflicts and finds a global minimum. There are two basic mechanisms involved in the sorting out process. Consider the conflict between the first and fourth units in Figure 3b, which are both employing the code ⟨–, +⟩. When the system is running without environmental input, the two units will be on together quite frequently. Consequently, p_{ij} will be higher than p_{ij}' because the environmental input tends to prevent the two units from being on together. Hence, the learning algorithm keeps decreasing the weight of the connection between the first and fourth units in each group, and they come to inhibit each other strongly. (This effect explains the variations in inhibitory weights in Figure 2. Visible units with similar codes are the ones that inhibit each other strongly.) Visible units thus compete for “territory” in the space of possible codes, and this repulsion effect causes codes to migrate away from similar neighbors. In addition to the repulsion effect, we observed another process that tends to eventually bring the unused codes adjacent (in terms of Hamming distance) to codes that are involved in a conflict. The mechanics of this process are somewhat subtle and we do not take the time to expand on them here. The third phase finishes when all the codes are being used, and the weights then tend to increase so that the solution locks in and remains stable against the fluctuations caused by random variations in the co-occurrence statistics. (Figure 2 is the same network shown in Figure 3, after 120 learning cycles.) In 250 different tests of the 4-2-4 encoder, it always found one of the global minima, and once there it remained there. The median time required to discover four different codes was 110 learning cycles. The longest time was 1810 learning cycles. 4.2. The 4-3-4 Encoder A variation on the binary encoder problem is to give H more units than are absolutely necessary for encoding the patterns in V₁ and V₂. A simple example is the 4-3-4 encoder which was run with the same parameters as the 4-2-4 encoder. In this case the learning algorithm quickly finds four different codes. Then it always goes on to modify the codes so that they are optimally spaced out and no pair differ by only a single bit, as shown in Figure 4. The median time to find four well-spaced codes was 270 learning cycles and the maximum time in 200 trials was 1090. 4.3. The 8-3-8 Encoder With v = 8 and h = 3 it took many more learning cycles to find all 8 three-bit codes. We did 20 simulations, running each for 4000 learning cycles using the same parameters as for the 4-2-4 case (but with a probability of 0.02 of reversing each of the visible units during noisy clamping). The algorithm found all 8 codes in 16 out of 20 simulations and found 7 codes in the rest. The median time to find 7 codes was 210 learning cycles and the median time to find all 8 was 1570 cycles. The difficulty of finding all 8 codes is not surprising since the fraction of the weight space that counts as a solution is much smaller than in the 4-2-4 case. Sets of weights that use 7 of the 8 different codes are found fairly often, and a solution with all 8 is found less frequently. 162 ACKLEY, HINTON. AND SEJNOWSKI

158 ACKLEY, HINTON. AND SEJNOWSKI Figure 2. A solution to an encoder problem. The link weights are displayed using a recur-sive notation. Each unit is represented by a shaded l-shaped box; from top to bottom the rows of boxes represent groups V,, H. and V,. Each shaded box is o mop of the entire net-work, showing the strengths of that unit’s connections to other units. At each position in obox, the size of the white (positive) or block (negative) rectangle indicates the magnitude of the weight. In the position that would correspond to o unit connecting to itself (the second position in the top row of the second unit in the top row, for example). the bias is displayed. All connections between units appear twice in the diagram, once in the box for each of the two units being connected. For example, the black square in the top right corner of the left-most unit of V, represents the same connection OS the block square in the top left corner of the rightmost unit of V,. This connection has a weight of -30. which v = 4 and h = 2. The interconnections between the visible groups and H have developed a binary coding-each visible unit causes a different pat-tern of on and off states in the units of If, and corresponding units in V, and V, support identical patterns in H. Note how the bias of the second unit of V, and VJ is positive to compensate for the fact that the code which repre-sents that unit has all the H units turned off. 4.1. The 4-2-4 Encoder The experiments on networks with v = 4 and h = 2 were performed using the following learning cycle: 1. Esfimation of p,j: Each environmental vector in turn was clamped over the visible units. For each environmental vector, the network was allowed to reach equilibrium twice. Statistics about how often pairs of units were both on together were gathered at equilibrium. To prevent the weights from growing too large we used the “noisy” clamping technique described in Section 3.2. Each on bit of aclamped vector was set to off with a probability of 0.15 and each off bit was set to on with a probability of 0.05. 2. Estimation of p,;: The network was completely unclamped and allowed to reach equilibrium at a temperature of 10. Statistics about BOLTZMANN MACHINE LEARNING 159

快速且它们构成局部最小值，这些局部最小值比全局最小值多得多，且具有几乎同样好的 G 值。在这种 G 空间中，学习算法必须仔细调优才能达到全局最小值，即使如此也非常慢。我们认为该算法非常适合的 G 空间是那些存在大量可能解且不必找到最佳解的空间。为了使大型网络在合理时间内学习，可能需要有足够的单元和权重以及足够宽松的任务规范，使得没有单个单元或权重是必不可少的。下一个例子说明了拥有一些冗余容量的优势。

rapidly and they constitute local minima which are far more numerous than the global minima and have almost as good a value of G. In this type of G-space, the learning algorithm must be carefully tuned to achieve a global minimum, and even then it is very slow. We believe that the G-spaces for which the algorithm is well-suited are ones where there are a great many possible solutions and it is not essential to get the very best one. For large networks to learn in a reasonable time, it may be necessary to have enough units and weights and a liberal enough specification of the task so that no single unit or weight is essential. The next example illustrates the advantages of having some spare capacity.

一个稍大的例子是 40-10-40 编码器。隐藏层中的 10 个单元

A somewhat larger example is the 40-10-40 encoder. The 10 units in hidden layer

几乎是理论最小值的两倍，但它仍然充当有限的带宽瓶颈。学习算法在这个问题上表现良好。图 5 显示了当给定 V 中的模式并要求收敛到 Vz 中的相应模式时其性能。每个学习周期涉及对 40 个环境向量各退火一次（钳制），以及相同次数的无钳制退火。最终性能渐近于 98.6%的正确率。

almost twice the theoretical minimum, but it still acts as a limited bandwidth bottleneck. The learning algorithm works well on this problem. Figure 5 shows its performance when given a pattern in V, and required to settle to the corresponding pattern in Vz. Each learning cycle involved annealing once with each of the 40 environmental vectors clamped, and the same number of times without clamping. The final performance asymptotes at 98.6% correct.

对 V，测试成功。对 VI 中的每个 40 单元重复 10 次。前 300 个学习周期，网络运行时未连接隐藏单元。这确保了每组 40 个可见单元发展出足够的侧向抑制以实现有效的赢家通吃网络。然后连接隐藏单元，在接下来的 500 个学习周期中，我们使用“噪声”钳制，将开通的比特以 0.1 的概率关闭，将关闭的比特以 0.0025 的概率开通。之后我们移除噪声，这解释了 800 周期后性能的陡升。最终性能渐近于 98.6%正确。波尔兹曼机学习 163

V, the test was successful. This was repeated 10 times for each of the 40 units in VI. For the first 300 learning cycles the network was run without connecting up the hidden units. This ensured that each group of 40 visible units developed enough lateral inhibition to implement an effective winner-take-all network. The hidden units were then connected up and for the next 500 learning cycles we used “noisy” clamping, switching on bits to off with a probability of 0.1 and off bits to on with a probability of 0.0025. After this we removed the noise and this explains the sharp rise in performance after 800 cycles. The final performance asymptotes at 98.6% correct. BOLTZMANN MACHINE LEARNING 163

网络选择来表示 V 和 V 中模式的代码全部至少相隔 2 的汉明距离，这不太可能是偶然发生的。作为测试，我们比较了可见单元和隐藏单元之间连接的权重。每个可见单元有 10 个连接到隐藏单元的权重，为了避免错误，两个不同可见单元的 10 维权重向量不应过于相似。使用两个向量之间夹角的余弦作为相似性度量，没有两个代码的相似性大于 0.73，而将相同的权重随机重排作为对照比较时，许多对具有 0.8 或更高的相似性。为了在完成测试中取得良好性能，必须在测试期间使用非常温和的退火计划。该计划在每个温度上花费两倍的时间，并降至学习计划最终温度的一半。随着退火速度加快，错误率上升，从而提供了一个非常自然的速度/准确性权衡。我们没有进一步探讨这个问题，但可能富有成效，因为目前人类反应时间实验中速度/准确性权衡的较好模型都涉及有偏随机游走的概念（Ratcliff, 1978），而退火搜索产生了类似的底层数学。

The codes that the network selected to represent the patterns in V, and V, were all separated by a hamming distance of at least 2, which is very unlikely to happen by chance. As a test, we compared the weights of the connections between visible and hidden units. Each visible unit has 10 weights connecting it to the hidden units, and to avoid errors, the 10 dimensional weight vectors for two different visible units should not be too similar. The cosine of the angle between two vectors was used as a measure of similarity, and no two codes had a similarity greater than 0.73, whereas many pairs had similarities of 0.8 or higher when the same weights were randomly rearranged to provide a control group for comparison. To achieve good performance on the completion tests, it was necessary to use a very gentle annealing schedule during testing. The schedule spent twice as long at each temperature and went down to half the final temperature of the schedule used during learning. As the annealing was made faster, the error rate increased, thus giving a very natural speed/accuracy trade-off. We have not pursued this issue any further, but it may prove fruitful because some of the better current models of the speed/accuracy trade-off in human reaction time experiments involve the idea of a biased random walk (Ratcliff, 1978), and the annealing search gives rise to similar underlying mathematics.

到目前为止，我们一直回避了复杂概念如何在玻尔兹曼机中表示的问题。各个单元代表“假设”，但这些假设与我们用词汇表达的概念之间是什么关系？一些研究者认为，概念应以本质上“局部”的方式表示：一个或几个计算单元的激活即为一个概念的表示（Feldman & Ballard, 1982）；而另一些研究者则认为概念是“分布式”实体：一大组单元上的特定活动模式表示一个概念，不同概念对应同一组单元上的不同活动模式（Hinton, 1981）。支持局部表示的一个较好理由是它们固有的模块化特性。关于概念之间关系的知识被定位在特定的连接中，因此，如果能找到某种合理的硬件连接形成方案，这些知识就易于添加、删除和修改（Fahlman, 1980; Feldman, 1982）。然而，在分布式表示中，知识是弥散的。这有利于对局部硬件损坏的容错性，但似乎使得执行特定功能的模块设计更加困难。尤其难以理解的是，新的分布式概念表示如何自发产生。

So far, we have avoided the issue of how complex concepts would be represented in a Boltzmann machine. The individual units stand for "hypotheses," but what is the relationship between these hypotheses and the kinds of concepts for which we have words? Some workers suggest that a concept should be represented in an essentially "local" fashion: The activation of one or a few computing units is the representation for a concept (Feldman & Ballard, 1982); while others view concepts as "distributed" entities: A particular pattern of activity over a large group of units represents a concept, and different concepts correspond to alternative patterns of activity over the same group of units (Hinton, 1981). One of the better arguments in favor of local representations is their inherent modularity. Knowledge about relationships between concepts is localized in specific connections and is therefore easy to add, remove, and modify, if some reasonable scheme for forming hardware connections can be found (Fahlman, 1980; Feldman, 1982). With distributed representations, however, the knowledge is diffuse. This is good for tolerance to local hardware damage, but it appears to make the design of modules to perform specific functions much harder. It is particularly difficult to see how new distributed representations of concepts could originate spontaneously.

在玻尔兹曼机中，一个分布式表示对应一个能量极小值，因此创建一组良好的分布式表示等价于创建一个良好的“能量景观”。我们提出的学习算法能够解决这个问题，从而使分布式表示变得更加可信。任何一条知识的弥散性不再是一个严重的反对理由，因为玻尔兹曼分布的数学简洁性使得我们可以在纯局部信息的基础上以一致的方式操作所有弥散的局部权重。一组简单分布式表示的形成通过编码器问题得到了说明。

In a Boltzmann machine, a distributed representation corresponds to an energy minimum, and so the problem of creating a good collection of distributed representations is equivalent to the problem of creating a good "energy landscape." The learning algorithm we have presented is capable of solving this problem, and it therefore makes distributed representations considerably more plausible. The diffuseness of any one piece of knowledge is no longer a serious objection, because the mathematical simplicity of the Boltzmann distribution makes it possible to manipulate all the diffuse local weights in a coherent way on the basis of purely local information. The formation of a simple set of distributed representations is illustrated by the encoder problems.

编码器问题的例子还提出了一种在并行计算网络的各个组件之间通信符号的方法。Feldman 和 Ballard（1982）给出了该任务的两种实现草图；以概念“wormy apple”从感知系统中识别的位置传输到语音系统可以生成短语“wormy apple”的位置为例。他们认为只有两种方法可以实现这一点。第一种方法中，感知信息被编码为一组符号，然后作为消息传输到语音系统，在那里被解码为适合发声的形式。在这种情况下，会有一组通用通信线路，类似于传统计算机中的总线，用作从视觉系统到语音系统的所有此类消息的媒介。Feldman 和 Ballard 描述了这种系统的问题如下：

The encoder problem examples also suggest a method for communicating symbols between various components of a parallel computational network. Feldman and Ballard (1982) present sketches of two implementations for this task; using the example of the transmission of the concept "wormy apple" from where it is recognized in the perceptual system to where the phrase "wormy apple" can be generated by the speech system. They argue that there appears to be only two ways that this could be accomplished. In the first method, the perceptual information is encoded into a set of symbols that are then transmitted as messages to the speech system, where they are decoded into a form suitable for utterance. In this case, there would be a set of general-purpose communication lines, analogous to a bus in a conventional computer, that would be used as the medium for all such messages from the visual system to the speech system. Feldman and Ballard describe the problems with such a system as:

复杂消息可能必须通过通信线路顺序传输。

Complex messages would presumably have to be transmitted sequentially over the communication lines.

发送方和接收方都必须为每个新概念学习共同的编码。

Both sender and receiver would have to learn the common code for each new concept.

该方法作为大脑机制在生物学上似乎不可信。他们提出的替代实现要求为每个从感知系统传递到言语系统的概念提供一条独立的专用硬件通路。其思想是，感知系统中“苹果”和“蠕虫”的同时激活可以通过私有链接传输到言语系统中对应的概念。这种实现的关键问题在于概念之间是否具备必要的连接，以及在两个系统中学习新概念时能否建立新的连接。

The method seems biologically implausible as a mechanism for the brain. The alternative implementation they suggest requires an individual, dedicated hardware pathway for each concept that is communicated from the perceptual system to the speech system. The idea is that the simultaneous activation of “apple” and “worm” in the perceptual system can be transmitted over private links to their counterparts in the speech system. The critical issues for such an implementation are having the necessary connections available between concepts, and being able to establish new BOLTZMANN MACHINE LEARNING 165

连接通路，以便在两个系统中学习新概念。这种方法的主要特点是，计算单元之间的链接携带简单、非符号化的信息，例如单个激活值。玻尔兹曼机在面对编码器问题时展现的行为，展示了一种在很大程度上融合了上述两种实现优点的概念通信方式。与第二种方法类似，计算单元小巧，链接携带简单的数值，计算和连接需求在生物学合理范围内。与第一种方法类似，其架构允许通过相同的通信线路传输许多不同的概念，从而有效利用有限的连接。通过 G-最小化学习算法，表示新概念的新编码会作为协作过程自动出现。

connection pathways as new concepts are learned in the two systems. The main point of this approach is that the links between the computing units carry simple, nonsymbolic information such as a single activation level. The behavior of the Boltzmann machine when presented with an encoder problem demonstrates a way of communicating concepts that largely combines the best of the two implementations mentioned. Like the second approach, the computing units are small, the links carry a simple numeric value, and the computational and connection requirements are within the range of biological plausibility. Like the first approach, the architecture is such that many different concepts can be transmitted over the same communication lines, allowing for effective use of limited connections. The learning of new codes to represent new concepts emerges automatically as a cooperative process from the G-minimization learning algorithm.

将统计力学应用于并行网络中的约束满足搜索，是一个有前景的新领域，已被其他几个研究小组独立发现（Geman & Geman, 1983; Smolensky, 1983）。有许多有趣的问题我们只是顺带提及。其中一些在其他地方有更详细的讨论：Hinton 和 Sejnowski (1983b) 以及 Geman 和 Geman (1983) 描述了与贝叶斯推理及更传统的松弛技术的关系；Fahlman、Hinton 和 Sejnowski (1983) 将玻尔兹曼机与其他并行方案进行了比较，并讨论了知识表示的问题。本文的扩充版本（Hinton, Sejnowski, & Ackley, 1984）更深入地介绍了这些内容，并讨论了与大脑的关系、顺序行为问题等相关议题。它还展示了如何利用高斯噪声实现概率决策函数，如何放宽物理连接对称性和传输无时间延迟的假设，并描述了其他任务上的模拟结果。具有对称权重的系统构成了一类有趣的计算设备，因为其动力学由能量函数支配。这使得分析它们的行为并将其用于迭代约束满足成为可能。在他们对感知机的影响深远的探索中，Minsky 和 Papert (1968, p. 231) 得出结论：“带循环的多层机器显然引发了自动机理论的所有问题。”尽管这一论断非常合理，但最近的发展

The application of statistical mechanics to constraint satisfaction searches in parallel networks is a promising new area that has been discovered independently by several other groups (Geman & Geman, 1983; Smolensky, 1983). There are many interesting issues that we have only mentioned in passing. Some of these issues are discussed in greater detail elsewhere: Hinton and Sejnowski (1983b) and Geman and Geman (1983) describe the relation to Bayesian inference and to more conventional relaxation techniques; Fahlman, Hinton, and Sejnowski (1983) compare Boltzmann machines with some alternative parallel schemes, and discuss some knowledge representation issues. An expanded version of this paper (Hinton, Sejnowski, & Ackley, 1984) presents this material in greater depth and discusses a number of related issues such as the relationship to the brain and the problem of sequential behavior. It also shows how the probabilistic decision function could be realized using Gaussian noise, how the assumptions of symmetry in the physical connections and of no time delay in transmission can be relaxed, and describes results of simulations on some other tasks. Systems with symmetric weights form an interesting class of computational device because their dynamics is governed by an energy function. This is what makes it possible to analyze their behavior and to use them for iterative constraint satisfaction. In their influential exploration of perceptrons, Minsky and Papert (1968, p. 231) concluded that: “Multilayer machines with loops clearly open up all the questions of the general theory of automata.” Although this statement is very plausible, recent developments

表明它可能具有误导性，因为它忽略了对称情况，并且似乎导致了一种普遍看法，即不可能为感知机类元素的网络找到强大的学习算法。我们认为玻尔兹曼机是一类有趣的随机模型的简单示例，这类模型利用了玻尔兹曼分布与信息论之间的紧密关系。

suggest that it may be misleading because it ignores the symmetric case, and seems to have led to the general belief that it would be impossible to find powerful learning algorithms for networks of perceptron-like elements. We believe that the Boltzmann Machine is a simple example of a class of interesting stochastic models that exploit the close relationship between Boltzmann distributions and information theory.

The method seems biologically implausible as a mechanism for the brain. The alternative implementation they suggest requires an individual, dedicated hardware pathway for each concept that is communicated from the perceptual system to the speech system. The idea is that the simulta-neous activation of “apple” and “worm” in the perceptual system can be transmitted over private links to their counterparts in the speech system. The critical issues for such an implementation are having the necessary con-nections available between concepts, and being able to establish new con-BOLTZMANN MACHINE LEARNING 165

这是热力学，主要以其从玻尔兹曼那里接受的形式，并且是那一部分

This is thermodynamics, primarily in the form it was received from Boltzmann, and is that part

（约翰·冯·诺依曼，文集卷 5，第 304 页）附录：学习算法的推导 当一个网络在平衡状态下自由运行时，可见单元上的概率分布由

(John Von Neumann, Collected Works Vol. 5, p. 304) APPENDIX: DERIVATION OF THE LEARNING ALGORITHM When a network is free-running at equilibrium the probability distribution over the visible units is given by

玻尔兹曼机学习 P’(V,)=CP’(V,AH,)= I:;: 8 (11) An

其中 \(V_\alpha\) 是可见单元的状态向量，\(H_\beta\) 是隐藏单元的状态向量，\(E_{\alpha\beta}\) 是系统在状态 \((V_\alpha, H_\beta)\) 下的能量。

where \(V_\alpha\) is a vector of states of the visible units, \(H_\beta\) is a vector of states of the hidden units, and \(E_{\alpha\beta}\) is the energy of the system in state \((V_\alpha, H_\beta)\).

因此，对(11)求导得到玻尔兹曼机学习 167

Hence, differentiating (11) then yields Boltzmann Machine Learning 167

1 = -T \left[ \sum_\alpha P'(V_\alpha) \frac{\partial E_{\alpha\beta}}{\partial w_{ij}} - \sum_{\alpha,\beta} P'(V_\alpha, H_\beta) \frac{\partial E_{\alpha\beta}}{\partial w_{ij}} \right] + 1.

1 = -T \left[ \sum_\alpha P'(V_\alpha) \frac{\partial E_{\alpha\beta}}{\partial w_{ij}} - \sum_{\alpha,\beta} P'(V_\alpha, H_\beta) \frac{\partial E_{\alpha\beta}}{\partial w_{ij}} \right] + 1.

该导数用于计算 G 测度的梯度

This derivative is used to compute the gradient of the G-measure

G = \sum_\alpha P(V_\alpha) \ln \frac{P(V_\alpha)}{P'(V_\alpha)} ，其中 \(P(V_\alpha)\) 是可见单元上的钳位概率分布，且与 \(w_{ij}\) 无关。因此

G = \sum_\alpha P(V_\alpha) \ln \frac{P(V_\alpha)}{P'(V_\alpha)} where P(V_\alpha) is the clamped probability distribution over the visible units and is independent of w_{ij}. So

\(\frac{\partial G}{\partial w_{ij}} = \frac{1}{c} \sum_{V,H} P'(V,H) \frac{\partial}{\partial w_{ij}} \ln P(V,H)\)

\(\frac{\partial G}{\partial w_{ij}} = \frac{1}{c} \sum_{V,H} P'(V,H) \frac{\partial}{\partial w_{ij}} \ln P(V,H)\)

方程(12)成立，因为在平衡状态下，给定某个可见状态时隐藏状态的概率必须相同，无论可见单元是被钳制在该状态还是通过自由运行到达该状态。因此，此外，因此，如(9)所示。玻尔兹曼机学习算法也可以表述为输入-输出模型。可见单元分为输入集\(I\)和输出集\(O\)，环境指定一组形如\(P(O|I)\)的条件概率。在“训练”阶段，环境同时钳制输入和输出单元，并估计\(p_{ij}^+\)。在“测试”阶段，输入单元被钳制，输出单元和隐藏单元自由运行，并估计\(p_{ij}^-\)。这种情况下合适的\(G\)度量是类似的。类似的数学公式也适用于此表述，且\(\partial G / \partial w_{ij}\)与之前相同。

Equation (12) holds because the probability of a hidden state given some visible state must be the same in equilibrium whether the visible units were clamped in that state or arrived there by free-running. Hence, Also, Therefore, and as given in (9). The Boltzmann Machine learning algorithm can also be formulated as an input-output model. The visible units are divided into an input set \(I\) and an output set \(O\), and an environment specifies a set of conditional probabilities of the form \(P(O | I)\). During the “training” phase the environment clamps both the input and output units, and \(p_{ij}^+\) are estimated. During the “testing” phase the input units are clamped and the output units and hidden units free-run, and \(p_{ij}^-\) are estimated. The appropriate \(G\) measure in this case is similar. Similar mathematics apply in this formulation and \(\partial G / \partial w_{ij}\) is the same as before.

Berliner, H. J. 和 Ackley, D. H. (1982 年 8 月). QBKG 系统：从非离散知识表示生成解释. 全国人工智能会议论文集, AAAI-82, 匹兹堡, PA, 213-216.

Berliner, H. J., & Ackley, D. H. (1982, August). The QBKG system: Generating explanations from a non-discrete knowledge representation. Proceedings of the National Conference on Artificial Intelligence, AAAI-82, Pittsburgh, PA, 213-216.

Binder, K. (编) (1978). 统计物理学中的蒙特卡罗方法. 纽约: Springer-Verlag. Fahlman, S. E. (1980 年 6 月). Hashnet 互连方案. (技术报告 No. CMU-CS-80-125). 卡内基梅隆大学, 匹兹堡, PA. Fahlman, S. E., Hinton, G. E., 和 Sejnowski, T. J. (1983 年 8 月). 用于 AI 的大规模并行架构：NETL, Thistle 和玻尔兹曼机. 全国人工智能会议论文集, AAAI-83, 华盛顿特区, 109-113. Feldman, J. A. (1982). 神经网络中的动态连接. 生物控制论, 46, 27-39. Feldman, J. A. 和 Ballard, D. H. (1982). 连接主义模型及其性质. 认知科学, 6, 205-254. Geman, S. 和 Geman, D. (1983). 随机松弛、吉布斯分布和图像的贝叶斯复原. 未发表手稿. Crimson, W. E. L. (1981). 从图像到表面. 剑桥, MA: MIT 出版社. Hinton, G. E. (1977). 松弛及其在视觉中的作用. 未发表博士论文, 爱丁堡大学. 描述于 D. H. Ballard 和 C. M. Brown (编), 计算机视觉. 恩格尔伍德克利夫斯, NJ: Prentice-Hall, 408-430. 玻尔兹曼机学习 169

Binder, K. (Ed.) (1978). The Monte Carlo Method in Statistical Physics. New York: Springer-Verlag. Fahlman, S. E. (1980, June). The Hashnet Interconnection Scheme. (Tech. Rep. No. CMU-CS-80-125). Carnegie-Mellon University, Pittsburgh, PA. Fahlman, S. E., Hinton, G. E., & Sejnowski, T. J. (1983, August). Massively parallel architectures for AI: NETL, Thistle, and Boltzmann Machines. Proceedings of the National Conference on Artificial Intelligence, AAAI-83, Washington, DC, 109-113. Feldman, J. A. (1982). Dynamic connections in neural networks. Biological Cybernetics, 46, 27-39. Feldman, J. A., & Ballard, D. H. (1982). Connectionist models and their properties. Cognitive Science, 6, 205-254. Geman, S., & Geman, D. (1983). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. Unpublished manuscript. Crimson, W. E. L. (1981). From Images to Surfaces. Cambridge, MA: MIT Press. Hinton, G. E. (1977). Relaxation and its role in vision. Unpublished doctoral dissertation, University of Edinburgh. Described in D. H. Ballard & C. M. Brown (Eds.), Computer Vision. Englewood Cliffs, NJ: Prentice-Hall, 408-430. BOLTZMANN MACHINE LEARNING 169

aG _ c p(k) apw,) -= _ aWij c( P’(V,) aw,

G. E. Hinton（1981）. 在并行硬件中实现语义网络. 见 G. E. Hinton 与 J. A. Anderson（主编），《并行联想记忆模型》. 希尔斯代尔，新泽西：Erlbaum 出版社. G. E. Hinton 与 J. A. Anderson（1981）. 《并行联想记忆模型》. 希尔斯代尔，新泽西：Erlbaum 出版社. G. E. Hinton 与 T. J. Sejnowski（1983a，5 月）. 分析协作计算. 第五届认知科学学会年会论文集. 罗切斯特，纽约. G. E. Hinton 与 T. J. Sejnowski（1983b，6 月）. 最优感知推理. IEEE 计算机学会计算机视觉与模式识别会议论文集. 华盛顿特区，第 448-453 页. G. E. Hinton，T. J. Sejnowski 与 D. H. Ackley（1984，5 月）. 玻尔兹曼机：能够学习的约束满足网络. （技术报告 No. CMU-CS-84-119）. 匹兹堡，宾夕法尼亚：卡内基梅隆大学. J. J. Hopfield（1982）. 具有涌现集体计算能力的神经网络与物理系统. 《美国国家科学院院刊》，79，2554-2558. S. Kirkpatrick，C. D. Gelatt 与 M. P. Vecchi（1983）. 通过模拟退火进行优化. 《科学》，220，611-680. S. Kullback（1959）. 《信息论与统计学》. 纽约：Wiley. N. Metropolis，A. Rosenbluth，M. Rosenbluth，A. Teller 与 E. Teller（1953）. 快速计算机的状态方程计算. 《化学物理学报》，6，1087. M. Minsky 与 S. Papert（1968）. 《感知机》. 剑桥，麻省：MIT 出版社. A. Newell（1982）. 人工智能史中的智力问题. （技术报告 No. CMU-CS-82-142）. 匹兹堡，宾夕法尼亚：卡内基梅隆大学. A. Newell 与 H. A. Simon（1972）. 《人类问题解决》. 恩格尔伍德克利夫斯，新泽西：Prentice-Hall，1972. R. Ratcliff（1978）. 记忆检索理论. 《心理学评论》，85，59-108. A. Renyi（1962）. 《概率论》. 阿姆斯特丹：North-Holland. F. Rosenblatt（1961）. 《神经动力学原理：感知机与大脑机制理论》. 华盛顿特区：Spartan. P. Smolensky（1983，8 月）. 模块化环境中的图式选择与随机推理. 全国人工智能会议（AAAI-83）论文集. 华盛顿特区，第 109-113 页. D. Terzopoulos（1984）. 可见表面表示的多分辨率计算. 未出版的博士论文，MIT，剑桥，麻省. D. L. Waltz（1975）. 理解带阴影场景的线条图. 见 P. Winston（主编），《计算机视觉心理学》. 纽约：McGraw-Hill. P. H. Winston（1984）. 《人工智能》. （第 2 版）雷丁，麻省：Addison-Wesley.

Hinton, G. E. (1981). Implementing semantic networks in parallel hardware. In G. E. Hinton & J. A. Anderson (Eds.), Parallel Models of Associative Memory. Hillsdale, NJ: Erlbaum. Hinton, G. E., & Anderson, J. A. (1981). Parallel models of associative memory. Hillsdale, NJ: Erlbaum. Hinton, G. E., & Sejnowski, T. J. (1983a, May). Analyzing cooperative computation. Proceedings of the Fifth Annual Conference of the Cognitive Science Society. Rochester, NY. Hinton, G. E., & Sejnowski, T. J. (1983b, June). Optimal perceptual inference. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, pp. 448-453. Hinton, G. E., Sejnowski, T. J., & Ackley, D. H. (1984, May). Boltzmann Machines: Constraint satisfaction networks that learn. (Tech. Rep. No. CMU-CS-84-119). Pittsburgh, PA: Carnegie-Mellon University. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554-2558. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 611-680. Kullback, S. (1959). Information theory and statistics. New York: Wiley. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., & Teller, E. (1953). Equation of state calculations for fast computing machines. Journal of Chemical Physics, 6, 1087. Minsky, M., & Papert, S. (1968). Perceptrons. Cambridge, MA: MIT Press. Newell, A. (1982). Intellectual issues in the history of artificial intelligence. (Tech. Rep. No. CMU-CS-82-142). Pittsburgh, PA: Carnegie-Mellon University. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall, 1972. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108. Renyi, A. (1962). Probability theory. Amsterdam: North-Holland. Rosenblatt, F. (1961). Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. Washington, DC: Spartan. Smolensky, P. (1983, August). Schema selection and stochastic inference in modular environments. Proceedings of the National Conference on Artificial Intelligence (AAAI-83). Washington, DC, pp. 109-113. Terzopoulos, D. (1984). Multi-resolution computation of visible-surface representations. Unpublished doctoral dissertation, MIT, Cambridge, MA. Waltz, D. L. (1975). Understanding line drawings of scenes with shadows. In P. Winston (Ed.), The Psychology of Computer Vision. New York: McGraw-Hill. Winston, P. H. (1984). Artificial intelligence. (2nd ed.) Reading, MA: Addison-Wesley.