通过误差传播学习内部表示 D. E. RUMELHART、G. E. HINTON 和 R. J. WILLIAMS 问题 我们现在对简单的两层关联网络有了相当好的理解，其中一组到达输入层的输入模式直接映射到输出层的一组输出模式。这种网络没有隐藏单元，仅涉及输入和输出
Learning Internal Representations by Error Propagation D. E. RUMELHART, G. E. HINTON, and R. J. WILLIAMS THE PROBLEM We now have a rather good understanding of simple two-layer associative networks in which a set of input patterns arriving at an input layer are mapped directly to a set of output patterns at an output layer. Such networks have no hidden units. They involve only input and output
核心贡献 · Key contributions
推导出适用于多层前馈网络和半线性单元的广义 delta 规则。 Derives the generalized delta rule for training multilayer feedforward networks with semilinear units.
证明误差信号可通过反向传播在隐藏层中递归计算。 Shows that error signals can be recursively computed by backpropagation through hidden layers.
展示该过程能够解决非线性可分问题，如 XOR、奇偶性和对称性。 Demonstrates that the procedure solves nonlinearly separable problems like XOR, parity, and symmetry.
实验发现局部最小值罕见，且可通过增加隐藏单元或提高学习率避免。 Empirically finds that local minima are rare and can be avoided by adding hidden units or increasing learning rate.
引入动量项和随机初始权重以加速学习并打破对称性。 Introduces momentum term and random initial weights to accelerate learning and break symmetry.
局限 · Limitations
该方法要求可微的激活函数，排除了线性阈值单元。 The method requires differentiable activation functions, excluding linear threshold units.
存在局部最小值，例如最小隐藏单元下的二进制加法问题，需要额外单元。 Local minima exist, e.g., in binary addition with minimal hidden units, requiring extra units.
对称性问题：等初始权重会阻止学习，必须使用随机权重。 Symmetry problem: equal initial weights prevent learning; random weights are necessary.
学习速度依赖于隐藏单元数量和学习率，某些任务中非常缓慢。 Learning speed depends on number of hidden units and learning rate; very slow for some tasks.
隐藏单元常因初始误差最小化偏向而形成抑制性连接。 Hidden units often develop inhibitory connections due to initial error minimization bias.
论文章节 · Sections（共 11）
通过误差传播学习内部表示S
反向传播算法1: aE paWj; p aWj;
公式推导-a -Lwk;op ;= 12 aE p Wkj=-L8p1cW kj' k anetpk aOpj k anetpk aOpj ; k anetpk k