深度残差学习在图像识别中的应用

Deep Residual Learning for Image Recognition

何恺明 Kaiming He · Microsoft · 2015-12-10 · arXiv:1512.03385 ↗ · 被引 231533

摘要 · Abstract

更深的神经网络更难训练。我们提出了一种残差学习框架，以简化比以往使用的网络更深网络的训练。我们明确地将层重新表述为学习相对于层输入的残差函数，而不是学习无参考的函数。我们提供了全面的经验证据，表明这些残差网络更容易优化，并且可以从显著增加的深度中获得准确性。在 ImageNet 数据集上，我们评估了深度高达 152 层的残差网络——比 VGG 网络深 8 倍，但复杂度更低。这些残差网络的集成在 ImageNet 测试集上实现了 3.57%的错误率。这一结果在 ILSVRC 2015 分类任务中获得了第一名。我们还对 CIFAR-10 进行了 100 层和 1000 层的分析。表示的深度对于许多视觉识别任务至关重要。仅由于我们极深的表示，我们在 COCO 目标检测数据集上获得了 28%的相对改进。深度残差网络是我们提交给 ILSVRC 和 COCO 2015 竞赛的基础，我们在 ImageNet 检测、ImageNet 定位、COCO 检测和 COCO 分割任务中也获得了第一名。

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

核心贡献 · Key contributions

提出残差学习框架，简化了极深网络的训练。
Proposed residual learning framework to ease training of substantially deeper networks.
证明深度残差网络更易优化，且随深度增加精度提升。
Showed deep residual nets are easier to optimize and gain accuracy from increased depth.
在 ImageNet 测试集上达到 3.57% top-5 错误率，赢得 ILSVRC 2015 分类任务。
Achieved 3.57% top-5 error on ImageNet test set, winning ILSVRC 2015 classification.
在 COCO 目标检测上因深度表示获得 28%的相对提升。
Demonstrated 28% relative improvement on COCO object detection due to deep representations.
引入恒等快捷连接，不增加额外参数或计算量。
Introduced identity shortcut connections adding no extra parameters or computation.
在 CIFAR-10 上成功训练了超过 1000 层的网络。
Successfully trained networks with over 1000 layers on CIFAR-10.

局限 · Limitations

残差学习可能无法直接泛化到非视觉任务。
Residual learning may not generalize to non-vision tasks without further adaptation.
极深模型（如 1202 层）在小数据集（如 CIFAR-10）上可能过拟合。
Very deep models (e.g., 1202 layers) can overfit on small datasets like CIFAR-10.
退化问题的根本原因未完全解释，仅被缓解。
The degradation problem's root cause is not fully explained; only mitigated.
恒等快捷连接假设输入输出维度匹配，否则需投影。
Identity shortcuts assume input and output dimensions match, requiring projections otherwise.
训练极深 ResNet 可能需要仔细的学习率预热。
Training very deep ResNets may require careful learning rate warm-up.

论文章节 · Sections（共 10）

摘要 Abstract
1 引言 1 Introduction
2 相关工作 2 Related Work
3.1 残差学习 3.1 Residual Learning
3.2 恒等映射通过快捷连接 3.2 Identity Mapping by Shortcuts
3.3 网络架构 3.3 Network Architectures
3.4 实现细节 3.4 Implementation
4.1 ImageNet 分类 4.1 ImageNet Classification
4.2 CIFAR-10 与分析 4.2 CIFAR-10 and Analysis
4.3 PASCAL 和 MS COCO 上的目标检测 4.3 Object Detection on PASCAL and MS COCO

阅读逐段中英对照全文 →