Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets, achieving the test set performance of 89.0\% and 82.1\% without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow at \url{https://github.com/tensorflow/models/tree/master/research/deeplab}.
核心贡献 · Key contributions
提出 DeepLabv3+，结合空间金字塔池化与编码器-解码器结构用于语义分割。 Proposes DeepLabv3+, combining spatial pyramid pooling and encoder-decoder for semantic segmentation.
在 DeepLabv3 基础上添加简单有效的解码器模块以细化物体边界。 Adds a simple yet effective decoder module to DeepLabv3 to refine object boundaries.
将深度可分离卷积应用于 ASPP 和解码器，实现更快更强的网络。 Applies depthwise separable convolution to ASPP and decoder for faster and stronger network.
改进 Xception 模型以适应语义分割，支持密集特征提取。 Adapts Xception model for semantic segmentation with modifications for dense feature extraction.
在 PASCAL VOC 2012（89.0%）和 Cityscapes（82.1%）上无需后处理即达到最优结果。 Achieves state-of-the-art results on PASCAL VOC 2012 (89.0%) and Cityscapes (82.1%) without post-processing.
提供公开的 TensorFlow 实现以促进可复现性。 Provides public TensorFlow implementation for reproducibility.
局限 · Limitations
性能仅在 PASCAL VOC 2012 和 Cityscapes 数据集上评估。 Performance evaluated only on PASCAL VOC 2012 and Cityscapes datasets.
使用输出步长 8 训练需要大量计算资源。 Requires large computational resources for training with output stride 8.
解码器设计未经调整可能无法泛化到其他任务。 Decoder design may not generalize to other tasks without tuning.
对严重遮挡物体和罕见视角分割困难。 Struggles with heavily occluded objects and rare viewpoints.
未使用 CRF 等后处理，限制了潜在性能提升。 No post-processing like CRF used, limiting potential further gains.
论文章节 · Sections（共 13）
摘要Abstract
1 引言1 Introduction
2 相关工作2 Related Work
3 方法3 Methods
3.1 带空洞卷积的编码器-解码器3.1 Encoder-Decoder with Atrous Convolution
3.2 改进的对齐 Xception3.2 Modified Aligned Xception
4 实验评估4 Experimental Evaluation
4.1 解码器设计选择4.1 Decoder Design Choices
4.2 ResNet-101 作为网络骨干4.2 ResNet-101 as Network Backbone
4.3 Xception 作为网络骨干4.3 Xception as Network Backbone
4.4 沿物体边界的改进4.4 Improvement along Object Boundaries
4.5 Cityscapes 上的实验结果4.5 Experimental Results on Cityscapes