In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or 'atrous convolution', as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed "DeepLab" system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7% mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.
核心贡献 · Key contributions
提出空洞卷积用于 DCNN 密集特征提取，在不增加参数的情况下控制分辨率。 Proposed atrous convolution for dense feature extraction in DCNNs, controlling resolution without increasing parameters.
引入空洞空间金字塔池化（ASPP），以鲁棒地分割多尺度物体。 Introduced Atrous Spatial Pyramid Pooling (ASPP) to robustly segment objects at multiple scales.
将 DCNN 与全连接条件随机场结合，改善物体边界定位。 Combined DCNN with fully connected CRF to improve object boundary localization.
在 PASCAL VOC 2012、PASCAL-Context、PASCAL-Person-Part 和 Cityscapes 上取得最先进结果。 Achieved state-of-the-art results on PASCAL VOC 2012, PASCAL-Context, PASCAL-Person-Part, and Cityscapes.
证明空洞卷积在不增加计算量的情况下扩大滤波器视野。 Demonstrated that atrous convolution enlarges filter field-of-view without extra computation.
表明 CRF 后处理细化分割边界并去除假阳性。 Showed that CRF post-processing refines segmentation boundaries and removes false positives.
局限 · Limitations
CRF 作为后处理，未与 DCNN 联合训练，限制了端到端优化。 CRF used as post-processing, not jointly trained with DCNN, limiting end-to-end optimization.
方法在自行车和椅子等精细物体边界上失败，因一元置信度低。 Method fails on fine object boundaries like bicycles and chairs due to low unary confidence.
高分辨率图像因 GPU 内存限制需分割为重叠区域。 High-resolution images require splitting into overlapped regions due to GPU memory limits.