Mask R-CNN

罗斯·吉尔希克 Ross Girshick · Facebook AI Research (FAIR) · 2017-03-20 · arXiv:1703.06870 ↗ · 被引 32324

摘要 · Abstract

我们提出了一个概念上简单、灵活且通用的物体实例分割框架。我们的方法在高效检测图像中物体的同时，为每个实例生成高质量的分割掩码。该方法称为 Mask R-CNN，它通过添加一个与现有边界框识别分支并行的物体掩码预测分支来扩展 Faster R-CNN。Mask R-CNN 易于训练，且仅对 Faster R-CNN 增加少量开销，运行速度为 5 fps。此外，Mask R-CNN 易于推广到其他任务，例如允许我们在同一框架中估计人体姿态。我们在 COCO 挑战赛的所有三个赛道（包括实例分割、边界框物体检测和人体关键点检测）中均展示了最佳结果。无需花哨技巧，Mask R-CNN 在每项任务上都优于所有现有的单模型参赛作品，包括 COCO 2016 挑战赛的获胜者。我们希望我们简单而有效的方法能够作为坚实的基线，并有助于简化未来在实例级识别方面的研究。代码已开源：https://github.com/facebookresearch/Detectron

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without bells and whistles, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code has been made available at: https://github.com/facebookresearch/Detectron

核心贡献 · Key contributions

提出 Mask R-CNN，通过向 Faster R-CNN 添加掩码分支，实现简单灵活的实例分割框架。
Proposes Mask R-CNN, a simple and flexible framework for instance segmentation by adding a mask branch to Faster R-CNN.
引入 RoIAlign，一种无量化层，精确保留空间位置，将掩码精度提升 10-50%。
Introduces RoIAlign, a quantization-free layer that preserves exact spatial locations, improving mask accuracy by 10-50%.
通过逐像素 sigmoid 和二元损失解耦掩码与类别预测，优于基于 softmax 的耦合方法。
Decouples mask and class prediction using per-pixel sigmoid and binary loss, outperforming softmax-based coupling.
在 COCO 实例分割、边界框检测和关键点检测上取得最佳结果，超越此前优胜者。
Achieves top results on COCO instance segmentation, bounding-box detection, and keypoint detection, surpassing previous winners.
通过最小修改扩展到人体姿态估计，运行速度 5 fps，展示了通用性。
Demonstrates generality by extending to human pose estimation with minimal modification, running at 5 fps.
提供开源代码，促进可重复性和未来研究。
Provides open-source code for reproducibility and future research.

局限 · Limitations

Mask R-CNN 对 Faster R-CNN 增加少量开销，运行速度 5 fps，可能限制实时应用。
Mask R-CNN adds small overhead to Faster R-CNN, running at 5 fps, which may limit real-time applications.
RoIAlign 需要双线性插值，相比 RoIPool 增加计算成本。
RoIAlign requires bilinear interpolation, increasing computational cost compared to RoIPool.
小物体（APS）上的性能低于中大型物体，如 COCO 结果所示。
Performance on small objects (APS) is lower than on medium and large objects, as shown in COCO results.
框架依赖预训练骨干网络（如 ResNet、ResNeXt），可能不适用于数据有限的领域。
The framework relies on pre-trained backbones (e.g., ResNet, ResNeXt), which may not generalize to domains with limited data.
在 Cityscapes 上训练少数类别（如卡车、公交车）时出现领域偏移和小样本学习挑战。
Training on Cityscapes with few categories (e.g., truck, bus) shows domain shift and low-shot learning challenges.

论文章节 · Sections（共 15）

摘要 Abstract
1 引言 1 Introduction
2 相关工作 2 Related Work
3 Mask R-CNN 3 Mask R-CNN
3.1 实现细节 3.1 Implementation Details
4 实验：实例分割 4 Experiments: Instance Segmentation
4.1 主要结果 4.1 Main Results
4.2 消融实验 4.2 Ablation Experiments
4.3 边界框检测结果 4.3 Bounding Box Detection Results
4.4 时间性能 4.4 Timing
5 用于人体姿态估计的 Mask R-CNN 5 Mask R-CNN for Human Pose Estimation
附录 A：Cityscapes 实验 Appendix A: Experiments on Cityscapes
附录 B：COCO 增强结果 Appendix B: Enhanced Results on COCO
实例分割与目标检测 Instance Segmentation and Object Detection
关键点检测 Keypoint Detection

阅读逐段中英对照全文 →