Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But recent deep learning object detectors have avoided pyramid representations, in part because they are compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using FPN in a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.
核心贡献 · Key contributions
提出特征金字塔网络（FPN），通过自上而下路径和横向连接，将低分辨率强语义特征与高分辨率弱语义特征结合。 Proposes Feature Pyramid Network (FPN) combining low-resolution strong semantics with high-resolution weak features via top-down pathway and lateral connections.
使用 FPN 与基础 Faster R-CNN，在 COCO 检测基准上取得单模型最优结果。 Achieves state-of-the-art single-model results on COCO detection benchmark using FPN with basic Faster R-CNN.
边界框提议的平均召回率（AR）提升 8.0 点，小物体检测 AR 提升 12.9 点。 Improves Average Recall (AR) by 8.0 points for bounding box proposals and boosts small object detection AR by 12.9 points.
支持所有尺度的端到端训练，训练/测试一致使用，避免图像金字塔的内存问题。 Enables end-to-end training with all scales and consistent train/test time usage, avoiding memory issues of image pyramids.
将 FPN 扩展到掩码提议生成，达到 48.1 AR 和 6-7 FPS，比先前方法 AR 提升超过 8.3 点。 Extends FPN to mask proposal generation, achieving 48.1 AR and 6-7 FPS, outperforming prior methods by over 8.3 AR.
证明尽管深度卷积网络具有尺度鲁棒性，显式多尺度金字塔表示仍然至关重要。 Demonstrates that explicit multi-scale pyramid representations remain critical despite deep ConvNets' scale robustness.
局限 · Limitations
评估仅限于 ResNet 骨干网络，未探索 Transformer 等其他架构的性能。 Evaluation limited to ResNet backbones; performance on other architectures like Transformer not explored.
方法依赖预定义的锚点尺度和长宽比，可能无法泛化到任意物体形状。 Method relies on predefined anchor scales and aspect ratios; may not generalize to arbitrary object shapes.
FPN 增加了额外层和计算量（虽小），在极端资源受限场景下可能非最优。 FPN adds extra layers and computations, though marginal; may not be optimal for extremely resource-constrained settings.
实验在 COCO 数据集上进行，未验证在其他领域（如医学影像）的泛化性。 Experiments conducted on COCO dataset; generalization to other domains (e.g., medical imaging) not validated.
共享分类器/回归器头假设金字塔各层语义一致，语义差距大时可能失效。 Shared classifier/regressor heads assume uniform semantics across pyramid levels; may fail if semantic gaps are large.
论文章节 · Sections（共 14）
摘要Abstract
1 引言1 Introduction
2 相关工作2 Related Work
3 特征金字塔网络3 Feature Pyramid Networks
4 应用4 Applications
4.1 用于 RPN 的特征金字塔网络4.1 Feature Pyramid Networks for RPN
4.2 用于 Fast R-CNN 的特征金字塔网络4.2 Feature Pyramid Networks for Fast R-CNN
5 目标检测实验5 Experiments on Object Detection
5.1 基于 RPN 的区域提议5.1 Region Proposal with RPN
5.2 基于 Fast/Faster R-CNN 的目标检测5.2 Object Detection with Fast/Faster R-CNN
6 扩展：分割提议6 Extensions: Segmentation Proposals
6.1 分割提议结果6.1 Segmentation Proposal Results
7 结论7 Conclusion
附录 A 分割提议的实现Appendix A Implementation of Segmentation Proposals