Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state. This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs. Different from the recent CNNs that focus on large dense kernels, InternImage takes deformable convolution as the core operator, so that our model not only has the large effective receptive field required for downstream tasks such as detection and segmentation, but also has the adaptive spatial aggregation conditioned by input and task information. As a result, the proposed InternImage reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large-scale parameters from massive data like ViTs. The effectiveness of our model is proven on challenging benchmarks including ImageNet, COCO, and ADE20K. It is worth mentioning that InternImage-H achieved a new record 65.4 mAP on COCO test-dev and 62.9 mIoU on ADE20K, outperforming current leading CNNs and ViTs. The code will be released at https://github.com/OpenGVLab/InternImage.
核心贡献 · Key contributions
提出 InternImage，一种以可变形卷积为核心算子的大规模 CNN 基础模型。 Proposes InternImage, a large-scale CNN foundation model using deformable convolution as core operator.
引入 DCNv3，包含权重共享、多组机制和 softmax 归一化，以实现稳定训练。 Introduces DCNv3 with shared weights, multi-group mechanism, and softmax normalization for stable training.
开发堆叠和缩放规则，将 CNN 高效扩展至超过 10 亿参数。 Develops stacking and scaling rules to efficiently scale CNNs to over 1 billion parameters.
在 COCO（65.4 mAP）和 ADE20K（62.9 mIoU）上取得最先进结果，超越领先的 ViT。 Achieves state-of-the-art results on COCO (65.4 mAP) and ADE20K (62.9 mIoU), outperforming leading ViTs.
证明通过适当设计，基于 CNN 的模型在大规模设置下可达到或超越 ViT。 Demonstrates that CNN-based models can match or exceed ViTs in large-scale settings with proper designs.
验证了对平移、旋转和缩放的鲁棒性，以及在有限数据上的数据效率。 Validates robustness to translation, rotation, and scaling, and data efficiency on limited data.
局限 · Limitations
基于 DCN 的算子在高速下游任务中延迟仍是一个问题。 Latency remains an issue for DCN-based operators in high-speed downstream tasks.
大规模 CNN 仍处于早期发展阶段；InternImage 作为一个起点。 Large-scale CNNs are still in early development; InternImage serves as a starting point.
在 ImageNet 上与顶级 ViT 的性能差距（约 1 个点）可能源于私有数据差异。 Performance gap with top ViTs on ImageNet (~1 point) may be due to private data discrepancy.
由于 DCN 算子开销，吞吐量低于 ConvNeXt。 Throughput is lower than ConvNeXt due to DCN operator overhead.
从有限搜索空间导出的缩放规则可能并非对所有规模最优。 Scaling rules derived from limited search space may not be optimal for all scales.
论文章节 · Sections（共 27）
摘要Abstract
1 引言1 Introduction
2 相关工作2 Related Work
3 提出方法3 Proposed Method
3.1 可变形卷积 v33.1 Deformable Convolution v3
3.2 InternImage 模型3.2 InternImage Model
4 实验4 Experiment
4.1 图像分类4.1 Image Classification
4.2 目标检测4.2 Object Detection
4.3 语义分割4.3 Semantic Segmentation
4.4 消融实验4.4 Ablation Study
5 结论与局限5 Conclusion & Limitations
附录 A 详细训练设置Appendix A Detailed Training Settings
A.1 骨干网络对比设置A.1 Settings for Backbone-Level Comparison
A.2 系统级对比设置A.2 Settings for System-Level Comparison
B.1 模型堆叠B.1 Model Stacking
B.2 模型缩放B.2 Model Scaling
B.3 卷积核大小B.3 Kernel Size
C.1 分类C.1 Classification
C.2 目标检测C.2 Object Detection
C.3 语义分割C.3 Semantic Segmentation
附录 D 吞吐量分析Appendix D Throughput Analysis
附录 E ImageNet 鲁棒性评估Appendix E Robustness Evaluation on ImageNet
E.1 平移不变性E.1 Translation Invariance
E.2 旋转不变性E.2 Rotation Invariance
E.3 缩放不变性E.3 Scaling Invariance
E.4 模型对数据规模的依赖程度E.4 How Hungry the Model is for Data Scale?