DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow convergence and limited feature spatial resolution, due to the limitation of Transformer attention modules in processing image feature maps. To mitigate these issues, we proposed Deformable DETR, whose attention modules only attend to a small set of key sampling points around a reference. Deformable DETR can achieve better performance than DETR (especially on small objects) with 10 times less training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach. Code is released at https://github.com/fundamentalvision/Deformable-DETR.
核心贡献 · Key contributions
提出可变形 DETR，其注意力模块仅关注参考点周围的一小组关键采样点。 Proposes Deformable DETR with deformable attention modules that attend to a small set of key sampling points.
相比 DETR，性能更优（尤其对小物体），训练周期减少 10 倍。 Achieves better performance than DETR, especially on small objects, with 10× fewer training epochs.
引入多尺度可变形注意力，无需 FPN 即可聚合多尺度特征。 Introduces multi-scale deformable attention to aggregate multi-scale features without FPN.
探索迭代边界框细化和两阶段可变形 DETR 以提升检测性能。 Explores iterative bounding box refinement and two-stage Deformable DETR for improved detection.
将编码器注意力的计算复杂度从二次降至线性（相对于空间尺寸）。 Reduces computational complexity from quadratic to linear in spatial size for encoder attention.
在 COCO 基准上通过大量实验证明了方法的有效性。 Demonstrates effectiveness on COCO benchmark with extensive experiments.
局限 · Limitations
可变形注意力仍涉及无序内存访问，比传统卷积稍慢。 Deformable attention still involves unordered memory access, slightly slower than traditional convolution.
性能依赖于采样点数 K 和注意力头数 M 等超参数。 Performance relies on hyperparameters like number of sampling points K and attention heads M.
两阶段变体增加了复杂性，且不一定总是优于单阶段。 Two-stage variant adds complexity and may not always improve over single-stage.
方法仅在 COCO 上评估，未验证在其他领域的泛化性。 Method is evaluated only on COCO; generalization to other domains is not verified.