We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.
核心贡献 · Key contributions
提出动量对比（MoCo）用于无监督视觉表示学习。 Proposes Momentum Contrast (MoCo) for unsupervised visual representation learning.
构建包含队列和动量更新编码器的动态字典，用于对比学习。 Builds a dynamic dictionary with a queue and a momentum-updated encoder for contrastive learning.
在 ImageNet 线性分类上取得有竞争力的结果，并在 7 个检测/分割任务上超越有监督预训练。 Achieves competitive results on ImageNet linear classification and outperforms supervised pre-training on 7 detection/segmentation tasks.
引入打乱批归一化以防止对比学习中的作弊行为。 Introduces shuffled batch normalization to prevent cheating in contrastive learning.
证明 MoCo 可扩展到十亿图像数据集，并缩小了无监督与有监督表示学习之间的差距。 Demonstrates that MoCo scales to billion-image datasets and closes the gap between unsupervised and supervised representation learning.
局限 · Limitations
MoCo 从 ImageNet 到 Instagram-1B 的提升相对较小，表明对更大规模数据的利用有限。 MoCo's improvement from ImageNet to Instagram-1B is relatively small, suggesting limited exploitation of larger-scale data.
预文本任务为简单的实例判别；更高级的预文本任务可能进一步提升性能。 The pretext task is simple instance discrimination; more advanced pretext tasks may further improve performance.