通过对比聚类分配进行视觉特征的无监督学习

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

玛蒂尔德·卡龙 Mathilde Caron · Meta AI · 2020-06-17 · arXiv:2006.09882 ↗ · 被引 5059

打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→

摘要 · Abstract

无监督图像表示已显著缩小了与有监督预训练的差距，尤其是近期对比学习方法的成就。这些对比方法通常在线工作，依赖大量显式的成对特征比较，计算上具有挑战性。本文提出在线算法 SwAV，利用对比方法的优势而无需计算成对比较。具体而言，我们的方法在聚类数据的同时，强制同一图像不同增强（或视图）产生的聚类分配之间的一致性，而非像对比学习那样直接比较特征。简单来说，我们使用交换预测机制，从一个视图的表示预测另一个视图的聚类分配。我们的方法可在大批量和小批量下训练，并能扩展到无限数据。与之前的对比方法相比，我们的方法更节省内存，因为它不需要大型内存库或特殊动量网络。此外，我们还提出新的数据增强策略 multi-crop，使用不同分辨率的视图混合替代两个全分辨率视图，而不显著增加内存或计算需求。我们通过 ResNet-50 在 ImageNet 上达到 75.3%的 top-1 准确率，并在所有考虑的迁移任务上超越有监督预训练，验证了我们的发现。

Unsupervised image representations have significantly reduced the gap with supervised pretraining, notably with the recent achievements of contrastive learning methods. These contrastive methods typically work online and rely on a large number of explicit pairwise feature comparisons, which is computationally challenging. In this paper, we propose an online algorithm, SwAV, that takes advantage of contrastive methods without requiring to compute pairwise comparisons. Specifically, our method simultaneously clusters the data while enforcing consistency between cluster assignments produced for different augmentations (or views) of the same image, instead of comparing features directly as in contrastive learning. Simply put, we use a swapped prediction mechanism where we predict the cluster assignment of a view from the representation of another view. Our method can be trained with large and small batches and can scale to unlimited amounts of data. Compared to previous contrastive methods, our method is more memory efficient since it does not require a large memory bank or a special momentum network. In addition, we also propose a new data augmentation strategy, multi-crop, that uses a mix of views with different resolutions in place of two full-resolution views, without increasing the memory or compute requirements much. We validate our findings by achieving 75.3% top-1 accuracy on ImageNet with ResNet-50, as well as surpassing supervised pretraining on all the considered transfer tasks.

核心贡献 · Key contributions

• 提出 SwAV，一种在线聚类算法，避免显式的成对特征比较。
  Proposes SwAV, an online clustering algorithm that avoids explicit pairwise feature comparisons.
• 引入交换预测机制，在不同视图间预测聚类分配。
  Introduces a swapped prediction mechanism that predicts cluster assignments across different views.
• 开发多裁剪增强策略，使用混合分辨率视图而不增加内存或算力。
  Develops multi-crop augmentation strategy using mixed-resolution views without extra memory or compute.
• 在 ImageNet 上用 ResNet-50 达到 75.3% top-1 准确率，在迁移任务上超越监督预训练。
  Achieves 75.3% top-1 accuracy on ImageNet with ResNet-50, surpassing supervised pretraining on transfer tasks.
• 证明在线聚类可扩展到大数据集并适用于小批量。
  Demonstrates online clustering scales to large datasets and works with small batches.
• 显示多裁剪持续提升多种自监督方法 2-4% top-1 准确率。
  Shows multi-crop consistently improves multiple self-supervised methods by 2-4% top-1 accuracy.

局限 · Limitations

• SwAV 需要额外的 Sinkhorn-Knopp 迭代计算编码，增加开销。
  SwAV requires additional Sinkhorn-Knopp iterations for code computation, adding overhead.
• 多裁剪可能不适用于监督模型，且需仔细调整裁剪尺寸。
  Multi-crop may not benefit supervised models and requires careful tuning of crop sizes.
• 方法假设每批原型均匀分配，对极小批量可能失效。
  Method assumes equipartition of prototypes in each batch, which may fail for very small batches.
• 性能依赖于原型数量和温度参数，需要调优。
  Performance depends on number of prototypes and temperature parameter, requiring tuning.
• 在线聚类在极端噪声数据上可能不如离线方法稳定。
  Online clustering may not converge as stably as offline methods on extremely noisy data.

论文章节 · Sections（共 15）

• 摘要 Abstract
• 1 引言 1 Introduction
• 2 相关工作 2 Related Work
• 3 方法 3 Method
• 3.1 在线聚类 3.1 Online clustering
• 3.2 多裁剪：用小图像增强视图 3.2 Multi-crop: Augmenting views with smaller images
• 4 主要结果 4 Main Results
• 4.1 在 ImageNet 上评估无监督特征 4.1 Evaluating the unsupervised features on ImageNet
• 4.2 将无监督特征迁移到下游任务 4.2 Transferring unsupervised features to downstream tasks
• 4.3 小批量训练 4.3 Training with small batches
• 5.1 基于聚类的自监督学习 5.1 Clustering-based self-supervised learning
• 5.2 将多裁剪策略应用于不同方法 5.2 Applying the multi-crop strategy to different methods
• 5.3 更长训练的影响 5.3 Impact of longer training
• 5.4 在大型未整理数据集上的无监督预训练 5.4 Unsupervised pretraining on a large uncurated dataset
• 6 讨论 6 Discussion

阅读逐段中英对照全文 →