We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches $74.3\%$ top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and $79.6\%$ with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.
核心贡献 · Key contributions
提出 BYOL，一种无需负样本对即可在 ImageNet 线性评估中达到最先进水平的自监督学习方法。 Introduces BYOL, a self-supervised learning method achieving state-of-the-art ImageNet linear evaluation without negative pairs.
证明 BYOL 在半监督和迁移基准上优于先前方法。 Demonstrates BYOL outperforms prior art on semi-supervised and transfer benchmarks.
展示 BYOL 对批次大小和数据增强变化的鲁棒性优于对比方法。 Shows BYOL is more robust to batch size and augmentation changes than contrastive methods.
提供理论假设，解释 BYOL 如何通过预测器和慢速移动目标网络避免坍塌。 Provides theoretical hypothesis on why BYOL avoids collapse via predictor and slow-moving target network.
使用更大 ResNet 在 ImageNet 上达到 79.6% top-1 准确率，缩小与监督学习的差距。 Achieves 79.6% top-1 accuracy on ImageNet with a larger ResNet, narrowing gap to supervised learning.
在 GitHub 上发布实现和预训练模型以促进可重复性。 Releases implementation and pretrained models on GitHub for reproducibility.
局限 · Limitations
BYOL 仍依赖针对视觉的手工数据增强，限制了向其他模态的泛化。 BYOL still relies on handcrafted image augmentations specific to vision, limiting generalization to other modalities.
该方法需要仔细调整目标衰减率和预测器架构以避免坍塌。 The method requires careful tuning of target decay rate and predictor architecture to avoid collapse.
性能依赖于大批次和计算资源（如 512 个 TPU v3 核心）。 Performance depends on large batch sizes and compute resources (e.g., 512 TPU v3 cores).
对避免坍塌的理论理解尚不完整，仅提供经验证据。 Theoretical understanding of collapse avoidance is incomplete; only empirical evidence is provided.
BYOL 可能复制训练数据中的偏见，引发双重用途问题。 BYOL may replicate biases present in training datasets, raising dual-use concerns.