本文提出了自注意力生成对抗网络（SAGAN），它允许在图像生成任务中进行注意力驱动的长距离依赖建模。传统的卷积 GAN 仅根据低分辨率特征图中的空间局部点生成高分辨率细节。在 SAGAN 中，可以利用所有特征位置的线索生成细节。此外，判别器可以检查图像中远距离部分的高度详细特征是否相互一致。进一步地，最近的研究表明生成器条件化影响 GAN 性能。利用这一见解，我们对 GAN 生成器应用谱归一化，发现这改善了训练动态。所提出的 SAGAN 取得了最先进的结果，在具有挑战性的 ImageNet 数据集上，将最佳公布的 Inception 分数从 36.8 提高到 52.52，并将 Frechet Inception 距离从 27.62 降低到 18.65。注意力层的可视化显示，生成器利用对应于物体形状的邻域，而不是固定形状的局部区域。
In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks. Traditional convolutional GANs generate high-resolution details as a function of only spatially local points in lower-resolution feature maps. In SAGAN, details can be generated using cues from all feature locations. Moreover, the discriminator can check that highly detailed features in distant portions of the image are consistent with each other. Furthermore, recent work has shown that generator conditioning affects GAN performance. Leveraging this insight, we apply spectral normalization to the GAN generator and find that this improves training dynamics. The proposed SAGAN achieves the state-of-the-art results, boosting the best published Inception score from 36.8 to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Visualization of the attention layers shows that the generator leverages neighborhoods that correspond to object shapes rather than local regions of fixed shape.
核心贡献 · Key contributions
提出自注意力生成对抗网络（SAGAN），在图像生成中实现注意力驱动的长程依赖建模。 Proposes Self-Attention Generative Adversarial Network (SAGAN) for attention-driven long-range dependency modeling in image generation.
将谱归一化应用于 GAN 生成器，改善训练动态和稳定性。 Applies spectral normalization to the GAN generator, improving training dynamics and stability.
在 ImageNet 上取得最优结果：Inception 分数 52.52，FID 18.65。 Achieves state-of-the-art results on ImageNet: Inception score 52.52 and FID 18.65.
可视化注意力层，显示生成器利用物体形状邻域而非固定局部区域。 Visualizes attention layers showing generator leverages object-shaped neighborhoods rather than fixed local regions.
证明自注意力与卷积互补，用于特征图中的长程依赖建模。 Demonstrates that self-attention is complementary to convolution for modeling long-range dependencies in feature maps.
局限 · Limitations
实验仅限于 ImageNet 上 128x128 图像生成，未测试更高分辨率下的可扩展性。 Experiments limited to 128x128 image generation on ImageNet; scalability to higher resolutions not tested.
与纯卷积 GAN 相比，自注意力模块增加了计算成本。 Self-attention module increases computational cost compared to pure convolutional GANs.
训练需在 4 个 GPU 上进行两周，资源密集，不利于广泛采用。 Training requires two weeks on 4 GPUs; resource-intensive for broader adoption.
评估指标（IS、FID）存在已知局限，未完全与人类感知对齐。 Evaluation metrics (IS, FID) have known limitations; not fully aligned with human perception.