One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques.
核心贡献 · Key contributions
提出频谱归一化，通过约束判别器的 Lipschitz 常数来稳定 GAN 训练。 Proposes spectral normalization to stabilize GAN training by constraining the Lipschitz constant of the discriminator.
证明频谱归一化计算量小且易于集成到现有 GAN 实现中。 Demonstrates that spectral normalization is computationally light and easy to integrate into existing GAN implementations.
表明频谱归一化允许判别器使用更多特征，避免权重归一化导致的秩亏缺问题。 Shows that spectral normalization allows the discriminator to use more features compared to weight normalization, avoiding rank deficiency.
在 CIFAR-10、STL-10 和 ImageNet 上获得优于或相当于先前稳定技术的图像质量。 Achieves better or comparable image quality on CIFAR-10, STL-10, and ImageNet relative to previous stabilization techniques.
提供理论分析，将频谱归一化与判别器的 Lipschitz 连续性和正则化联系起来。 Provides theoretical analysis linking spectral normalization to Lipschitz continuity and regularization of the discriminator.
展示了对激进学习率和动量参数的鲁棒性，在此类设置中优于 WGAN-GP。 Demonstrates robustness to aggressive learning rates and momentum parameters, outperforming WGAN-GP in such settings.
局限 · Limitations
该方法仅在图像生成任务上评估，未探索其他数据模态的有效性。 The method is evaluated only on image generation tasks; its effectiveness on other data modalities is not explored.
频谱归一化需要幂迭代估计谱范数，增加了轻微的计算开销。 Spectral normalization requires power iteration to estimate the spectral norm, adding slight computational overhead.
研究未提供频谱归一化在所有 GAN 设置中最优性的理论保证。 The study does not provide a theoretical guarantee for the optimality of spectral normalization in all GAN settings.
未在 ImageNet 之外的大规模数据集（如视频或 3D 数据）上测试性能。 Performance on very large-scale datasets beyond ImageNet (e.g., video or 3D data) is not tested.
该方法与其他正则化技术（如批归一化）的相互作用未充分研究。 The method's interaction with other regularization techniques (e.g., batch normalization) is not fully investigated.
论文章节 · Sections（共 11）
摘要Abstract
1 引言1 Introduction
2 方法2 Method
2.1 谱归一化2.1 Spectral Normalization
2.2 谱范数σ(W)的快速近似2.2 Fast Approximation of the Spectral Norm σ(W)𝜎𝑊\sigma(W)
2.3 谱归一化权重的梯度分析2.3 Gradient analysis of the spectrally normalized weights
3 谱归一化与其他正则化技术的比较3 Spectral Normalization vs Other Regularization Techniques
4 实验4 Experiments
4.1 CIFAR10 和 STL-10 上的结果4.1 Results on CIFAR10 and STL-10
4.2 ImageNet 上的图像生成4.2 Image Generation on ImageNet