We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.
核心贡献 · Key contributions
提出条件生成对抗网络作为图像到图像翻译的通用框架，同时学习映射和损失函数。 Proposes conditional GANs as a general-purpose framework for image-to-image translation, learning both mapping and loss function.
在标签到照片、边缘到照片、着色等多种任务上展示有效性，无需任务特定工程。 Demonstrates effectiveness across diverse tasks: label-to-photo, edge-to-photo, colorization, etc., without task-specific engineering.
引入带跳跃连接的 U-Net 生成器以传递低级信息，提升输出质量。 Introduces U-Net generator with skip connections to shuttle low-level information, improving output quality.
设计 PatchGAN 判别器在图像块尺度惩罚结构，实现高频细节建模。 Designs PatchGAN discriminator that penalizes structure at patch scale, enabling high-frequency detail modeling.
展示 L1 损失与对抗损失结合可减少模糊并提升色彩丰富度。 Shows that L1 loss combined with adversarial loss reduces blurring and improves colorfulness.
发布 pix2pix 软件，促进社区广泛采用和创意应用。 Releases pix2pix software, enabling wide community adoption and creative applications.
局限 · Limitations
条件 GAN 产生的随机性有限；dropout 噪声仅带来微小输出变化。 Conditional GANs produce limited stochasticity; dropout noise yields only minor output variation.
方法需要成对训练数据；不适用于未配对的图像翻译任务。 Method requires paired training data; not applicable to unpaired image translation tasks.
PatchGAN 假设局部像素独立，可能不适用于全局结构任务。 PatchGAN assumes local pixel independence, which may not hold for global structure tasks.
在语义分割等视觉任务上的表现不如简单的 L1 回归。 Performance on vision tasks like semantic segmentation is inferior to simple L1 regression.
全图 GAN（ImageGAN）因参数增多和训练困难而表现不佳。 Full-image GAN (ImageGAN) underperforms due to increased parameters and training difficulty.
论文章节 · Sections（共 16）
摘要Abstract
1 引言1 Introduction
2 相关工作2 Related work
3 方法3 Method
3.1 目标函数3.1 Objective
3.2 网络架构3.2 Network architectures
3.3 优化与推理3.3 Optimization and inference
4 实验4 Experiments
4.1 评估指标4.1 Evaluation metrics
4.2 目标函数分析4.2 Analysis of the objective function
4.3 生成器架构分析4.3 Analysis of the generator architecture
4.4 从 PixelGAN 到 PatchGAN 再到 ImageGAN4.4 From PixelGANs to PatchGANs to ImageGANs