We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.
核心贡献 · Key contributions
提出使用感知损失函数训练前馈网络，结合逐像素损失和感知损失。 Proposes perceptual loss functions for training feed-forward networks, combining per-pixel and perceptual losses.
实现实时风格迁移，质量与基于优化的方法相当，但速度快三个数量级。 Achieves real-time style transfer with quality comparable to optimization-based methods but three orders of magnitude faster.
证明感知损失在单图像超分辨率中改善细节重建。 Demonstrates that perceptual loss improves fine detail reconstruction in single-image super-resolution.
使用预训练的 VGG-16 网络作为固定损失网络，定义特征和风格重建损失。 Uses a pretrained VGG-16 network as a fixed loss network to define feature and style reconstruction losses.
表明感知损失实现从损失网络到变换网络的语义知识迁移。 Shows that perceptual loss enables semantic knowledge transfer from the loss network to the transformation network.
局限 · Limitations
感知损失可能在超分辨率中引入交叉影线伪影，损害 PSNR 和 SSIM。 Perceptual loss may introduce cross-hatch artifacts in super-resolution, harming PSNR and SSIM.
方法需要预训练的损失网络，限制了在无合适预训练模型任务上的适用性。 Method requires a pretrained loss network, limiting applicability to tasks without suitable pretrained models.
风格迁移网络按风格训练，不重新训练无法泛化到未见风格。 Style transfer networks are trained per style, not generalizing to unseen styles without retraining.
超分辨率结果定性评估；定量指标如 PSNR 低于逐像素损失方法。 Super-resolution results are evaluated qualitatively; quantitative metrics like PSNR are lower than per-pixel loss methods.
方法性能依赖于损失网络中层的选择，需要手动调整。 The method's performance depends on the choice of layers in the loss network, requiring manual tuning.