Perceptual Adversarial Networks for Image-to-Image Transformation

• Published in: IEEE transactions on image processing Vol. 27; no. 8; pp. 4066 - 4079
• Main Authors: Wang, Chaoyue, Xu, Chang, Wanga, Chaohui, Tao, Dacheng
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-08-2018; Institute of Electrical and Electronics Engineers
• Subjects: Artificial Intelligence; Computer Science; Feature extraction; Gallium nitride; Generative adversarial networks; image de-raining; image inpainting; Image resolution; image-to-image transformation; Loss measurement; Semantics; Task analysis; Training
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2018.2836316

In this paper, we propose perceptual adversarial networks (PANs) for image-to-image transformations. Different from existing application driven algorithms, PAN provides a generic framework of learning to map from input images to desired images (Fig. 1), such as a rainy image to its de-rained counterpart, object edges to photos, and semantic labels to a scenes image. The proposed PAN consists of two feed-forward convolutional neural networks: the image transformation network T and the discriminative network D. Besides the generative adversarial loss widely used in GANs, we propose the perceptual adversarial loss, which undergoes an adversarial training process between the image transformation network T and the hidden layers of the discriminative network D. The hidden layers and the output of the discriminative network D are upgraded to constantly and automatically discover the discrepancy between the transformed image and the corresponding ground truth, while the image transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through integrating the generative adversarial loss and the perceptual adversarial loss, D and T can be trained alternately to solve image-to-image transformation tasks. Experiments evaluated on several image-to-image transformation tasks (e.g., image deraining and image inpainting) demonstrate the effectiveness of the proposed PAN and its advantages over many existing works.