Stacked Generative Adversarial Networks

• Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1866 - 1875
• Main Authors: Xun Huang, Yixuan Li, Poursaeed, Omid, Hopcroft, John, Belongie, Serge
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-07-2017
• Subjects: Data models; Entropy; Gallium nitride; Generators; Image reconstruction; Training
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2017.202

In this paper, we propose a novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network. Our model consists of a top-down stack of GANs, each learned to generate lower-level representations conditioned on higher-level representations. A representation discriminator is introduced at each feature hierarchy to encourage the representation manifold of the generator to align with that of the bottom-up discriminative network, leveraging the powerful discriminative representations to guide the generative model. In addition, we introduce a conditional loss that encourages the use of conditional information from the layer above, and a novel entropy loss that maximizes a variational lower bound on the conditional entropy of generator outputs. We first train each stack independently, and then train the whole model end-to-end. Unlike the original GAN that uses a single noise vector to represent all the variations, our SGAN decomposes variations into multiple levels and gradually resolves uncertainties in the top-down generative process. Based on visual inspection, Inception scores and visual Turing test, we demonstrate that SGAN is able to generate images of much higher quality than GANs without stacking.