True Feature Distillation for Compression of GANs

The ResNet architecture based Pix2Pix model has a high number of dependent operations owing to the numerous ResNet Blocks. Thus, during the training and inference of the model, a high latency is observed even with powerful GPUs. More layers reduce the number of independent operations, thus underutil...

Full description

Saved in:

Bibliographic Details
Published in:	2024 International Conference on Emerging Smart Computing and Informatics (ESCI) pp. 1 - 7
Main Authors:	Ghadekar, Premanand, Sadany, Raghav, Kale, Ishaan, Kulkarni, Nikhil, Korate, Omkar
Format:	Conference Proceeding
Language:	English
Published:	IEEE 05-03-2024
Subjects:	Computational modeling Computer architecture Distortion Filters Generative Adversarial Network (GAN) Generative adversarial networks GPU Knowledge Distillation Parallel processing Pix2Pix ResNet Training
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The ResNet architecture based Pix2Pix model has a high number of dependent operations owing to the numerous ResNet Blocks. Thus, during the training and inference of the model, a high latency is observed even with powerful GPUs. More layers reduce the number of independent operations, thus underutilizing the high parallelism available in powerful GPUs. The proposed method of True Feature Distillation (TFD) implements GAN compression based on reducing layers for compression along with reducing the filters for increasing the parallel operations in the model and in turn increasing the usage of parallelism of powerful GPUs. Through TFD, a reduced latency is observed with negligible change in performance of the model even with a reduced number of layers in the proposed model. A ResNet based Pix2Pix model is used as an example for training through the proposed method via edges2shoes dataset. Speedup with respect to complete Pix2Pix achieved is 1.58x on the NVIDIA Tesla V100.
DOI:	10.1109/ESCI59607.2024.10497356