Using Winning Lottery Tickets in Transfer Learning for Convolutional Neural Networks

• Published in: 2019 International Joint Conference on Neural Networks (IJCNN) pp. 1 - 8
• Main Authors: Soelen, Ryan Van, Sheppard, John W.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-07-2019
• Subjects: Computer architecture; Convergence; Convolutional neural networks; Kernel; Neurons; Training
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN.2019.8852405

Neural network pruning can be an effective method for creating more efficient networks without incurring a significant penalty in accuracy. It has been shown that the topology induced by pruning after training can be used to re-train a network from scratch on the same data set, with comparable or better performance. In the context of convolutional neural networks, we build on this work to show that not only can networks be pruned to 10% of their original parameters, but that these sparse networks can also be re-trained on similar data sets with only a slight reduction in accuracy. We use the Lottery Ticket Hypothesis as the basis for our pruning method and discuss how this method can be an alternative to transfer learning, with positive initial results. This paper lays the groundwork for a transfer learning method that reduces the original network to its essential connections and does not require freezing entire layers.