E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learning

• Published in: 2022 International Joint Conference on Neural Networks (IJCNN) pp. 1 - 8
• Main Authors: Gormez, Alperen, Dasari, Venkat R., Koyuncu, Erdem
• Format: Conference Proceeding
• Language: English
• Published: IEEE 18-07-2022
• Subjects: class means; Computational efficiency; early exit; Image edge detection; Neural networks; Performance evaluation; Supervised learning; Training; Wireless communication
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN55064.2022.9891952

State-of-the-art neural networks with early exit mechanisms often need considerable amount of training and fine tuning to achieve good performance with low computational cost. We propose a novel early exit technique, Early Exit Class Means (E 2 CM), based on class means of samples. Unlike most existing schemes, ECM does not require gradient-based training of internal classifiers and it does not modify the base network by any means. This makes it particularly useful for neural network training in low-power devices, as in wireless edge networks. We evaluate the performance and overheads of E 2 CM over various base neural networks such as MobileNetV3, EfficientNet, ResNet, and datasets such as CIFAR-100, ImageNet, and KMNIST. Our results show that, given a fixed training time budget, E 2 CM achieves higher accuracy as compared to existing early exit mechanisms. Moreover, if there are no limitations on the training time budget, E 2 CM can be combined with an existing early exit scheme to boost the latter's performance, achieving a better trade-off between computational cost and network accuracy. We also show that E 2 CM can be used to decrease the computational cost in unsupervised learning tasks.