EDEN: Evolutionary Deep Networks for Efficient Machine Learning

• Main Authors: Dufourq, Emmanuel, Bassett, Bruce A
• Format: Journal Article
• Language: English
• Published: 26-09-2017
• Subjects: Computer Science - Learning; Computer Science - Neural and Evolutionary Computing; Statistics - Machine Learning
• DOI: 10.48550/arxiv.1709.09161

Deep neural networks continue to show improved performance with increasing
depth, an encouraging trend that implies an explosion in the possible
permutations of network architectures and hyperparameters for which there is
little intuitive guidance. To address this increasing complexity, we propose
Evolutionary DEep Networks (EDEN), a computationally efficient
neuro-evolutionary algorithm which interfaces to any deep neural network
platform, such as TensorFlow. We show that EDEN evolves simple yet successful
architectures built from embedding, 1D and 2D convolutional, max pooling and
fully connected layers along with their hyperparameters. Evaluation of EDEN
across seven image and sentiment classification datasets shows that it reliably
finds good networks -- and in three cases achieves state-of-the-art results --
even on a single GPU, in just 6-24 hours. Our study provides a first attempt at
applying neuro-evolution to the creation of 1D convolutional networks for
sentiment analysis including the optimisation of the embedding layer.