Embracing Change: Continual Learning in Deep Neural Networks

• Published in: Trends in cognitive sciences Vol. 24; no. 12; pp. 1028 - 1040
• Main Authors: Hadsell, Raia, Rao, Dushyant, Rusu, Andrei A., Pascanu, Razvan
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01-12-2020
• Subjects: Artificial Intelligence; Humans; Learning; lifelong; Memory; meta-learning; Neural Networks, Computer; non-stationary; non-stationary; memory; lifelong; meta-learning; artificial intelligence
• ISSN: 1364-6613; 1879-307X
• DOI: 10.1016/j.tics.2020.09.004

Artificial intelligence research has seen enormous progress over the past few decades, but it predominantly relies on fixed datasets and stationary environments. Continual learning is an increasingly relevant area of study that asks how artificial systems might learn sequentially, as biological systems do, from a continuous stream of correlated data. In the present review, we relate continual learning to the learning dynamics of neural networks, highlighting the potential it has to considerably improve data efficiency. We further consider the many new biologically inspired approaches that have emerged in recent years, focusing on those that utilize regularization, modularity, memory, and meta-learning, and highlight some of the most promising and impactful directions.
Modern machine learning excels at training powerful models from fixed datasets and stationary environments, often exceeding human-level ability.Yet, these models fail to emulate the process of human learning, which is efficient, robust, and able to learn incrementally, from sequential experience in a non-stationary world.Insights into this limitation can be gleaned from the nature of neural network optimization, which implies that continual learning techniques could radically improve deep learning as well as open the door to new application areas.Promising approaches for continual learning can be found at the most granular level, with gradient-based methods, as well as at the architectural level, with modular and memory-based approaches. We also consider meta-learning as a potentially important direction.