Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

• Published in: Nature communications Vol. 10; no. 1; pp. 5113 - 10
• Main Authors: Mahmoodi, M. R., Prezioso, M., Strukov, D. B.
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 08-11-2019; Nature Publishing Group; Nature Portfolio
• Subjects: 631/378/116/1925; 639/166/987; 639/925/927/1007; Circuits; Energy efficiency; Humanities and Social Sciences; Information theory; Learning algorithms; Machine learning; Memory devices; Memristors; Metal oxides; multidisciplinary; Neural networks; Neurocomputing; Neurons; Science; Science (multidisciplinary); Stochasticity
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-019-13103-7

The key operation in stochastic neural networks, which have become the state-of-the-art approach for solving problems in machine learning, information theory, and statistics, is a stochastic dot-product. While there have been many demonstrations of dot-product circuits and, separately, of stochastic neurons, the efficient hardware implementation combining both functionalities is still missing. Here we report compact, fast, energy-efficient, and scalable stochastic dot-product circuits based on either passively integrated metal-oxide memristors or embedded floating-gate memories. The circuit’s high performance is due to mixed-signal implementation, while the efficient stochastic operation is achieved by utilizing circuit’s noise, intrinsic and/or extrinsic to the memory cell array. The dynamic scaling of weights, enabled by analog memory devices, allows for efficient realization of different annealing approaches to improve functionality. The proposed approach is experimentally verified for two representative applications, namely by implementing neural network for solving a four-node graph-partitioning problem, and a Boltzmann machine with 10-input and 8-hidden neurons.
Providing efficient and scalable specialized hardware for stochastic neural networks remains a challenge. Here, the authors propose a fast, energy-efficient and scalable stochastic dot-product circuit that may use either of two types of memory devices – metal-oxide memristors and floating-gate memories.