Neural image caption generator based on crossbar array design of memristor module

• Published in: Neurocomputing (Amsterdam) Vol. 560; p. 126766
• Main Authors: Yu, Yongbin, Yang, Daijin, Tang, Qian, Wang, Xiangxiang, Yang, Nijing, Cheng, Man, Zhong, Yuanjingyang, Adu, Kwabena, Favour, Ekong
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-12-2023
• Subjects: Convolutional Neural Network (CNN); Cross-model; Image captioning; Long Short-term Memory network (LSTM); Memristor module; Long Short-term Memory network (LSTM); Memristor module; Cross-model; Image captioning; Convolutional Neural Network (CNN)
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2023.126766

The crossbar array design method based on a single memristor provides an excessive writing error and cannot achieve the network’s requirements for the generation of image captions. In order to solve this problem, this paper highlights the following techniques. (1) A crossbar array of memristor modules is built, and the writing error is theoretically reduced to a maximum of 0.0619‰. (2) A method of linearly mapping the weights of neural networks to memristance values is proposed, and a memristive neural network based on a memristor module is designed. (3) A neural image caption generator based on VGG-16-LSTM is designed in light of the crossbar array of the memristor module. The design can achieve a maximum of 99.38% of the reference network in BLEU scoring performance. The performance of neural networks based on memristor modules under different input conditions is discussed, and the paper’s best hardware design scheme, named it-10, is proposed. The it-8 simulation can obtain 97.37% of the reference network in BLEU scoring performance, and it requires far less demanding input conditions. Moreover, this paper finds a way to design a novel crossbar array based on the memristor module and make a breakthrough in memristive neural network solution for the cross-modal image caption generation task.