A Real-Time Convolutional Neural Network Based Speech Enhancement for Hearing Impaired Listeners Using Smartphone

• Published in: IEEE access Vol. 7; pp. 78421 - 78433
• Main Authors: Bhat, Gautam S., Shankar, Nikhil, Reddy, Chandan K. A., Panahi, Issa M. S.
• Format: Journal Article
• Language: English
• Published: United States IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Auditory system; Computer architecture; Convolutional neural network (CNN); Convolutional neural networks; hearing aid (HA); Hearing aids; Intelligibility; Learning; log power spectra (LPS); Microprocessors; Neural networks; Noise measurement; Real time; real-time implementation; Real-time systems; smartphone; Smartphones; Speech; Speech enhancement; speech enhancement (SE); Speech processing; real-time implementation; smartphone; speech enhancement (SE); Convolutional neural network (CNN); hearing aid (HA); log power spectra (LPS)
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2922370

This paper presents a Speech Enhancement (SE) technique based on multi-objective learning convolutional neural network to improve the overall quality of speech perceived by Hearing Aid (HA) users. The proposed method is implemented on a smartphone as an application that performs real-time SE. This arrangement works as an assistive tool to HA. A multi-objective learning architecture including primary and secondary features uses a mapping-based convolutional neural network (CNN) model to remove noise from a noisy speech spectrum. The algorithm is computationally fast and has a low processing delay which enables it to operate seamlessly on a smartphone. The steps and the detailed analysis of real-time implementation are discussed. The proposed method is compared with existing conventional and neural network-based SE techniques through speech quality and intelligibility metrics in various noisy speech conditions. The key contribution of this paper includes the realization of CNN SE model on a smartphone processor that works seamlessly with HA. The experimental results demonstrate significant improvements over the state-of-the-art techniques and reflect the usability of the developed SE application in noisy environments.