Speaker Activity Driven Neural Speech Extraction

• Published in: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 6099 - 6103
• Main Authors: Delcroix, Marc, Zmolikova, Katerina, Ochiai, Tsubasa, Kinoshita, Keisuke, Nakatani, Tomohiro
• Format: Conference Proceeding
• Language: English
• Published: IEEE 06-06-2021
• Subjects: Conferences; Error analysis; Meeting recognition; Neural network; Neural networks; Signal processing; Speaker activity; Speech enhancement; Speech extraction; Speech recognition; Visualization
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP39728.2021.9414998

Target speech extraction, which extracts the speech of a target speaker in a mixture given auxiliary speaker clues, has recently received increased interest. Various clues have been investigated such as pre-recorded enrollment utterances, direction information, or video of the target speaker. In this paper, we explore the use of speaker activity information as an auxiliary clue for single-channel neural network-based speech extraction. We propose a speaker activity driven speech extraction neural network (ADEnet) and show that it can achieve performance levels competitive with enrollment-based approaches, without the need for pre-recordings. We further demonstrate the potential of the proposed approach for processing meeting-like recordings, where the speaker activity is obtained from a diarization system. We show that this simple yet practical approach can successfully extract speakers after diarization, which results in improved ASR performance, especially in high overlapping conditions, with a relative word error rate reduction of up to 25%.