Dialog Act Guided Contextual Adapter for Personalized Speech Recognition

• Published in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 1 - 5
• Main Authors: Chang, Feng-Ju, Muniyappa, Thejaswi, Sathyendra, Kanthashree Mysore, Wei, Kai, Strimel, Grant P., McGowan, Ross
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04-06-2023
• Subjects: Acoustics; Adaptation models; Context modeling; Contextual adapter; dialog act; early-late fusion; Error analysis; personalized speech recognition; RNN-T; Semantics; Signal processing; Speech processing
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP49357.2023.10094707

Personalization in multi-turn dialogs has been a long standing challenge for end-to-end automatic speech recognition (E2E ASR) models. Recent work on contextual adapters has tackled rare word recognition using user catalogs. This adaptation, however, does not incorporate an important cue, the dialog act, which is available in a multi-turn dialog scenario. In this work, we propose a dialog act guided contextual adapter network. Specifically, it leverages dialog acts to select the most relevant user catalogs and creates queries based on both - the audio as well as the semantic relationship between the carrier phrase and user catalogs to better guide the contextual biasing. On industrial voice assistant datasets, our model outperforms both the baselines - dialog act encoder-only model, and the contextual adaptation, leading to the most improvement over the no-context model: 58% average relative word error rate reduction (WERR) in the multi-turn dialog scenario, in comparison to the prior-art contextual adapter, which has achieved 39% WERR over the no-context model.