Accent Conversion with Articulatory Representations

• Main Authors: Siriwardena, Yashish M, Swedlow, Nathan, Howard, Audrey, Gitterman, Evan, Darcy, Dan, Espy-Wilson, Carol, Fanelli, Andrea
• Format: Journal Article
• Language: English
• Published: 09-06-2024
• DOI: 10.48550/arxiv.2406.05947

Conversion of non-native accented speech to native (American) English has a
wide range of applications such as improving intelligibility of non-native
speech. Previous work on this domain has used phonetic posteriograms as the
target speech representation to train an acoustic model which is then used to
extract a compact representation of input speech for accent conversion. In this
work, we introduce the idea of using an effective articulatory speech
representation, extracted from an acoustic-to-articulatory speech inversion
system, to improve the acoustic model used in accent conversion. The idea to
incorporate articulatory representations originates from their ability to well
characterize accents in speech. To incorporate articulatory representations
with conventional phonetic posteriograms, a multi-task learning based acoustic
model is proposed. Objective and subjective evaluations show that the use of
articulatory representations can improve the effectiveness of accent
conversion.