Handling emotional speech: a prosody based data augmentation technique for improving neutral speech trained ASR systems

Handling emotional speech: a prosody based data augmentation technique for improving neutral speech trained ASR systems

In this paper, the effect of emotional speech on the performance of neutral speech trained ASR systems is studied. Prosody-modification based data augmentation is explored to compensate the affected ASR performance due to emotional speech. The primary motive is to develop an Telugu ASR system that i...

Full description

Saved in:
Bibliographic Details
Published in:International journal of speech technology Vol. 25; no. 1; pp. 197 - 204
Main Authors: Kammili, Pavan Raju, Ramakrishnam Raju, B. H. V. S., Krishna, A. Sri
Format: Journal Article
Language:English
Published: New York Springer US 01-03-2022
Springer Nature B.V
Subjects:
Artificial Intelligence
Business metrics
Corpus linguistics
Data augmentation
Emotions
Engineering
Fundamental frequency
Linguistics
Parameter modification
Performance degradation
Performance measurement
Prosody
Resonant frequencies
Signal,Image and Speech Processing
Social Sciences
Speaking
Speech
Speech rate
Speech tests
Telugu
Training
Voice recognition
ASR
Data augmentation
Prosody
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, the effect of emotional speech on the performance of neutral speech trained ASR systems is studied. Prosody-modification based data augmentation is explored to compensate the affected ASR performance due to emotional speech. The primary motive is to develop an Telugu ASR system that is least affected by these emotion based intrinsic speaker related acoustic variations. Two factors contributing towards the intrinsic speaker related variability that are focused in this research are the fundamental frequency [ ( F 0 ) or pitch] and the speaking rate variations. To simulate ASR task, we performed the training of our ASR system on neutral speech and tested it for data from emotional as well as neutral speech. Compared to the performance metrics of neutral speech at testing stage, emotional speech performance metrics are extremely degraded. This performance degradation is observed due to the difference in the prosody and speaking rate parameters of neutral and emotional speech. To overcome this performance degradation problem, prosody and speaking rate parameters are varied and modified to create the newer augmented versions of the training data. The original and augmented versions of the training data are pooled together and re-trained in order to capture greater emotion-specific variations. For the Telugu ASR experiments, we used Microsoft speech corpus for Indian languages(MSC-IL) for training neutral speech and Indian Institute of Technology Kharagpur Simulated Emotion Speech Corpus (IITKGP-SESC) for evaluating emotional speech. The basic emotions of anger, happiness and sad are considered for evaluation along with neutral speech.
ISSN:1381-2416
1572-8110
DOI:10.1007/s10772-021-09897-x