A Method of Real-Time Dynamic Measurement of a Speaker’s Emotional State from a Speech Waveform

• Published in: Measurement techniques Vol. 64; no. 4; pp. 319 - 327
• Main Authors: Savchenko, L. V., Savchenko, A. V.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-07-2021; Springer; Springer Nature B.V
• Subjects: Acoustic measurement; Acoustic Measurements; Analytical Chemistry; Characterization and Evaluation of Materials; Emotional factors; Information theory; Measurement Science and Instrumentation; Perturbation; Physical Chemistry; Physics; Physics and Astronomy; Real time; Speech; System effectiveness; Time measurement; Waveforms; automatic speech processing; measurement of dynamics in emotional state; speech acoustics; small-sample-size problem; information-theoretic approach; acoustic measurements
• ISSN: 0543-1972; 1573-8906
• DOI: 10.1007/s11018-021-01935-z

The problems of implementing systems with a voice interface for remote service of the population are examined. The effectiveness of such systems can be enhanced by automatic analysis of the changes of the emotional state of the user during dialogue. In order to do real-time measurements of the index of the dynamics of the emotional state, it is proposed to use the effect of the sound (phonetic) variability of speech of the user at observation intervals that are of small duration (fractions of a minute). Based on an information-theoretic approach, a method was developed for acoustic measurements of the dynamics of the emotional state under conditions of small samples, using a scale-invariant measure of the variations of the speech waveform in the frequency domain. An example of the practical instantiation of this method in real-time conditions is examined. It is shown that in this case the delay in obtaining measurement results does not exceed 10–20 s. The results of experimental studies confirmed the rapid response of the proposed method and its sensitivity to modifications of the dynamics of the emotional state under the effect of external perturbations. The developed method can be used to introduce automated monitoring of the quality of voice samples of users of the unified biometric systems. Also, the method will be useful to enhance security by noncontact detection of potentially dangerous persons with short-term disturbance of the psychoemotional state.