SMG: A Micro-gesture Dataset Towards Spontaneous Body Gestures for Emotional Stress State Analysis

• Published in: International journal of computer vision Vol. 131; no. 6; pp. 1346 - 1366
• Main Authors: Chen, Haoyu, Shi, Henglin, Liu, Xin, Li, Xiaobai, Zhao, Guoying
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-06-2023; Springer; Springer Nature B.V
• Subjects: Artificial Intelligence; Benchmarks; Computer Imaging; Computer Science; Computer vision; Conveying; Datasets; Deep learning; Emotional factors; Emotions; Image Processing and Computer Vision; Machine vision; Nonverbal communication; Pattern Recognition; Pattern Recognition and Graphics; Psychological aspects; Psychology; Recognition; Source code; Statistical analysis; Stress (Psychology); Stress analysis; Stress state; Vision; Deep learning; Micro-gestures; Emotion recognition; Affective computing; Statistical modeling; Gesture recognition
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-023-01761-6

We explore using body gestures for hidden emotional state analysis. As an important non-verbal communicative fashion, human body gestures are capable of conveying emotional information during social communication. In previous works, efforts have been made mainly on facial expressions, speech, or expressive body gestures to interpret classical expressive emotions. Differently, we focus on a specific group of body gestures, called micro-gestures (MGs), used in the psychology research field to interpret inner human feelings. MGs are subtle and spontaneous body movements that are proven, together with micro-expressions, to be more reliable than normal facial expressions for conveying hidden emotional information. In this work, a comprehensive study of MGs is presented from the computer vision aspect, including a novel spontaneous micro-gesture (SMG) dataset with two emotional stress states and a comprehensive statistical analysis indicating the correlations between MGs and emotional states. Novel frameworks are further presented together with various state-of-the-art methods as benchmarks for automatic classification, online recognition of MGs, and emotional stress state recognition. The dataset and methods presented could inspire a new way of utilizing body gestures for human emotion understanding and bring a new direction to the emotion AI community. The source code and dataset are made available:
https://github.com/mikecheninoulu/SMG
.