iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis

• Main Authors: Liu, Xin, Shi, Henglin, Chen, Haoyu, Yu, Zitong, Li, Xiaobai, Zhaoz?, Guoying
• Format: Journal Article
• Language: English
• Published: 01-07-2021
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2107.00285

We introduce a new dataset for the emotional artificial intelligence
research: identity-free video dataset for Micro-Gesture Understanding and
Emotion analysis (iMiGUE). Different from existing public datasets, iMiGUE
focuses on nonverbal body gestures without using any identity information,
while the predominant researches of emotion analysis concern sensitive
biometric data, like face and speech. Most importantly, iMiGUE focuses on
micro-gestures, i.e., unintentional behaviors driven by inner feelings, which
are different from ordinary scope of gestures from other gesture datasets which
are mostly intentionally performed for illustrative purposes. Furthermore,
iMiGUE is designed to evaluate the ability of models to analyze the emotional
states by integrating information of recognized micro-gesture, rather than just
recognizing prototypes in the sequences separately (or isolatedly). This is
because the real need for emotion AI is to understand the emotional states
behind gestures in a holistic way. Moreover, to counter for the challenge of
imbalanced sample distribution of this dataset, an unsupervised learning method
is proposed to capture latent representations from the micro-gesture sequences
themselves. We systematically investigate representative methods on this
dataset, and comprehensive experimental results reveal several interesting
insights from the iMiGUE, e.g., micro-gesture-based analysis can promote
emotion understanding. We confirm that the new iMiGUE dataset could advance
studies of micro-gesture and emotion AI.