Two-Person Graph Convolutional Network for Skeleton-Based Human Interaction Recognition

• Published in: IEEE transactions on circuits and systems for video technology Vol. 33; no. 7; pp. 3333 - 3342
• Main Authors: Li, Zhengcen, Li, Yueran, Tang, Linlin, Zhang, Tong, Su, Jingyong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-07-2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: action recognition; Artificial neural networks; Convolution; Correlation; Feature extraction; graph convolutional networks; Human activity recognition; Joints; Labeling; Recognition; Skeleton; skeleton topology; Skeleton-based interaction recognition; Topology; Videos
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2022.3232373

Graph convolutional networks (GCNs) have been the predominant methods in skeleton-based human action recognition, including human-human interaction recognition. However, when dealing with interaction sequences, current GCN-based methods simply split the two-person skeleton into two discrete graphs and perform graph convolution separately as done for single-person action classification. Such operations ignore rich interactive information and hinder effective spatial inter-body relationship modeling. To overcome the above shortcoming, we introduce a novel unified two-person graph to represent inter-body and intra-body correlations between joints. Experimental results show accuracy improvements in recognizing both interactions and individual actions when utilizing the proposed two-person graph topology. In addition, several graph labeling strategies are designed to supervise the model to learn discriminant spatial-temporal interactive features. Finally, we propose a two-person graph convolutional network (2P-GCN). Our model outperforms state-of-the-art methods on four benchmarks of three interaction datasets: SBU, interaction subsets of NTU-RGB+D and NTU-RGB+D 120.