Learning Actionlet Ensemble for 3D Human Action Recognition

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 36; no. 5; pp. 914 - 927
• Main Authors: Wang, Jiang, Liu, Zicheng, Wu, Ying, Yuan, Junsong
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01-05-2014; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Actigraphy - methods; Algorithms; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Computer vision; Data processing. List processing. Character string processing; Devices; Exact sciences and technology; Feature extraction; Gesture; Hidden Markov models; Human; Human motion; Humans; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Imaging, Three-Dimensional - methods; Invariants; Joints; Memory organisation. Data processing; Movement - physiology; Noise; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Photography - methods; Recognition; Reproducibility of Results; Robustness; Sensitivity and Specificity; Software; Subtraction Technique; Temporal logic; Three dimensional; Three-dimensional displays; Video analysis; Video Recording - methods; Whole Body Imaging - methods; Action recognition; Kinect; human pose; human-object interaction; ensemble method; Invariant; Computer vision; Action; Data analysis; Motion estimation; Video cameras; Modeling; Aggregate model; Multiple view; Behavioral analysis; Supervised learning; Natural interface; Scene analysis; Body movement; Alignment defect
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2013.198

Human action recognition is an important yet challenging task. Human actions usually involve human-object interactions, highly articulated motions, high intra-class variations, and complicated temporal structures. The recently developed commodity depth sensors open up new possibilities of dealing with this problem by providing 3D depth data of the scene. This information not only facilitates a rather powerful human motion capturing technique, but also makes it possible to efficiently model human-object interactions and intra-class variations. In this paper, we propose to characterize the human actions with a novel actionlet ensemble model, which represents the interaction of a subset of human joints. The proposed model is robust to noise, invariant to translational and temporal misalignment, and capable of characterizing both the human motion and the human-object interactions. We evaluate the proposed approach on three challenging action recognition datasets captured by Kinect devices, a multiview action recognition dataset captured with Kinect device, and a dataset captured by a motion capture system. The experimental evaluations show that the proposed approach achieves superior performance to the state-of-the-art algorithms.