Human action recognition based on estimated weak poses

• Published in: EURASIP journal on advances in signal processing Vol. 2012; no. 1; pp. 1 - 14
• Main Authors: Gong, Wenjuan, Gonzàlez, Jordi, Roca, Francesc Xavier
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 25-07-2012; Springer Nature B.V; BioMed Central Ltd
• Subjects: Engineering; Human; Mathematical models; Quantum Information Technology; Recognition; Representations; Signal,Image and Speech Processing; Spintronics; Temporal logic; Three dimensional; Gaussian process regression; Human action recognition; Bag of words; Human pose estimation
• ISSN: 1687-6180; 1687-6172; 1687-6180
• DOI: 10.1186/1687-6180-2012-162

We present a novel method for human action recognition (HAR) based on estimated poses from image sequences. We use 3D human pose data as additional information and propose a compact human pose representation, called a
weak pose
, in a low-dimensional space while still keeping the most discriminative information for a given pose. With predicted poses from image features, we map the problem from image feature space to pose space, where a Bag of Poses (BOP) model is learned for the final goal of HAR. The BOP model is a modified version of the classical bag of words pipeline by building the vocabulary based on the most representative
weak poses
for a given action. Compared with the standard
k
-means clustering, our vocabulary selection criteria is proven to be more efficient and robust against the inherent challenges of action recognition. Moreover, since for action recognition the ordering of the poses is discriminative, the BOP model incorporates temporal information: in essence, groups of consecutive poses are considered together when computing the vocabulary and assignment. We tested our method on two well-known datasets: HumanEva and IXMAS, to demonstrate that
weak poses
aid to improve action recognition accuracies. The proposed method is scene-independent and is comparable with the state-of-art method.