Flow-pose Net: an effective two-stream network for fall detection

• Published in: The Visual computer Vol. 39; no. 6; pp. 2305 - 2320
• Main Authors: Fei, Kexin, Wang, Chao, Zhang, Jiaxu, Liu, Yuanzhong, Xie, Xing, Tu, Zhigang
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01-06-2023; Springer Nature B.V
• Subjects: Accuracy; Artificial Intelligence; Background noise; Cameras; Computer Graphics; Computer Science; Deep learning; Design; Fall detection; Feature extraction; Human mechanics; Human motion; Image Processing and Computer Vision; Machine learning; Neural networks; Optical flow (image analysis); Original Article; Pose estimation; Sensors; Video; Flow-pose Net; Human pose; Simultaneous motion and appearance feature extraction; Fall detection; Optical flow
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-022-02416-2

Aging society gives rise to the need of fall detection for the elderly. The interference of the environmental noise and the loss of motion information causing fall detection still challenging. In this work, we present a novel two-stream network, called Flow-pose Net (FP-Net), which integrates the optical flow and human pose information to achieve robust and accurate fall detection in videos. Specifically, we use a human pose estimation model to detect the joints of the human body and design a GCN-based network to learn the body appearance feature from human pose. For motion feature extraction, we estimate optical flow from raw videos and utilize a CNN-based network to learn rich motion feature. Finally, the appearance feature and the motion feature are concatenated and then fed into a classifier to perform the classification of fall. To the best of our knowledge, we are the first to combine the optical flow and the human pose to simultaneously extract motion and appearance features for fall detection. Extensive experiments are conducted on two popular datasets URFD and Le2i, and the results show that our FP-Net achieves the state-of-the-art performance and has high robustness.