Video key frame extraction based on scale and direction analysis

• Published in: Journal of engineering (Stevenage, England) Vol. 2022; no. 9; pp. 910 - 918
• Main Authors: Dong, Yanlei, Zhang, Yunzuo, Zhang, Jiayu, Zhang, Xiuqing
• Format: Journal Article
• Language: English
• Published: London John Wiley & Sons, Inc 01-09-2022; Wiley
• Subjects: Accuracy; Algorithms; Cameras; Frames (data processing); Human body; Methods; Segments; Surveillance; Trajectory analysis; Video
• ISSN: 2051-3305; 2051-3305
• DOI: 10.1049/tje2.12173

Currently, efficient and accurate key frame extraction for massive videos remains a challenge, especially in surveillance applications. To overcome the inaccuracy of traditional key frame extraction, this paper proposes a method for key frame extraction via analyzing scale and direction of motion trajectory on spatiotemporal slice (MTSS). The proposed method employs both local and global motion state changes of object as the metric to extract key frames and meanwhile gives a strategy which integrates coarse extraction with partial fine re‐extraction of spatiotemporal slices to capture these state changes. Firstly, coarse extraction of spatiotemporal slices is employed to detect motion segments. Then partial fine re‐extraction is performed on the motion segments to obtain MTSS. Finally, the frames at the inflexions of scale and direction of MTSS are extracted as key frames. The experimental results have demonstrated that the proposed method outperforms existing state‐of‐the‐art methods in terms of accuracy for various videos while requiring a comparable or even less computation time.