Image Conditioned Keyframe-Based Video Summarization Using Object Detection

• Main Authors: Baghel, Neeraj, Raikwar, Suresh C, Bhatnagar, Charul
• Format: Journal Article
• Language: English
• Published: 11-09-2020
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2009.05269

Video summarization plays an important role in selecting keyframe for
understanding a video. Traditionally, it aims to find the most representative
and diverse contents (or frames) in a video for short summaries. Recently,
query-conditioned video summarization has been introduced, which considers user
queries to learn more user-oriented summaries and its preference. However,
there are obstacles in text queries for user subjectivity and finding
similarity between the user query and input frames. In this work, (i) Image is
introduced as a query for user preference (ii) a mathematical model is proposed
to minimize redundancy based on the loss function & summary variance and (iii)
the similarity score between the query image and input video to obtain the
summarized video. Furthermore, the Object-based Query Image (OQI) dataset has
been introduced, which contains the query images. The proposed method has been
validated using UT Egocentric (UTE) dataset. The proposed model successfully
resolved the issues of (i) user preference, (ii) recognize important frames and
selecting that keyframe in daily life videos, with different illumination
conditions. The proposed method achieved 57.06% average F1-Score for UTE
dataset and outperforms the existing state-of-theart by 11.01%. The process
time is 7.81 times faster than actual time of video Experiments on a recently
proposed UTE dataset show the efficiency of the proposed method