Efficient Foreground Extraction From HEVC Compressed Video for Application to Real-Time Analysis of Surveillance 'Big' Data

• Published in: IEEE transactions on image processing Vol. 24; no. 11; pp. 3574 - 3585
• Main Authors: Dey, Bhaskar, Kundu, Malay K.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-11-2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: background subtraction; Big Data; Computational modeling; Feature extraction; Image coding; Real-time systems; Redundancy; statistical signal processing; Streaming media; Video coding; Video compression; statistical signal processing; transform coding; HEVC; Background subtraction; video surveillance
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2015.2445631

While surveillance video is the biggest source of unstructured Big Data today, the emergence of high-efficiency video coding (HEVC) standard is poised to have a huge role in lowering the costs associated with transmission and storage. Among the benefits of HEVC over the legacy MPEG-4 Advanced Video Coding (AVC), is a staggering 40 percent or more bitrate reduction at the same visual quality. Given the bandwidth limitations, video data are compressed essentially by removing spatial and temporal correlations that exist in its uncompressed form. This causes compressed data, which are already de-correlated, to serve as a vital resource for machine learning with significantly fewer samples for training. In this paper, an efficient approach to foreground extraction/segmentation is proposed using novel spatio-temporal de-correlated block features extracted directly from the HEVC compressed video. Most related techniques, in contrast, work on uncompressed images claiming significant storage and computational resources not only for the decoding process prior to initialization but also for the feature selection/extraction and background modeling stage following it. The proposed approach has been qualitatively and quantitatively evaluated against several other state-of-the-art methods.