E-LAMP: integration of innovative ideas for multimedia event detection

• Published in: Machine vision and applications Vol. 25; no. 1; pp. 5 - 15
• Main Authors: Tong, Wei, Yang, Yi, Jiang, Lu, Yu, Shoou-I, Lan, ZhenZhong, Ma, Zhigang, Sze, Waito, Younessian, Ehsan, Hauptmann, Alexander G.
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 2014; Springer Nature B.V
• Subjects: Algorithms; Communications Engineering; Computer Science; Computer vision; Empirical analysis; Feature extraction; Image Processing and Computer Vision; Multimedia; Networks; Pattern Recognition; Special Issue Paper; Video data; Vision systems; Multimedia event detection; Multimedia content analysis
• ISSN: 0932-8092; 1432-1769
• DOI: 10.1007/s00138-013-0529-6

Detecting multimedia events in web videos is an emerging hot research area in the fields of multimedia and computer vision. In this paper, we introduce the core methods and technologies of the framework we developed recently for our Event Labeling through Analytic Media Processing (E-LAMP) system to deal with different aspects of the overall problem of event detection. More specifically, we have developed efficient methods for feature extraction so that we are able to handle large collections of video data with thousands of hours of videos. Second, we represent the extracted raw features in a spatial bag-of-words model with more effective tilings such that the spatial layout information of different features and different events can be better captured, thus the overall detection performance can be improved. Third, different from widely used early and late fusion schemes, a novel algorithm is developed to learn a more robust and discriminative intermediate feature representation from multiple features so that better event models can be built upon it. Finally, to tackle the additional challenge of event detection with only very few positive exemplars, we have developed a novel algorithm which is able to effectively adapt the knowledge learnt from auxiliary sources to assist the event detection. Both our empirical results and the official evaluation results on TRECVID MED’11 and MED’12 demonstrate the excellent performance of the integration of these ideas.