Understanding Video Events: A Survey of Methods for Automatic Interpretation of Semantic Occurrences in Video

• Published in: IEEE transactions on systems, man and cybernetics. Part C, Applications and reviews Vol. 39; no. 5; pp. 489 - 504
• Main Authors: Lavee, G., Rivlin, E., Rudzsky, M.
• Format: Journal Article
• Language: English
• Published: New-York, NY IEEE 01-09-2009; Institute of Electrical and Electronics Engineers
• Subjects: Action; activity; Airports; Applied sciences; Artificial intelligence; behavior; Computer science; control theory; systems; Computer systems and distributed systems. User interface; event; Exact sciences and technology; Handicapped aids; Hidden Markov models; Indexing; Interactive systems; Pattern recognition. Digital image processing. Computational geometry; recognition; Software; Speech and sound recognition and synthesis. Linguistics; Surveillance; Taxonomy; Traffic control; video; Video sequences; Action; activity; Taxonomy; Video signal; recognition; Aerial photography; video; Modeling; Interest; Semantics; Image sequence; behavior; event; Monitoring; Content analysis; Computer vision; Molding; Parking; Pattern recognition; Abstraction; Subways; Terminal; Interactive system; Sign language; Surveillance; Interactive video; Airport; Hidden Markov model; Subway station
• ISSN: 1094-6977; 1558-2442
• DOI: 10.1109/TSMCC.2009.2023380

Understanding video events, i.e., the translation of low-level content in video sequences into high-level semantic concepts, is a research topic that has received much interest in recent years. Important applications of this paper include smart surveillance systems, semantic video database indexing, and interactive systems. This technology can be applied to several video domains including airport terminal, parking lot, traffic, subway stations, aerial surveillance, and sign language data. In this paper, we identify the two main components of the event understanding process: abstraction and event modeling. Abstraction is the process of molding the data into informative units to be used as input to the event model. Due to space restrictions, we will limit the discussion on the topic of abstraction. See the study by Lavee et al. (Understanding video events: A survey of methods for automatic interpretation of semantic occurrences in video, Technion-Israel Inst. Technol., Haifa, Israel, Tech. Rep. CIS-2009-06, 2009) for a more complete discussion. Event modeling is devoted to describing events of interest formally and enabling recognition of these events as they occur in the video sequence. Event modeling can be further decomposed in the categories of pattern-recognition methods, state event models, and semantic event models. In this survey, we discuss this proposed taxonomy of the literature, offer a unifying terminology, and discuss popular event modeling formalisms (e.g., hidden Markov model) and their use in video event understanding using extensive examples from the literature. Finally, we consider the application domain of video event understanding in light of the proposed taxonomy, and propose future directions for research in this field.