Visual Focus of Attention Estimation With Unsupervised Incremental Learning

• Published in: IEEE transactions on circuits and systems for video technology Vol. 26; no. 12; pp. 2264 - 2272
• Main Authors: Duffner, Stefan, Garcia, Christophe
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-12-2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Clustering algorithms; Clusters; Computer Science; Computer Vision and Pattern Recognition; Hidden Markov models; Image Processing; Image sequence analysis; Machine Learning; Neural and Evolutionary Computing; pattern clustering; Robotics; Training; Unsupervised learning; Visualization; pattern clustering; Unsupervised learning; image sequence analysis
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2015.2501920

In this paper, we propose a new method for estimating the visual focus of attention (VFOA) in a video stream captured by a single distant camera and showing several persons sitting around a table, like in formal meeting or video conferencing settings. The visual targets for a given person are automatically extracted online using an unsupervised algorithm that incrementally learns the different appearance clusters from low-level visual features computed from face patches provided by a face tracker without the need of an intermediate error-prone step of head pose estimation as in classical approaches. The clusters learned in that way can then be used to classify the different visual attention targets of the person during a tracking run, without any prior knowledge on the environment and the configuration of the room or the visible persons. The experiments on public datasets containing almost 2 h of annotated videos from meetings and video conferencing show that the proposed algorithm produces state-of-the-art results and even outperforms a traditional supervised method that is based on head orientation estimation and that classifies VFOA using Gaussian mixture models.