Multimodal Breathing Rate Estimation Using Facial Motion and RPPG From RGB Camera

• Published in: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 2046 - 2050
• Main Authors: Gwak, Migyeong, Vatanparvar, Korosh, Zhu, Li, Rashid, Nafiul, Ahmed, Mohsin, Bae, Jungmok, Kuang, Jilong, Gao, Alex
• Format: Conference Proceeding
• Language: English
• Published: IEEE 14-04-2024
• Subjects: breathing motion tracking; Breathing rate; camera; Cameras; Estimation; Faces; Monitoring; multimodal; remote PPG; Tracking; Wearable devices
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP48485.2024.10446086

Camera-based respiratory monitoring is contactless, non-invasive, unobtrusive, and easily accessible compared to conventional wearable devices. This paper presents a novel multimodal approach to estimating breathing rate based on tracking the movement and color changes of the face through an RGB camera. A machine learning model determines the final breathing rate between two separately calculated ones from breathing motion and remote photoplethysmography (rPPG) to improve the measurement performance in a broader range of breathing frequencies. Our proposed pipeline is evaluated with 140 facial video recordings from 22 healthy subjects, including 6 controlled and 2 spontaneous breathing tasks ranging from 5 to 30 BPM. The estimation accuracy achieves 1.33 BPM mean absolute error and 86.53% pass rate within 2 BPM error criteria. To the best of our knowledge, our approach outperforms previous works that use a face region alone with a single RGB camera.