Non-Contrastive Unsupervised Learning of Physiological Signals from Video

• Published in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 14464 - 14474
• Main Authors: Speth, Jeremy, Vance, Nathan, Flynn, Patrick, Czajka, Adam
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-06-2023
• Subjects: Biomedical monitoring; Estimation; Representation learning; Self-supervised or unsupervised representation learning; Sensors; Task analysis; Training; Visualization
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52729.2023.01390

Subtle periodic signals such as blood volume pulse and respiration can be extracted from RGB video, enabling non-contact health monitoring at low cost. Advancements in remote pulse estimation - or remote photoplethysmography (rPPG) - are currently driven by deep learning solutions. However, modern approaches are trained and evaluated on benchmark datasets with ground truth from contact-P PG sensors. We present the first non-contrastive unsuper-vised learning framework for signal regression to mitigate the need for labelled video data. With minimal assumptions of periodicity and finite bandwidth, our approach discovers the blood volume pulse directly from unlabelled videos. We find that encouraging sparse power spectra within normal physiological bandlimits and variance over batches of power spectra is sufficient for learning visual features of periodic signals. We perform the first experiments utilizing unlabelled video data not specifically created for rPPG to train robust pulse rate estimators. Given the limited inductive biases and impressive empirical results, the approach is theoretically capable of discovering other periodic signals from video, enabling multiple physiological measurements without the need for ground truth signals.