Cross-Subject Activity Detection for COVID-19 Infection Avoidance Based on Automatically Annotated IMU Data

• Published in: IEEE sensors journal Vol. 22; no. 13; pp. 13125 - 13135
• Main Authors: Rizk, Hamada, Amano, Tatsuya, Yamaguchi, Hirozumi, Youssef, Moustafa
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-07-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: activity detection; Convolutional neural networks; Coronaviruses; COVID-19; Covid-19 infection avoidance; Data collection; Data models; Disease transmission; Face recognition; Face-touch prediction; Feature extraction; Influenza; Learning; Modules; Public safety; Sensors; Smartphones; smartwatch-based sensing; Smartwatches; Viral diseases; Viruses; Wearable computers
• DOI: 10.1109/JSEN.2022.3176291
• EISSN: 1558-1748
• ISSN: 1530-437X

The World Health Organization reported that face touching is a primary source of infection transmission of viral diseases, including COVID-19, seasonal Influenza, Swine flu, Ebola virus, etc. Thus, people have been advised to avoid such activity to break the viral transmission chain. However, empirical studies showed that it is either impossible or difficult to avoid as it is unconsciously a human habit. This gives rise to the need to develop means enabling the automatic prediction of the occurrence of such activity. In this paper, we propose SafeSense , a cross-subject face-touch prediction system that combines the sensing capability of smartwatches and smartphones. The system includes innovative modules for automatically labeling the smartwatches' sensor measurements using smartphones' proximity sensors during normal phone use. Additionally, SafeSense uses a multi-task learning approach based on autoencoders for learning a subject-invariant representation without any assumptions about the target subjects. SafeSense also improves the deep model's generalization ability and incorporates different modules to boost the per-subject system's accuracy and robustness at run-time. We evaluated the proposed system on ten subjects using three different smartwatches and their connected phones. Results show that SafeSense can obtain as high as 97.9% prediction accuracy with a F1-score of 0.98. This outperforms the state-of-the-art techniques in all the considered scenarios without extra data collection overhead. These results highlight the feasibility of the proposed system for boosting public safety.