Show me How You Use Your Mouse and I Tell You How You Feel? Sensing Affect With the Computer Mouse

• Published in: IEEE transactions on affective computing Vol. 15; no. 3; pp. 1490 - 1501
• Main Authors: Freihaut, Paul, Goritz, Anja S.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-07-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: affect; Affective computing; Behavioral sciences; computer mouse; Data collection; field study; Longitudinal studies; Machine learning; measurement; Mice; Mouse devices; nonverbal signals; Remuneration; Sensors; Statistical analysis; Task analysis; Tracking
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2024.3357733

Computer mouse tracking is a simple and cost-efficient way to gather continuous behavioral data. As theory suggests a relationship between affect and sensorimotor processes, the computer mouse might be usable for affect sensing. However, the processes underlying a connection between mouse usage and affect are complex, hitherto empirical evidence is ambiguous, and the research area lacks longitudinal studies. The present work brings forward a longitudinal field study in which 179 participants hourly self-reported their affect while their mouse usage was tracked both during their self-directed, contextless as well as task-bound computer use over the course of 14 days, resulting in a dataset comprising 10,760 instances of data collection. Extensive statistical analysis using null hypothesis significance testing and machine learning reveal weak and sporadic relationships between mouse usage and longitudinal self-reported affect at best. The results of this study challenge the use of computer mouse tracking for longitudinal affect sensing and point to a necessity for more research.