Enabling visually impaired people to learn three-dimensional tactile graphics with a 3DOF haptic mouse

• Published in: Journal of neuroengineering and rehabilitation Vol. 18; no. 1; pp. 1 - 146
• Main Authors: Memeo, Mariacarla, Jacono, Marco, Sandini, Giulio, Brayda, Luca
• Format: Journal Article
• Language: English
• Published: London BioMed Central Ltd 25-09-2021; BioMed Central; BMC
• Subjects: Actuators; Blind, Apparatus for the; Blindness; Cognitive models; Design and construction; Dictionaries; Elevation; Feedback; Geometry; Graphical user interface; Hands; Haptics; Haptics technology; Inclination; Interfaces; Object recognition; People with disabilities; Pitch (inclination); Proprioception; Psychological aspects; Rolling motion; Scalars; Spatial data; Technology application; Three dimensional printing; Three-dimensional graphics; Vision; Visual impairment; Visually disabled persons; Workload; Italy
• ISSN: 1743-0003; 1743-0003
• DOI: 10.1186/s12984-021-00935-y

Background In this work, we present a novel sensory substitution system that enables to learn three dimensional digital information via touch when vision is unavailable. The system is based on a mouse-shaped device, designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. The device hosts a tactile actuator with three degrees of freedom: elevation, roll and pitch. The actuator approximates the tactile interaction with a plane tangential to the contact point between the finger and the field. Spatial information can therefore be mentally constructed by integrating local and global tactile cues: the actuator provides local cues, whereas proprioception associated with the mouse motion provides the global cues. Methods The efficacy of the system is measured by a virtual/real object-matching task. Twenty-four gender and age-matched participants (one blind and one blindfolded sighted group) matched a tactile dictionary of virtual objects with their 3D-printed solid version. The exploration of the virtual objects happened in three conditions, i.e., with isolated or combined height and inclination cues. We investigated the performance and the mental cost of approximating virtual objects in these tactile conditions. Results In both groups, elevation and inclination cues were sufficient to recognize the tactile dictionary, but their combination worked at best. The presence of elevation decreased a subjective estimate of mental effort. Interestingly, only visually impaired participants were aware of their performance and were able to predict it. Conclusions The proposed technology could facilitate the learning of science, engineering and mathematics in absence of vision, being also an industrial low-cost solution to make graphical user interfaces accessible for people with vision loss. Keywords: Object recognition, Haptics, Visual impairment, Workload, Geometry