HoloUS: Augmented reality visualization of live ultrasound images using HoloLens for ultrasound-guided procedures

• Published in: International journal for computer assisted radiology and surgery Vol. 17; no. 2; pp. 385 - 391
• Main Authors: Nguyen, Trong, Plishker, William, Matisoff, Andrew, Sharma, Karun, Shekhar, Raj
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01-02-2022; Springer Nature B.V
• Subjects: Augmented Reality; Computer Imaging; Computer Science; Eyewear; Field of view; Health Informatics; Humans; Imaging; Medicine; Medicine & Public Health; Needles; Network latency; Original Article; Pattern Recognition and Graphics; Performance measurement; Phantoms, Imaging; Radiology; Surgery; Three dimensional printing; Tracking control; Ultrasonic imaging; Ultrasonic transducers; Ultrasonography; Ultrasonography, Interventional; User experience; Vision; Visualization; Voice control; Wireless networks; HoloLens; Ultrasound guidance; Head-mounted display; Ultrasound registration; Augmented reality; Ultrasound visualization
• ISSN: 1861-6410; 1861-6429
• DOI: 10.1007/s11548-021-02526-7

Purpose
Microsoft HoloLens is a pair of augmented reality (AR) smart glasses that could improve the intraprocedural visualization of ultrasound-guided procedures. With the wearable HoloLens headset, an ultrasound image can be virtually rendered and registered with the ultrasound transducer and placed directly in the practitioner’s field of view.
Methods
A custom application, called HoloUS, was developed using the HoloLens and a portable ultrasound machine connected through a wireless network. A custom 3D-printed case with an AR-pattern for the ultrasound transducer permitted ultrasound image tracking and registration. Voice controls on the HoloLens supported the scaling and movement of the ultrasound image as desired. The ultrasound images were streamed and displayed in real-time. A user study was performed to assess the effectiveness of the HoloLens as an alternative display platform. Novices and experts were timed on tasks involving targeting simulated vessels using a needle in a custom phantom.
Results
Technical characterization of the HoloUS app was conducted using frame rate, tracking accuracy, and latency as performance metrics. The app ran at 25 frames/s, had an 80-ms latency, and could track the transducer with an average reprojection error of 0.0435 pixels. With AR visualization, the novices’ times improved by 17% but the experts’ times decreased slightly by 5%, which may reflect the experts’ training and experience bias.
Conclusion
The HoloUS application was found to enhance user experience and simplify hand–eye coordination. By eliminating the need to alternately observe the patient and the ultrasound images presented on a separate monitor, the proposed AR application has the potential to improve efficiency and effectiveness of ultrasound-guided procedures.