WFH-VR: Teleoperating a Robot Arm to set a Dining Table across the Globe via Virtual Reality

• Published in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 4927 - 4934
• Main Authors: Yim, Lai Sum, Vo, Quang TN, Huang, Ching-I, Wang, Chi-Ruei, McQueary, Wren, Wang, Hsueh-Cheng, Huang, Haikun, Yu, Lap-Fai
• Format: Conference Proceeding
• Language: English
• Published: IEEE 23-10-2022
• Subjects: Manipulators; Pose estimation; Real-time systems; Robot vision systems; Streaming media; Three-dimensional displays; Virtual reality
• DOI: 10.1109/IROS47612.2022.9981729
• EISSN: 2153-0866

This paper presents an easy-to-deploy, virtual reality-based teleoperation system for controlling a robot arm. The proposed system is based on a consumer-grade virtual reality device (Oculus Quest 2) with a low-cost robot arm (a LoCoBot) to allow easy replication and set up. The proposed Work-from-Home Virtual Reality (WFH-VR) system allows the user to feel an intimate connection with the real remote robot arm. Virtual representations of the robot and objects to be manipulated in the real-world are presented in VR by streaming data pertaining to orientation and poses. The user studies suggest that 1) the proposed telerobotic system is effective under conditions both with and without network latency, whereas a method that simply streams video does not. This design enables the system implemented at an arbitrary distance from the actual work site. 2) The proposed system allows novices to perform manipulation tasks requiring higher dexterity than traditional keyboard controls can support, such as setting tableware. All results, hardware settings, and questionnaire feedback can be obtained at https://arg-nctu.github.io/projects/vr-robot-arm.html.