Mixed Reality Teleoperation Assistance for Direct Control of Humanoids

• Published in: IEEE robotics and automation letters Vol. 9; no. 2; pp. 1 - 8
• Main Authors: Penco, Luigi, Momose, Kazuhiko, McCrory, Stephen, Anderson, Dexton, Kitchel, Nicholas, Calvert, Duncan, Griffin, Robert J.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-02-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Autonomy; Effectiveness; Feasibility studies; Human motion; Humanoid; Humanoid Robot Systems; Humanoid robots; Learning from Demonstration; Mixed reality; Motion perception; Object recognition; Probabilistic logic; Robot dynamics; Robot motion; Robots; Task analysis; Telerobotics and Teleoperation; Trajectory; Virtual Reality and Interfaces; Whole-Body Motion Planning and Control
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2024.3349809

Teleoperation plays a crucial role in enabling robot operations in challenging environments, yet existing limitations in effectiveness and accuracy necessitate the development of innovative strategies for improving teleoperated tasks. This article introduces a novel approach that utilizes mixed reality and assistive autonomy to enhance the efficiency and precision of humanoid robot teleoperation. By leveraging Probabilistic Movement Primitives, object detection, and Affordance Templates, the assistance combines user motion with autonomous capabilities, achieving task efficiency while maintaining human-like robot motion. Experiments and feasibility studies on the Nadia robot confirm the effectiveness of the proposed framework. Supplementary video available at https://youtu.be/oN-FD6YnF2c .