Learning Human-to-Robot Handovers from Point Clouds

• Main Authors: Christen, Sammy, Yang, Wei, Pérez-D'Arpino, Claudia, Hilliges, Otmar, Fox, Dieter, Chao, Yu-Wei
• Format: Journal Article
• Language: English
• Published: 30-03-2023
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning; Computer Science - Robotics
• DOI: 10.48550/arxiv.2303.17592

We propose the first framework to learn control policies for vision-based
human-to-robot handovers, a critical task for human-robot interaction. While
research in Embodied AI has made significant progress in training robot agents
in simulated environments, interacting with humans remains challenging due to
the difficulties of simulating humans. Fortunately, recent research has
developed realistic simulated environments for human-to-robot handovers.
Leveraging this result, we introduce a method that is trained with a
human-in-the-loop via a two-stage teacher-student framework that uses motion
and grasp planning, reinforcement learning, and self-supervision. We show
significant performance gains over baselines on a simulation benchmark,
sim-to-sim transfer and sim-to-real transfer.