Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks

• Main Authors: Chen, Haonan, Niu, Yilong, Hong, Kaiwen, Liu, Shuijing, Wang, Yixuan, Li, Yunzhu, Driggs-Campbell, Katherine
• Format: Journal Article
• Language: English
• Published: 28-09-2023
• Subjects: Computer Science - Robotics
• DOI: 10.48550/arxiv.2309.16873

Stowing, the task of placing objects in cluttered shelves or bins, is a
common task in warehouse and manufacturing operations. However, this task is
still predominantly carried out by human workers as stowing is challenging to
automate due to the complex multi-object interactions and long-horizon nature
of the task. Previous works typically involve extensive data collection and
costly human labeling of semantic priors across diverse object categories. This
paper presents a method to learn a generalizable robot stowing policy from
predictive model of object interactions and a single demonstration with
behavior primitives. We propose a novel framework that utilizes Graph Neural
Networks to predict object interactions within the parameter space of
behavioral primitives. We further employ primitive-augmented trajectory
optimization to search the parameters of a predefined library of heterogeneous
behavioral primitives to instantiate the control action. Our framework enables
robots to proficiently execute long-horizon stowing tasks with a few keyframes
(3-4) from a single demonstration. Despite being solely trained in a
simulation, our framework demonstrates remarkable generalization capabilities.
It efficiently adapts to a broad spectrum of real-world conditions, including
various shelf widths, fluctuating quantities of objects, and objects with
diverse attributes such as sizes and shapes.