Position‐aware pushing and grasping synergy with deep reinforcement learning in clutter

• Published in: CAAI Transactions on Intelligence Technology Vol. 9; no. 3; pp. 738 - 755
• Main Authors: Zhao, Min, Zuo, Guoyu, Yu, Shuangyue, Gong, Daoxiong, Wang, Zihao, Sie, Ouattara
• Format: Journal Article
• Language: English
• Published: 01-06-2024
• Subjects: deep learning; deep neural networks; intelligent robots
• ISSN: 2468-2322; 2468-2322
• DOI: 10.1049/cit2.12264

The positional information of objects is crucial to enable robots to perform grasping and pushing manipulations in clutter. To effectively perform grasping and pushing manipulations, robots need to perceive the position information of objects, including the coordinates and spatial relationship between objects (e.g., proximity, adjacency). The authors propose an end‐to‐end position‐aware deep Q‐learning framework to achieve efficient collaborative pushing and grasping in clutter. Specifically, a pair of conjugate pushing and grasping attention modules are proposed to capture the position information of objects and generate high‐quality affordance maps of operating positions with features of pushing and grasping operations. In addition, the authors propose an object isolation metric and clutter metric based on instance segmentation to measure the spatial relationships between objects in cluttered environments. To further enhance the perception capacity of position information of the objects, the authors associate the change in the object isolation metric and clutter metric in cluttered environment before and after performing the action with reward function. A series of experiments are carried out in simulation and real‐world which indicate that the method improves sample efficiency, task completion rate, grasping success rate and action efficiency compared to state‐of‐the‐art end‐to‐end methods. Noted that the authors’ system can be robustly applied to real‐world use and extended to novel objects. Supplementary material is available at https://youtu.be/NhG\_k5v3NnM}{https://youtu.be/NhG\_k5v3NnM.