Scheduling of Dual-Gripper Robotic Cells With Reinforcement Learning

• Published in: IEEE transactions on automation science and engineering Vol. 19; no. 2; pp. 1120 - 1136
• Main Authors: Kim, Hyun-Jung, Lee, Jun-Ho
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-04-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Dual-gripper robotic cell; Job shop scheduling; Job shops; Learning; Lower bounds; Manufacturing; Materials handling; Mathematical analysis; Mathematical model; Petri nets; reinforcement learning (RL); Robotics; Robots; Scheduling; Service robots; Task analysis; Task scheduling; time variations
• ISSN: 1545-5955; 1558-3783
• DOI: 10.1109/TASE.2020.3047924

A dual-gripper robotic cell consists of multiple processing machines and one material handling robot, which can perform an unloading or a loading task one at a time but can hold two parts at the same time. We address a scheduling problem of the robotic cell that determines a robot task sequence when two part types are processed in a different set of machines and all machines have variable processing times within a given interval. The objective is to minimize the makespan. This study proposes a learning-based method, i.e., a reinforcement learning (RL) approach, for the first time, to address a dual-gripper robotic cell scheduling problem. The problem is modeled with a Petri net, a graphical and mathematical modeling tool, which is used as an environment in RL. The states, actions, and rewards are defined by using flow shop scheduling properties, features from a Petri net, and knowledge from previous studies of scheduling robotized tools. Then, the RL approach is compared to the first-in-first-out (FIFO) rule, which is generally used in practice, a swap sequence, which is widely used for cyclic scheduling of dual-gripper robotic cells, and a lower bound. The extensive experiments show that the proposed method performs better than FIFO and the swap sequence; moreover, the gap between the makespan of the proposed method and the lower bound is not large. Note to Practitioners -We address a scheduling problem of dual-gripper robotic cells with two-part types when all machines have processing time variations. We propose an RL approach to obtain an efficient robot task sequence in order to minimize makespan. The proposed method is performed offline, and a robot task sequence is then obtained instantaneously. The proposed method performs better than the FIFO rule used in practice and the swap sequence used for cyclic scheduling of robotic cells. It can be easily extended for scheduling other configurations of robotic cells.