MARL-Based Cooperative Multi-AGV Control in Warehouse Systems

• Published in: IEEE access Vol. 10; pp. 100478 - 100488
• Main Authors: Choi, Ho-Bin, Kim, Ju-Bong, Han, Youn-Hee, Oh, Se-Won, Kim, Kwihoon
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: AGV; Algorithms; Automated guided vehicles; Automation; Convergence; Cooperation; Cooperative control; Decision making; fulfillment center; Inventory management; Layout; Machine learning; multi-agent reinforcement learning; Multiagent systems; Path planning; Productivity; Real-time systems; Reinforcement learning; Warehouses
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3206537

Automated guided vehicles (AGVs) are essential components for the automation of fulfillment centers, a type of warehouse, where goods are stored on shelves and carried by AGVs. To increase the productivity in inventory management, a well-organized cooperative path control is required to transport goods to the designated picking stations. In this paper, we propose a QMIX-based scheme for the cooperative path control of multiple AGVs. Although QMIX is the one of popular cooperative multi-agent reinforcement learning algorithms, we find that QMIX alone was not enough to increase productivity in warehouse systems. So, we develop two novel techniques that can be used with QMIX: 1) sequential action masking that eliminates all the collision cases and 2) additional local loss that improves collaboration of individual AGVs. They help to encourage the AGVs to cooperate more for high productivity. By extensive simulations, we present the superiority of the proposed scheme on several layouts in fulfillment centers. The effect of cooperation among AGVs in the proposed scheme is verified through the comparison study with the existing algorithms. Additionally, we show the generalization performance by investigating the reusability of the model trained with the proposed scheme.