Multi-agent DRL for joint completion delay and energy consumption with queuing theory in MEC-based IIoT

• Published in: Journal of parallel and distributed computing Vol. 176; pp. 80 - 94
• Main Authors: Wu, Guowen, Xu, Zhiqi, Zhang, Hong, Shen, Shigen, Yu, Shui
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-06-2023
• Subjects: Industrial Internet of things; Mobile edge computing; Multi-agent deep reinforcement learning; Queuing theory; Task offloading; Multi-agent deep reinforcement learning; Industrial Internet of things; Queuing theory; Task offloading; Mobile edge computing
• ISSN: 0743-7315; 1096-0848
• DOI: 10.1016/j.jpdc.2023.02.008

In the Industrial Internet of Things (IIoT), there exist numerous sensor devices with weak computing power and small energy storage. To meet the real-time and big data computing requirements of industrial production, EIIoT (Edge computing-based IIoT) that combines mobile edge computing with IIoT has emerged. It is necessary to offload computing tasks to nearby edge servers for data storage and processing in EIIoT, thus inevitably causing the edge servers to overload. To this end, we propose a jointly constrained optimization model of delay and energy consumption based on queuing theory; this model can effectively solve the task offloading problem in EIIoT. Subsequently, to satisfy the unique offloading requirements of EIIoT, we improve the MAPPO (multi agent proximal policy optimization) algorithm structure to form a lightweight optimal task offloading algorithm called Multi-Agent Deep Reinforcement Learning based on Queuing theory (MAQDRL), which is more suitable for EIIoT. In the algorithm, we systematically integrate queuing theory and use Multi-Agent Deep Reinforcement Learning (MADRL) to obtain the optimal offloading strategy in dynamic and random multiuser offloading environments. We also improve the structure of neural networks of MADRL by analyzing the structural characteristics of the input data. As a result, the algorithm that we proposed exhibits good convergence and exceptional performance in terms of the task arrival rate, bandwidth, energy consumption, latency and other indicators. The simulation results indicate that compared with other classical algorithms, MAQDRL is effective for solving the EIIoT offloading problem.
•Construct an EIIoT characteristics-based intelligent lightweight task offloading framework.•Design an M/M/N model fully considering calculation delays based on queuing theory.•Develop a lightweight MADRL algorithm to obtain the optimal task offloading strategy for EIIoT.