Multi-Agent Deep Reinforcement Learning for Distributed Load Restoration

• Published in: IEEE transactions on smart grid Vol. 15; no. 2; pp. 1749 - 1760
• Main Authors: Vu, Linh, Vu, Tuyen, Vu, Thanh Long, Srivastava, Anurag
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-03-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Circuit breakers; Deep learning; Deep reinforcement learning; Distributed generation; distribution systems; invalid action masking; Learning curves; Linear programming; Load modeling; load restoration; Manganese; Microgrids; multi-agent systems; Multiagent systems; networked microgrids; Nonstationary environments; OpenDSS; Optimization; Restoration; Switches; Training
• ISSN: 1949-3053; 1949-3061
• DOI: 10.1109/TSG.2023.3310893

This paper addresses the load restoration problem after power outage events. Our primary proposed methodology is using multi-agent deep reinforcement learning to optimize the load restoration process in distribution systems, modeled as networked microgrids, via determining the optimal operational sequence of circuit breakers (switches). An innovative invalid action masking technique is incorporated into the multi-agent method to handle both the physical constraints in the restoration process and the curse of dimensionality as the action space of operational decisions grows exponentially with the number of circuit breakers. The features of our proposed method include centralized training for multi-agents to overcome non-stationary environment problems, decentralized execution to ease the deployment, and zero constraint violations to prevent harmful actions. Our simulations are performed in OpenDSS and Python environments to demonstrate the effectiveness of the proposed approach using the IEEE 13, 123, and 8500-node distribution test feeders. The results show that the proposed algorithm can achieve a significantly better learning curve and stability than the conventional methods.