Safe Exploration Reinforcement Learning for Load Restoration using Invalid Action Masking

• Published in: 2023 IEEE Power & Energy Society General Meeting (PESGM) pp. 1 - 5
• Main Authors: Vu, Linh, Vu, Tuyen, Vu, Thanh-Long, Srivastava, Anurag
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16-07-2023
• Subjects: Deep learning; deep reinforcement learning; distribution systems; Invalid action masking; Large-scale systems; load restoration; multi-agent systems; networked microgrids; OpenDSS; Power system reliability; Reinforcement learning; Resilience; Simulation; Training
• ISSN: 1944-9933
• DOI: 10.1109/PESGM52003.2023.10253213

This paper addresses the load restoration problem after a power outage event. Our primary proposed methodology uses a multi-agent reinforcement learning method to make the optimal sequential decisions on picking up critical loads. Typically, a negative reward is provided to discourage the agents from selecting decisions that violate physical constraints during the restoration process. However, the main disadvantage of this approach is its difficulty in applying it to large-scale systems due to the curse of dimensionality. This paper introduces the invalid action masking technique to overcome this limitation. The features of this technique include zero physical constraint violations, reduced training time, and stabilization of the exploration process. Simulation results are performed in IEEE 13-node and IEEE 123-node systems showing the performance of the proposed algorithm in comparison to the conventional approach both in terms of restored power and learning curve.