On the Use of Reinforcement Learning for Attacking and Defending Load Frequency Control

• Published in: IEEE transactions on smart grid Vol. 15; no. 3; pp. 3262 - 3277
• Main Authors: Mohamed, Amr S., Kundur, Deepa
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-05-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: anomaly detection; autoencoder; Computer crime; cyber-physical security; Deep learning; Frequency control; Frequency measurement; Power measurement; power system; Power system dynamics; Reinforcement learning; Security; Transmission line measurements
• DOI: 10.1109/TSG.2023.3343100
• EISSN: 1949-3061
• ISSN: 1949-3053

The electric grid is an attractive target for cyberattackers given its critical nature in society. With the increasing sophistication of cyberattacks, effective grid defense will benefit from proactively identifying vulnerabilities and attack strategies. We develop a deep reinforcement learning-based method that recognizes vulnerabilities in load frequency control, an essential process that maintains grid security and reliability. We demonstrate how our method can synthesize a variety of attacks involving false data injection and load switching, while specifying the attack and threat models - providing insight into potential attack strategies and impact. We discuss how our approach can be employed for testing electric grid vulnerabilities. Moreover our method can be employed to generate data to inform the design of defense strategies and develop attack detection methods. For this, we design and compare a (deep learning-based) supervised attack detector with an unsupervised anomaly detector to highlight the benefits of developing defense strategies based on identified attack strategies.