Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach

• Published in: IEEE transactions on smart grid Vol. 10; no. 5; pp. 5174 - 5185
• Main Authors: Kurt, Mehmet Necip, Ogundijo, Oyetunji, Li, Chong, Wang, Xiaodong
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-09-2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Anomalies; Computational modeling; cyber-attack; Cybersecurity; Data analysis; Kalman filter; Machine learning; Markov analysis; Markov processes; Meters; model-free reinforcement learning; Noise measurement; online detection; Outliers (statistics); partially observable Markov decision process (POMDP); Probability density function; Robustness (mathematics); Smart grid; Smart grids; State estimation; Time measurement
• ISSN: 1949-3053; 1949-3061
• DOI: 10.1109/TSG.2018.2878570

Early detection of cyber-attacks is crucial for a safe and reliable operation of the smart grid. In the literature, outlier detection schemes making sample-by-sample decisions and online detection schemes requiring perfect attack models have been proposed. In this paper, we formulate the online attack/anomaly detection problem as a partially observable Markov decision process (POMDP) problem and propose a universal robust online detection algorithm using the framework of model-free reinforcement learning (RL) for POMDPs. Numerical studies illustrate the effectiveness of the proposed RL-based algorithm in timely and accurate detection of cyber-attacks targeting the smart grid.