Deep Reinforcement Learning-Based Smart Joint Control Scheme for On/Off Pumping Systems in Wastewater Treatment Plants

• Published in: IEEE access Vol. 9; pp. 95360 - 95371
• Main Authors: Seo, Giup, Yoon, Seungwook, Kim, Myungsun, Mun, Changho, Hwang, Euiseok
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; cost-effective energy efficiency; Deep learning; deep neural network; Electricity pricing; Energy consumption; Energy costs; Forecasting; Integer programming; Job shop scheduling; Linear programming; Machine learning; Neural networks; Optimization; Predictive control; Predictive models; Pumping; Pumping stations; pumping system; reinforcement learning; Tariffs; Wastewater; Wastewater treatment; Water treatment
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3094466

In this paper, we propose a deep reinforcement learning (DRL) based predictive control scheme for reducing the energy consumption and energy cost of pumping systems in wastewater treatment plants (WWTP), in which the pumps are operated in a binary mode, using on/off signals. As global energy consumption increases, the efficient operation of energy-intensive facilities has also become important. A WWTP in Busan, Republic of Korea is used as the target of this study. This WWTP is a large energy-consuming facility, and the pumping station accounts for a significant portion of the energy consumption of the WWTP. The framework of the proposed scheme consists of a deep neural network (DNN) model for forecasting wastewater inflow and a DRL agent for controlling the on/off signals of the pumping system, where proximal policy optimization (PPO) and deep Q-neural network (DQN) are employed as the DRL agents. To implement smart control with DRL, a reward function is designed to consider the energy consumption amount and electricity price information. In particular, new features and penalty factors for pump switching, which are essential for preventing pump wear, are also considered. The performance of our designed DRL agents is compared with those of WWTP experts and conventional approaches such as scheduling method and model predictive control (MPC), in which integer linear programming (ILP) optimization is employed. Results show that the designed agents outperform the other approaches in terms of compliance with operating rules and reducing energy costs.