Condition-based maintenance with reinforcement learning for refrigeration systems with selected monitored features

• Published in: Engineering applications of artificial intelligence Vol. 122; p. 106067
• Main Authors: de Lima Munguba, Caio Filipe, de Novaes Pires Leite, Gustavo, Ochoa, Alvaro Antonio Villa, Lopez Droguett, Enrique
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01-06-2023
• Subjects: Condition-based maintenance; Degradation; Energy; Refrigeration; Reinforcement learning; Degradation; Energy; Refrigeration; Condition-based maintenance; Reinforcement learning
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2023.106067

Worldwide, buildings are responsible for almost 30% of energy consumption, and those buildings that intensively use refrigeration systems, such as supermarkets and grocery stores, are also among the most energy-intensive consumers. Refrigeration devices, either commercial or residential, are responsible for a significant part of net emissions. Based on careful measurements, it is possible to reduce energy consumption in these devices by up to 15% only by improving the fault detection and diagnosis techniques. Thus, improving maintenance programs has become a crucial area in energy management in recent years. Nowadays, the market has experienced a hike after smart systems and new network interfaces applied to smart buildings that have allowed previously isolated devices to become smart devices, interacting with control algorithms smartly and, to some extent, autonomously. Here, we propose a reinforcement learning framework to develop a maintenance policy for mechanical compression refrigeration devices. Firstly, a test bench is built in which each component is assigned to be individually repairable and individually degradable in parallel and interconnected processes. Then, the degradation of the components is combined to formulate the system degradation, and the optimal maintenance policy is modeled via Markov decision processes and solved by a reinforcement learning algorithm. The agent-proposed maintenance program if compared to corrective maintenance, managed to reduce energy use and emissions by around 6% while avoiding shortfalls, as well as about the preventive program, where the agent managed to accomplish the same level of energy efficiency while reducing the maintenance costs by 31% and the time under maintenance in 10%. It was found that the reinforcement learning frameworks applied to maintenance have a series of challenges but are innovative and can show promising results compared to traditional maintenance techniques, such as preventive and corrective ones.
•Several frameworks of Condition Based Maintenance had been proposed for freezers. In this paper, we present Reinforcement Learning as an alternative tool.•Hidden Markov Models and Markov Decision Processes were implemented to model degradation schemes in parallel and competing processes.•The Double Deep Q-Learn algorithm is used to solve the problem of maintenance scheduling for freezers.•Reinforcement Learning applied to maintenance schedule can reduce emissions, and maintenance costs and increase the availability of refrigeration devices.