Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking

• Published in: Engineering (Beijing, China) Vol. 7; no. 9; pp. 1248 - 1261
• Main Authors: Dogru, Oguzhan, Velswamy, Kirubakaran, Huang, Biao
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01-09-2021; Department of Chemical and Materials Engineering,University of Alberta,Edmonton,AB T6G 1H9,Canada; Elsevier
• Subjects: Interface tracking; Object tracking; Occlusion; Reinforcement learning; Uniform manifold approximation and projection; Occlusion; Object tracking; Interface tracking; Uniform manifold approximation and projection; Reinforcement learning
• ISSN: 2095-8099
• DOI: 10.1016/j.eng.2021.04.027

This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process. A reinforcement learning (RL) agent successfully tracks an interface between two liquids, which is often a critical variable to track in many chemical, petrochemical, metallurgical, and oil industries. This method utilizes less than 100 images for creating an environment, from which the agent generates its own data without the need for expert knowledge. Unlike supervised learning (SL) methods that rely on a huge number of parameters, this approach requires far fewer parameters, which naturally reduces its maintenance cost. Besides its frugal nature, the agent is robust to environmental uncertainties such as occlusion, intensity changes, and excessive noise. From a closed-loop control context, an interface location-based deviation is chosen as the optimization goal during training. The methodology showcases RL for real-time object-tracking applications in the oil sands industry. Along with a presentation of the interface tracking problem, this paper provides a detailed review of one of the most effective RL methodologies: actor–critic policy.