Soft Actor-Critic for Navigation of Mobile Robots

• Published in: Journal of intelligent & robotic systems Vol. 102; no. 2
• Main Authors: de Jesus, Junior Costa, Kich, Victor Augusto, Kolling, Alisson Henrique, Grando, Ricardo Bedin, Cuadros, Marco Antonio de Souza Leite, Gamarra, Daniel Fernando Tello
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01-06-2021; Springer; Springer Nature B.V
• Subjects: Algorithms; Angular velocity; Artificial Intelligence; Comparative analysis; Control; Electrical Engineering; Employee motivation; Engineering; Mechanical Engineering; Mechatronics; Navigation; Regular Paper; Robotics; Robots; Topical collection on ICAR 2019 Special Issue; Deep reinforcement learning; Soft actor-critic; Deep deterministic policy gradients; Navigation for mobile robots
• ISSN: 0921-0296; 1573-0409
• DOI: 10.1007/s10846-021-01367-5

This paper provides a study of two deep reinforcement learning techniques for application in navigation of mobile robots, one of the techniques is the Soft Actor Critic (SAC) that is compared with the Deep Deterministic Policy Gradients (DDPG) algorithm in the same situation. In order to make a robot to arrive at a target in an environment, both networks have 10 laser range findings, the previous linear and angular velocity, and relative position and angle of the mobile robot to the target are used as the network inputs. As outputs, the networks have the linear and angular velocity of the mobile robot. The reward function created was designed in a way to only give a positive reward to the agent when it gets to the target and a negative reward when colliding with any object. The proposed architecture was applied successfully in two simulated environments, and a comparison between the two referred techniques was made using the results obtained as a basis and it was demonstrated that the SAC algorithm has a superior performance for the navigation of mobile robots than the DDPG algorithm (Code available at
https://github.com/dranaju/project
).