Neurorobotic reinforcement learning for domains with parametrical uncertainty

• Published in: Frontiers in neurorobotics Vol. 17; p. 1239581
• Main Authors: Amaya, Camilo, von Arnim, Axel
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 25-10-2023
• Subjects: domain randomization; neuromorphic computing; neurorobotics; Neuroscience; reinforcement learning; robot control; spiking neural networks; spiking neural networks; neurorobotics; robot control; domain randomization; neuromorphic computing; reinforcement learning
• ISSN: 1662-5218; 1662-5218
• DOI: 10.3389/fnbot.2023.1239581

Neuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this study, we used the neurorobotics platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implemented a force-torque feedback-based classic object insertion task ("peg-in-hole") and controlled the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to real-world parameter variations in the target domain, filling the sim-to-real gap.To the best of our knowledge, it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains.