A Review of Nine Physics Engines for Reinforcement Learning Research

• Main Authors: Kaup, Michael, Wolff, Cornelius, Hwang, Hyerim, Mayer, Julius, Bruni, Elia
• Format: Journal Article
• Language: English
• Published: 11-07-2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Learning; Computer Science - Multiagent Systems
• DOI: 10.48550/arxiv.2407.08590

We present a review of popular simulation engines and frameworks used in
reinforcement learning (RL) research, aiming to guide researchers in selecting
tools for creating simulated physical environments for RL and training setups.
It evaluates nine frameworks (Brax, Chrono, Gazebo, MuJoCo, ODE, PhysX,
PyBullet, Webots, and Unity) based on their popularity, feature range, quality,
usability, and RL capabilities. We highlight the challenges in selecting and
utilizing physics engines for RL research, including the need for detailed
comparisons and an understanding of each framework's capabilities. Key findings
indicate MuJoCo as the leading framework due to its performance and
flexibility, despite usability challenges. Unity is noted for its ease of use
but lacks scalability and simulation fidelity. The study calls for further
development to improve simulation engines' usability and performance and
stresses the importance of transparency and reproducibility in RL research.
This review contributes to the RL community by offering insights into the
selection process for simulation engines, facilitating informed
decision-making.