Combining Motion Planner and Deep Reinforcement Learning for UAV Navigation in Unknown Environment

• Published in: IEEE robotics and automation letters Vol. 9; no. 1; pp. 635 - 642
• Main Authors: Xue, Yuntao, Chen, Weisheng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-01-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Autonomous aerial vehicles; Barriers; Deep learning; Deep reinforcement learning; Heuristic algorithms; Kinematics; Machine learning; Motion planning; Navigation; Obstacle avoidance; partially observable Markov decision process; Planning; Task analysis; Training; Trajectories; Trajectory; UAV navigation; Unknown environments; Unmanned aerial vehicles
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2023.3334978

Navigation of unmanned aerial vehicles (UAVs) in unknown environments is a challenging problem, and it is worth considering how to reach the target through static obstacles in a safe and energy-efficient manner. The traditional motion planning algorithm is easy to get into trouble when the obstacles are dense. The navigation algorithm based on reinforcement learning has better generalization and robustness, but the trajectory generated by the end-to-end method is not smooth and dynamic enough. In this work, a classical motion planning algorithm and deep reinforcement learning (DRL) algorithm are combined named RLPlanNav, which aims to solve the problem of safe and dynamic navigation of UAVs in unknown environments. The upper-layer DRL algorithm part of the framework receives the sensor raw information to generate the next local target, and the lower-layer classical planner generates a smooth and safe trajectory to reach the target. The DRL algorithm incorporates an LSTM network to add memory capabilities, thereby ensuring the effectiveness of local target selections. The proposed navigation framework is tested in a simulated environment where static obstacles are randomly generated, and has higher navigation success rates and more kinematic-compliant navigation trajectories compared to traditional motion planning methods and end-to-end methods.