Adaptive Height Optimization for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach

• Published in: IEEE access Vol. 11; pp. 5966 - 5980
• Main Authors: Fonseca, Erika, Galkin, Boris, Amer, Ramy, DaSilva, Luiz A., Dusparic, Ivana
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Autonomous aerial vehicles; Connectivity; Deep learning; Density; Experimental measurements; Ground stations; Interference; Machine learning; Massive MIMO; Optimization; Predictive models; Quality of service; Reinforcement learning; Solid modeling; Tall buildings; two-tier networks; Unmanned aerial vehicles; unmanned aerial vehicles (UAVs)
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3232077

Providing reliable connectivity to cellular-connected Unmanned Aerial Vehicles (UAVs) can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground Base Stations (BSs). On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connectivity of UAVs in such environments, this paper proposes a Reinforcement Learning (RL) algorithm to dynamically optimise the height of a UAV as it moves through the environment, with the goal of increasing the throughput or spectrum efficiency that it experiences. The proposed solution is evaluated in two settings: using a series of generated environments where we vary the number of BS and building densities, and in a scenario using real-world data obtained from an experiment in Dublin, Ireland. Results show that our proposed RL-based solution improves UAV Quality of Service (QoS) by 6% to 41%, depending on the scenario. We also conclude that, when flying at heights higher than the buildings, building density variation has no impact on UAV QoS. On the other hand, BS density can negatively impact UAV QoS, with higher numbers of BSs generating more interference and deteriorating UAV performance.