Intelligent Routing based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

• Published in: 2019 IEEE Symposium on Computers and Communications (ISCC) pp. 1 - 6
• Main Author: Liu, Wai-xi
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-06-2019
• Subjects: Bandwidth; Buildings; Data-Center Networks; Deep Reinforcement Learning; Delays; Network resource; Reinforcement learning; Resource management; Routing; SDN; Throughput
• ISSN: 2642-7389
• DOI: 10.1109/ISCC47284.2019.8969579

In software-defined data-center networks, Elephant flow/Mice flow/Coflow coexist and multiple resources (bandwidth, cache and computing) coexist. However, the conventional routing methods cannot overcome the large gap between different performance requirements of flow and efficient resource allocation. Therefore, this paper proposes DRL-R (Deep Reinforcement Learning-based Routing) to bridge this gap. First, we recombine multiple resources (node's cache, link's bandwidth) by quantifying the contribution score of them reducing the delay. This actually converts the performance requirements of flow into resource requirements of this flow, hence, the routing problem can be converted into a job-scheduling problem in resource management. Second, a DRL agent deployed on an SDN controller continually interacts with the network for adaptively performing reasonable routing according to the network state, and optimally allocating network resources for traffic. We employ Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) to build the DRL-R. Finally, we demonstrate the effectiveness of DRL-R through extensive simulations. Benefitted from continually learning with a global view, DRL-R can improve throughput highest up to 40% and flow completion time highest up to 47% over OSPF. DRL-R can improve the load balance of link highest up to 18.8% over OSPF. Additionally, DDPG has better performance than DQN.