Coflow scheduling and placement for packet-switched optical datacenter networks

• Published in: Photonic network communications Vol. 43; no. 2; pp. 156 - 164
• Main Authors: Wang, Lin, Wang, Xinbo, Tornatore, Massimo, Kim, Kwangjoon, Mukherjee, Biswanath
• Format: Journal Article
• Language: English
• Published: New York Springer US 2022; Springer Nature B.V
• Subjects: Algorithms; Characterization and Evaluation of Materials; Completion time; Computer Communication Networks; Computer networks; Computer Science; Data centers; Data transmission; Electrical Engineering; Network latency; Optical communication; Original Paper; Parallel flow; Placement; Priority scheduling; Routers; Scheduling; Semantics; Switching theory; Waveguide routers; Wavelength division multiplexing; Datacenter; Coflow placement and scheduling; Priority; Packet-switched optical network
• ISSN: 1387-974X; 1572-8188
• DOI: 10.1007/s11107-021-00958-4

Data-parallel computing applications (DPCAs) (e.g., MapReduce, web search, etc.) are driving the need of scalable, low-latency, high-speed, and energy-efficient datacenters, because a DPCA consists of a series of heavy-computation stages within a datacenter, and each stage contains multiple parallel flows that must be completed before next stage starts, referred to as “Coflow”. These parallel flows are grouped as a Coflow. Coflow is a networking abstraction to convey application-level communication requirements by exposing rich semantics of DPCAs to underlying networks, e.g., latency of data transmission between two computation stages, known as “Coflow Completion Time” (CCT). Packet-switched optical network (PSON) is a practical intra-datacenter interconnect solution for DPCAs, as it is designed as a low-complexity and scalable one-stage switching architecture, using advanced optical networking technologies, such as Arrayed Waveguide Grating Routers and wavelength-division multiplexing. In this work, we study how to minimize CCT in PSON-enabled datacenters by placing senders and receivers of Coflows to proper transceiver nodes and scheduling data transmission wisely, for which we propose a Coflow-aware placement and scheduling algorithm, consisting of Min-Priority placement algorithm and Priority-aware scheduling algorithm. They are designed to cooperate with each other to jointly minimize CCT. Numerical simulations demonstrate the benefits of joint design of Coflow placement and scheduling algorithm, compared to state-of-the-art scheduling and placement algorithms designed without correlation.