Performance and quality of service of data and video movement over a 100 Gbps testbed

• Published in: Future generation computer systems Vol. 29; no. 1; pp. 230 - 240
• Main Authors: Kluge, Michael, Simms, Stephen, William, Thomas, Henschel, Robert, Georgi, Andy, Meyer, Christian, Mueller, Matthias S., Stewart, Craig A., Wünsch, Wolfgang, Nagel, Wolfgang E.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-01-2013
• Subjects: High performance computing; Lustre; Networking; Performance analysis; Lustre; Performance analysis; High performance computing; Networking
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2012.05.028

Digital instruments and simulations are creating an ever-increasing amount of data. The need for institutions to acquire these data and transfer them for analysis, visualization, and archiving is growing as well. In parallel, networking technology is evolving, but at a much slower rate than our ability to create and store data. Single fiber 100 Gbps networking solutions have recently been deployed as national infrastructure. This article describes our experiences with data movement and video conferencing across a networking testbed, using the first commercially available single fiber 100 Gbps technology. The testbed is unique in its ability to be configured for a total length of 60, 200, or 400 km, allowing for tests with varying network latency. We performed low-level TCP tests and were able to use more than 99.9% of the theoretical available bandwidth with minimal tuning efforts. We used the Lustre file system to simulate how end users would interact with a remote file system over such a high performance link. We were able to use 94.4% of the theoretical available bandwidth with a standard file system benchmark, essentially saturating the wide area network. Finally, we performed tests with H.323 video conferencing hardware and quality of service (QoS) settings, showing that the link can reliably carry a full high-definition stream. Overall, we demonstrated the practicality of 100 Gbps networking and Lustre as excellent tools for data management.
► The need for institutions to acquire and transfer data is growing. ► We tested data transfer on the first commercial single fiber 100 Gbps network. ► We used Lustre to simulate user interaction with a remote file system. ► We were able to use more than 94.4% of the theoretical available bandwidth. ► 100 Gbps networking and Lustre are excellent tools for data management.