SCTuner: An Autotuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems

• Published in: 2021 IEEE/ACM Sixth International Parallel Data Systems Workshop (PDSW) pp. 29 - 34
• Main Authors: Tang, Houjun, Xie, Bing, Byna, Suren, Carns, Philip, Koziol, Quincey, Kannan, Sudarsun, Lofstead, Jay, Oral, Sarp
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-11-2021
• Subjects: Benchmark testing; File systems; Libraries; Production; Runtime; Supercomputers; Tuners
• DOI: 10.1109/PDSW54622.2021.00010

In high-performance computing (HPC), scientific applications often manage a massive amount of data using I/O libraries. These libraries provide convenient data model abstractions, help ensure data portability, and, most important, empower end users to improve I/O performance by tuning configurations across multiple layers of the HPC I/O stack. We propose SCTuner, an autotuner integrated within the I/O library itself to dynamically tune both the I/O library and the underlying I/O stack at application runtime. To this end, we introduce a statistical benchmarking method to profile the behaviors of individual supercomputer I/O subsystems with varied configurations across I/O layers. We use the benchmarking results as the built-in knowledge in SCTuner, implement an I/O pattern extractor, and plan to implement an online performance tuner as the SCTuner runtime. We conducted a benchmarking analysis on the Summit supercomputer and its GPFS file system Alpine. The preliminary results show that our method can effectively extract the consistent I/O behaviors of the target system under production load, building the base for I/O autotuning at application runtime.