Distributed smart disks for I/O-intensive workloads on switched interconnects

• Published in: Future generation computer systems Vol. 22; no. 5; pp. 643 - 656
• Main Authors: Chiu, Steve C., Liao, Wei-keng, Choudhary, Alok N.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-04-2006
• Subjects: Cluster computing; InfiniBand; Parallel I/O; Storage interconnect; Parallel I/O; InfiniBand; Storage interconnect; Cluster computing
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2005.09.007

Disk-based smart storage represents processor-embedded active I/O devices that are equipped with a dedicated network interface controller and on-disk storage (memory and disk). Today’s I/O-intensive workloads require architectures whose performance scales with increasing storage capacity. The advent of switch-based, low-latency and high-bandwidth network protocols, such as Virtual Interface Architecture and InfiniBand, promises to improve both performance and scalability beyond current state-of-the-art I/O interconnect topologies. This paper evaluates a disk-based distributed smart storage architecture composed of fully distributed processor-embedded disks that are interconnected by a high-performance switched network, and investigates the performance of the proposed processing model on different storage interconnects with representative workloads, including TPC-H queries, association rules mining, high-dimensional data clustering, and two-dimensional fast Fourier transform (FFT).