Experiences on adaptive grid scheduling of parameter sweep applications

• Published in: 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing, 2004. Proceedings pp. 28 - 33
• Main Authors: Huedo, E., Montero, R.S., Llorente, I.M.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2004
• Subjects: Adaptive scheduling; Bioinformatics; Computational fluid dynamics; Computer applications; Dynamic scheduling; Fault tolerance; Fluid dynamics; Grid computing; Processor scheduling; Scheduling algorithm
• ISBN: 0769520839; 9780769520834
• ISSN: 1066-6192; 2377-5750
• DOI: 10.1109/EMPDP.2004.1271423

Grids offer a dramatic increase in the number of available compute and storage resources that can be delivered to applications. This new computational infrastructure provides a promising platform to execute loosely coupled, high-throughput parameter sweep applications. This kind of applications arises naturally in many scientific and engineering fields like bioinformatics, computational fluid dynamics (CFD), particle physics, etc. The efficient execution and scheduling of parameter sweep applications is challenging because of the dynamic and heterogeneous nature of grids. We present a scheduling algorithm built on top of the GridWay framework that combines: (i) adaptive scheduling to reflect the dynamic grid characteristics; (ii) adaptive execution to migrate running jobs to better resources and provide fault tolerance; (iii) re-use of common files between tasks to reduce the file transfer overhead. The efficiency of the approach is demonstrated in the execution of a CFD application on a highly heterogeneous research testbed.