Multi-Objective Scientific-Workflow Scheduling With Data Movement Awareness in Cloud

• Published in: IEEE access Vol. 7; pp. 177063 - 177081
• Main Authors: Wangsom, Peerasak, Lavangnananda, Kittichai, Bouvry, Pascal
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cloud computing; Clustering; cost; Data centers; data movement; directed acyclic graph (DAG); Dynamic scheduling; Energy consumption; extreme nondominated sorting genetic algorithm-III (E-NSGA-III); Genetic algorithms; makespan; multi-objective optimization; multilevel dependent node clustering (MDNC); Multiple objective analysis; nondominated sorting genetic algorithm-III (NSGA-III); Optimization; peer-to-peer clustering; Peer-to-peer computing; Processor scheduling; Scheduling; scientific workflows; Sorting algorithms; Task analysis; Workflow
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2957998

Due to serving several purposes simultaneously, running scientific workflows on dynamic environments such as cloud computing, has become multi-objective scheduling. Among these purposes, Cost and Makespan are probably the most two primitive objectives. Another critical factor in a large-scale scientific workflow is tremendous amount of data during execution. Therefore, this work also includes Data Movement as an additional objective as it has a major impact on network utilization and energy consumption in network equipment in cloud data center. In considering these three objectives, this work proposes a framework for scheduling solutions which combines a new nodes clustering technique in Directed Acyclic Graph (DAG) model known as Multilevel Dependent Node Clustering (MDNC) and the multi-objective optimization, Extreme Nondominated Sorting Genetic Algorithm-III (E-NSGA-III). E-NSGA-III is the recent extension of Nondominated Sorting Genetic Algorithm (NSGA-III). Five well-known scientific workflows, CyberShake, Epigenomics, LIGO, Montage, and SIPHT are selected as testbeds, while the commonly known Hypervolume is chosen as the performance metric. In this work, MDNC is also experimented with both NSGA-III. Comparison among three approaches, E-NSGA-III alone, E-NSGA-III with Peer-to-Peer clustering and E-NSGA-III with MDNC are carried out. The superiority of the proposed framework among them and its limitation are discussed.