Scheduling HPC Workflows with Intel Optane Persistent Memory

HPC workloads and their Increasing data processing demands have led to using in situ execution, which couples simulation and analytics to reduce cross node memory accesses and their negative impact on overall performance. In situ executions can benefit from new classes of persistent memory technolog...

Full description

Saved in:

Bibliographic Details
Published in:	2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) pp. 56 - 65
Main Authors:	Venkatesh, Ranjan Sarpangala, Mason, Tony, Fernando, Pradeep, Eisenhauer, Greg, Gavrilovska, Ada
Format:	Conference Proceeding
Language:	English
Published:	IEEE 01-06-2021
Subjects:	Analytical models Benchmark testing Conferences Data models Data processing Distributed processing HPC In situ workflow Intel Optane Persistent Memory Random access memory Workflow scheduling
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	HPC workloads and their Increasing data processing demands have led to using in situ execution, which couples simulation and analytics to reduce cross node memory accesses and their negative impact on overall performance. In situ executions can benefit from new classes of persistent memory technologies, such as Intel ® Optane™ DC Persistent Memory (PMEM), which provide a denser, lower cost, and lower performance memory option for server class machines. However, PMEM creates a new set of trade-offs that must be considered to further improve performance for these HPC workloads and to realize the expected benefits. Prior work has only focused on describing how to tune for a single workload component, which may not yield optimal results for the entire workload.In this paper, we use a suite of workflows with different characteristics to understand the impact of using PMEM for in situ workflow executions with respect to different decisions on how PMEM is shared. Based on our experimental observations, we make recommendations for the considerations that must be incorporated for future workflow schedulers to maximize the benefits of the PMEM resource.
DOI:	10.1109/IPDPSW52791.2021.00017