HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics

• Main Authors: Götz, Markus, Coquelin, Daniel, Debus, Charlotte, Krajsek, Kai, Comito, Claudia, Knechtges, Philipp, Hagemeier, Björn, Tarnawa, Michael, Hanselmann, Simon, Siggel, Martin, Basermann, Achim, Streit, Achim
• Format: Journal Article
• Language: English
• Published: 11-11-2020
• Subjects: Computer Science - Distributed, Parallel, and Cluster Computing; Computer Science - Learning; Computer Science - Mathematical Software
• DOI: 10.48550/arxiv.2007.13552

To cope with the rapid growth in available data, the efficiency of data
analysis and machine learning libraries has recently received increased
attention. Although great advancements have been made in traditional
array-based computations, most are limited by the resources available on a
single computation node. Consequently, novel approaches must be made to exploit
distributed resources, e.g. distributed memory architectures. To this end, we
introduce HeAT, an array-based numerical programming framework for large-scale
parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch
as a node-local eager execution engine and distributes the workload on
arbitrarily large high-performance computing systems via MPI. It provides both
low-level array computations, as well as assorted higher-level algorithms. With
HeAT, it is possible for a NumPy user to take full advantage of their available
resources, significantly lowering the barrier to distributed data analysis.
When compared to similar frameworks, HeAT achieves speedups of up to two orders
of magnitude.