Remote visual analysis of large turbulence databases at multiple scales

The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visua...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of parallel and distributed computing Vol. 120; pp. 115 - 126
Main Authors:	Pulido, Jesus, Livescu, Daniel, Kanov, Kalin, Burns, Randal, Canada, Curtis, Ahrens, James, Hamann, Bernd
Format:	Journal Article
Language:	English
Published:	United States Elsevier Inc 01-10-2018 Elsevier
Subjects:	Computer Science Data Reduction Distributed Systems MATHEMATICS AND COMPUTING Remote Visualization Turbulence Wavelets Wavelets Turbulence Databases Distributed systems Data reduction Remote visualization
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visualization of such datasets was restricted to scientists with access to large supercomputers. The public Johns Hopkins Turbulence database simplifies access to multi-terabyte turbulence datasets and facilitates the computation of statistics and extraction of features through the use of commodity hardware. We present a framework designed around wavelet-based compression for high-speed visualization of large datasets and methods supporting multi-resolution analysis of turbulence. By integrating common technologies, this framework enables remote access to tools available on supercomputers and over 230 terabytes of DNS data over the Web. The database toolset is expanded by providing access to exploratory data analysis tools, such as wavelet decomposition capabilities and coherent feature extraction. •Data-level wavelet compression reduces bandwidth, memory and compute footprint.•Latency is improved between database components and multi-user support is introduced.•Remote visualization is enabled on a large database cluster using commodity hardware.•New wavelet analysis tools are demonstrated for existing turbulence data cluster.
Bibliography:	LA-UR-17-20757 AC52-06NA25396 USDOE National Nuclear Security Administration (NNSA)
ISSN:	0743-7315 1096-0848
DOI:	10.1016/j.jpdc.2018.05.011