Riveter: Adaptive Query Suspension and Resumption Framework for Cloud Native Databases

• Published in: 2024 IEEE 40th International Conference on Data Engineering (ICDE) pp. 3975 - 3988
• Main Authors: Liu, Rui, Elmore, Aaron J., Franklin, Michael J., Krishnan, Sanjay
• Format: Conference Proceeding
• Language: English
• Published: IEEE 13-05-2024
• Subjects: Analytical models; Benchmark testing; cloud-native; cost-model; Costs; Estimation; Pipelines; query execution; query optimization; resource-adaptive; resumption; Runtime; suspension
• ISSN: 2375-026X
• DOI: 10.1109/ICDE60146.2024.00304

In modern cloud environments, ephemeral resources with intermittent availability and fluctuating monetary costs are becoming common. This dynamic nature presents a new challenge when deploying cloud-native databases: adaptive query execution, which can suspend queries when the resources are scarce or costs unexpectedly soar, and then resume them when the resources become available or cost-effective. Addressing this challenge requires the design and implementation of query suspension and resumption with a mechanism that can adaptively determine when, if, and how to suspend queries. In this paper, we propose Riveter, a query suspension and resumption framework that can adaptively pause ongoing queries using various strategies, including (1) a redo strategy that terminates queries and subsequently re-runs them, (2) a pipeline-level strategy that suspends a query once one of its pipelines has completed to reduce the storage requirements for intermediate data, (3) and a process-level strategy that enables the suspension of query execution processes at any given moment but generates a substantial volume of intermediate data for query resumption. We also devise a cost model to estimate query latency using various strategies and an algorithm to select the one that causes minimum latency. To demonstrate the effectiveness of Riveter, we conduct evaluations based on the TPC-H benchmark to investigate intermediate data persistence, strategy selection, and cost model-based estimation. Our results not only present the difference among the strategies of Riveter in terms of the size of persisted intermediate data and the time of triggering the suspension but also confirm the adaptive and efficient query suspension and resumption delivered by Riveter.