Efficient Approximate Range Aggregation Over Large-Scale Spatial Data Federation

• Published in: IEEE transactions on knowledge and data engineering Vol. 35; no. 1; pp. 418 - 430
• Main Authors: Shi, Yexuan, Tong, Yongxin, Zeng, Yuxiang, Zhou, Zimu, Ding, Bolin, Chen, Lei
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-01-2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Agglomeration; Algorithms; Approximation algorithms; Emergency response; Federations; Indexes; Public health; Queries; Query processing; range aggregation; Real-time systems; Sampling; Spatial data; Spatial data federation; Spatial databases; Throughput; Time complexity
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/TKDE.2021.3084141

Range aggregation is a primitive operation in spatial data applications and there is a growing demand to support such operations over a data federation, where the entire spatial data are separately held by multiple data providers ( a.k.a. , data silos). Data federations notably increase the amount of data available for data-intensive applications such as smart mobility planning and public health emergency responses. Yet they also challenge the conventional implementation of range aggregation queries because the raw data cannot be shared within the federation and the data partition at each data silo is fixed during query processing. These constraints limit the design space of distributed range aggregation query processing and render existing solutions inefficient on large-scale data. In this work, we propose the first-of-its-kind approximate algorithms for efficient range aggregation over spatial data federation. We devise novel single-silo sampling algorithms that process queries in parallel and design a level sampling based algorithm which reduces the time complexity of local queries at each data silo to <inline-formula><tex-math notation="LaTeX">O(\log \frac{1}{\epsilon })</tex-math> <mml:math><mml:mrow><mml:mi>O</mml:mi><mml:mo>(</mml:mo><mml:mo form="prefix">log</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>ε</mml:mi></mml:mfrac><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="tong-ieq1-3084141.gif"/> </inline-formula>, where <inline-formula><tex-math notation="LaTeX">\epsilon</tex-math> <mml:math><mml:mi>ε</mml:mi></mml:math><inline-graphic xlink:href="tong-ieq2-3084141.gif"/> </inline-formula> is the approximation ratio of the accuracy guarantee. Extensive evaluations with real-world data show that compared with state-of-the-arts, our solutions reduce the time cost and communication cost by up to <inline-formula><tex-math notation="LaTeX">85.1\times</tex-math> <mml:math><mml:mrow><mml:mn>85</mml:mn><mml:mo>.</mml:mo><mml:mn>1</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="tong-ieq3-3084141.gif"/> </inline-formula> and <inline-formula><tex-math notation="LaTeX">5.5\times</tex-math> <mml:math><mml:mrow><mml:mn>5</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="tong-ieq4-3084141.gif"/> </inline-formula> respectively, with average approximate errors of below 2.8 percent. In addition, our solutions yield a throughput of over 250 queries per second, achieving real-time responses for real-world bike-sharing applications.