A hybrid estimator for selectivity estimation

• Published in: IEEE transactions on knowledge and data engineering Vol. 11; no. 2; pp. 338 - 354
• Main Authors: Yibei Ling, Wei Sun, Rishe, N.D., Xianjing Xiang
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-03-1999; IEEE Computer Society
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Computer science; control theory; systems; Computer Society; Estimators; Exact sciences and technology; Extrapolation; Information analysis; Information systems. Data bases; Memory organisation. Data processing; On-line systems; Online; Optimization; Parametric statistics; Performance analysis; Query processing; Runtime; Sampling methods; Selectivity; Software; Statistical distributions; Sun; Theoretical computing; Query language; Computer simulation; Information system; Estimation; Performance analysis; Algorithm; Database management system; Optimization
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/69.761667

Traditional sampling-based estimators infer the actual selectivity of a query based purely on runtime information gathering, excluding the previously collected information, which underutilizes the information available. Table-based and parametric estimators extrapolate the actual selectivity of a query based only on the previously collected information, ignoring online information, which results in inaccurate estimation in a frequently updated environment. We propose a novel hybrid estimator that utilizes and optimally combines the online and previously collected information. A theoretical analysis demonstrates that the online and previously collected information is complementary, and that the comprehensive utilization of the online and previously collected information is of value for further performance improvement. Our theoretical results are validated by a comprehensive experimental study using a practical database, in the presence of insert, delete and update operations. The hybrid approach is very promising in the sense that it provides an adaptive mechanism that allows the optimal combination of information obtained from different sources in order to achieve a higher estimation accuracy and reliability.