Joint tracking of multiple quantiles through conditional quantiles

• Published in: Information sciences Vol. 563; pp. 40 - 58
• Main Authors: Hammer, Hugo Lewi, Yazidi, Anis, Rue, Håvard
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-07-2021
• Subjects: Data mining; Data Stream; Joint estimates; Quantile tracking; Real time analytics; Real time analytics; Joint estimates; Quantile tracking; Data mining; Data Stream
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2021.02.014

The estimation of quantiles is one of the most fundamental data mining tasks. As most real-time data streams vary dynamically over time, there is a quest for adaptive quantile estimators. The most well-known type of adaptive quantile estimators is the incremental one which documents the state-of-the art performance in tracking quantiles. However, the absolute vast majority of incremental quantile estimators fail to jointly estimate multiple quantiles in a consistent manner without violating the monotone property of quantiles. In this paper, first we introduce the concept of conditional quantiles that can be used to extend incremental estimators to jointly track multiple quantiles. Second, we resort to the concept of conditional quantiles to propose two new estimators. Extensive experimental results, based on both synthetic and real-life data, show that the proposed estimators clearly outperform legacy state-of-the-art joint quantile tracking algorithms in terms of accuracy while achieving faster adaptivity in the face of dynamically varying data streams.