Mixture ratio modeling of dynamic systems

• Published in: International journal of adaptive control and signal processing Vol. 35; no. 5; pp. 660 - 675
• Main Authors: Kárný, Miroslav, Ruman, Marko
• Format: Journal Article
• Language: English
• Published: Bognor Regis Wiley Subscription Services, Inc 01-05-2021
• Subjects: Algorithms; approximate Bayesian estimation; black‐box dynamic model; data stream processing; Data transmission; Dynamic models; forgetting; Kullback–Leibler divergence; mixture model; Modelling; Probabilistic models; Statistical analysis; universal approximation
• ISSN: 0890-6327; 1099-1115
• DOI: 10.1002/acs.3219

Summary
Any knowledge extraction relies (possibly implicitly) on a hypothesis about the modelled‐data dependence. The extracted knowledge ultimately serves to a decision‐making (DM). DM always faces uncertainty and this makes probabilistic modelling adequate. The inspected black‐box modeling deals with “universal” approximators of the relevant probabilistic model. Finite mixtures with components in the exponential family are often exploited. Their attractiveness stems from their flexibility, the cluster interpretability of components and the existence of algorithms for processing high‐dimensional data streams. They are even used in dynamic cases with mutually dependent data records while regression and auto‐regression mixture components serve to the dependence modeling. These dynamic models, however, mostly assume data‐independent component weights, that is, memoryless transitions between dynamic mixture components. Such mixtures are not universal approximators of dynamic probabilistic models. Formally, this follows from the fact that the set of finite probabilistic mixtures is not closed with respect to the conditioning, which is the key estimation and predictive operation. The paper overcomes this drawback by using ratios of finite mixtures as universally approximating dynamic parametric models. The paper motivates them, elaborates their approximate Bayesian recursive estimation and reveals their application potential.