Simulation of variational Gaussian process NARX models with GPGPU

• Published in: ISA transactions Vol. 109; pp. 141 - 151
• Main Authors: Krivec, Tadej, Papa, Gregor, Kocijan, Juš
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01-03-2021
• Subjects: Gaussian process regression; General-purpose computing on graphics processing units; Monte Carlo simulation; Nonlinear identification of dynamical systems; Variational Gaussian process; Gaussian process regression; Variational Gaussian process; General-purpose computing on graphics processing units; Nonlinear identification of dynamical systems; Monte Carlo simulation
• ISSN: 0019-0578; 1879-2022
• DOI: 10.1016/j.isatra.2020.10.011

Gaussian processes (GP) regression is a powerful probabilistic tool for modeling nonlinear dynamical systems. The downside of the method is its cubic computational complexity with respect to the training data that can be partially reduced using pseudo-inputs. The dynamics can be represented with an autoregressive model, which simplifies the training to that of the static case. When simulating an autoregressive model, the uncertainty is propagated through a nonlinear function and the simulation cannot be evaluated in closed-form. This paper combines the variational methods of GP approximations with a nonlinear autoregressive model with exogenous inputs (NARX) to form variational GP (VGP-NARX) models. We show how VGP-NARX models, on average, better approximate a full GP-NARX model than more commonly used GP-NARX (FITC) model on 10 chaotic time-series. The modeling capabilities of VGP-NARX models are compared with the existing approaches on two benchmarks for modeling nonlinear dynamical systems. The advantage of general-purpose computing on graphics processing units (GPGPU) for Monte Carlo simulation on large validation data sets is addressed.
•Variational approximation of GP-NARX models for dynamic system identification.•Comparison of GP-NARX approximations for modeling chaotic time-series.•Comparison of VGP-NARX models with existing approaches on dynamic benchmark data.•General-purpose computing on graphics processing units for GP-NARX simulation.