OPTCON3: An Active Learning Control Algorithm for Nonlinear Quadratic Stochastic Problems

• Published in: Computational economics Vol. 56; no. 1; pp. 145 - 162
• Main Authors: Blueschke-Nikolaeva, V., Blueschke, D., Neck, R.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-06-2020; Springer Nature B.V
• Subjects: Active control; Active learning; Algorithms; Behavioral/Experimental Economics; Computer Appl. in Social and Behavioral Sciences; Control algorithms; Control theory; Dynamic models; Dynamical systems; Economic analysis; Economic models; Economic policy; Economic Theory/Quantitative Economics/Mathematical Methods; Economics; Economics and Finance; Learning; Linear analysis; Machine learning; Math Applications in Computer Science; Nonlinear control; Nonlinear dynamics; Operations Research/Decision Theory; Optimal control; Optimization; Policies; Stochastic models; Stochastic processes; Uncertainty; Algorithms; Dual control; Stochastic optimal control; Active learning
• ISSN: 0927-7099; 1572-9974
• DOI: 10.1007/s10614-019-09949-0

In this paper, we describe the new OPTCON3 algorithm, which serves to determine approximately optimal policies for stochastic control problems with a quadratic objective function and nonlinear dynamic models. It includes active learning and the dual effect of optimizing policies, whereby optimal policies are used to learn about the stochastics of the dynamic system in addition to their immediate effect on the performance of the system. The OPTCON3 algorithm approximates the nonlinear model with a time-varying linear model and applies a procedure similar to that of Kendrick to the series of linearized models to calculate approximately optimal policies. The results for two simple economic models serve to test the OPTCON3 algorithm and compare it to previous solutions of the stochastic control problem. Initial evaluations show that the OPTCON3 approach may be promising to enhance our understanding of the adaptive economic policy problem under uncertainty.