A Study on Numerical Solutions of Hamilton-Jacobi-Bellman Equations Based on Successive Approximation Approach

• Published in: SICE journal of control, measurement, and system integration Online Vol. 13; no. 3; pp. 157 - 163
• Main Authors: Ichiro Maruta, Shuhei Nishida, Kenji Fujimoto
• Format: Journal Article
• Language: English
• Published: Taylor & Francis Group 2020
• Subjects: hamilton-jacobi-bellman equation; nonlinear optimal control; successive approximation
• ISSN: 1884-9970
• DOI: 10.9746/jcmsi.13.157

This paper presents a numerical approach to solve the Hamilton-Jacobi-Bellman (HJB) equation, which arises in nonlinear optimal control. In this approach, we first use the successive approximation to reduce the HJB equation, a nonlinear partial differential equation (PDE), to a sequence of linear PDEs called a generalized-Hamilton-Jacobi-Bellman (GHJB) equation. Secondly, the solution of the GHJB equation is decomposed by basis functions whose coefficients are obtained by the collocation method. This step is conducted by solving quadratic programming under the constraints which reflect the conditions that the value function must satisfy. This approach enables us to obtain a stabilizing solution of problems with strong nonlinearity. The application to swing up and stabilization control of an inverted pendulum illustrates the effectiveness of the proposed approach.