A Note on State Parameterizations in Output Feedback Reinforcement Learning Control of Linear Systems

• Published in: IEEE transactions on automatic control Vol. 68; no. 10; pp. 1 - 8
• Main Authors: Rizvi, Syed Ali Asad, Lin, Zongli
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-10-2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: adaptive dynamic programming; Algorithms; Control systems; Convergence; Linear systems; Matrices (mathematics); Observability; Observers; optimal control; Output feedback; output feedback control; Parameterization; Q-learning; Regulators; Reinforcement learning; State feedback; state parameterization
• ISSN: 0018-9286; 1558-2523
• DOI: 10.1109/TAC.2022.3228969

This note presents an analysis of the state parameterizations used in output feedback reinforcement learning (RL) control. Output feedback algorithms based on state parameterization involve additional conditions on the state parameterization beyond the standard conditions on the system matrices for their convergence to the optimal solution. It is shown that the state parameterization matrix needs to be of full row rank to guarantee the convergence of the output feedback RL algorithms. We present conditions in terms of the system matrices and the user-defined observer dynamics that ensure full row rank of the state parameterization matrix.