Inverse Optimal Control with Regular Language Specifications

• Published in: 2018 Annual American Control Conference (ACC) pp. 770 - 777
• Main Authors: Papusha, Ivan, Min Wen, Topcu, Ufuk
• Format: Conference Proceeding
• Language: English
• Published: AACC 01-06-2018
• Subjects: Automata; Convex functions; Cost function; Inverse problems; Optimal control; Task analysis; Trajectory
• ISSN: 2378-5861
• DOI: 10.23919/ACC.2018.8431646

Given a description of system dynamics, input constraints, and a cost function, the problem of optimal control is to find a sequence of inputs and a state trajectory that minimizes the total cost. A related problem posed by Kalman in the 1960s, called the inverse problem of optimal control, is to determine the objective function given optimal inputs and state trajectories. In this work, we pose the inverse problem of optimal control under temporal behavior specifications. In our setting, we are given demonstrations of optimal trajectories of a finite transition system, which may come from an expert, simulation, or some other process. In addition to these demonstrations, we are also given extra side information that optimal trajectories must satisfy temporal behavior constraints, expressed as an automaton over the state labels. We explore the value of this temporal side information in imputing an approximately optimal policy from finitely many demonstrations, and give a gridworld example with an eye toward extending the framework to hybrid systems with continuous states.