LVIS: Learning from Value Function Intervals for Contact-Aware Robot Controllers

• Published in: 2019 International Conference on Robotics and Automation (ICRA) pp. 7762 - 7768
• Main Authors: Deits, Robin, Koolen, Twan, Tedrake, Russ
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-05-2019
• Subjects: Force; Humanoid robots; Neural networks; Training; Trajectory optimization
• ISSN: 2577-087X
• DOI: 10.1109/ICRA.2019.8794352

Guided policy search is a popular approach for training controllers for high-dimensional systems, but it has a number of pitfalls. Non-convex trajectory optimization has local minima, and non-uniqueness in the optimal policy itself can mean that independently-optimized samples do not describe a coherent policy from which to train. We introduce LVIS, which circumvents the issue of local minima through global mixed-integer optimization and the issue of non-uniqueness through learning the optimal value function rather than the optimal policy. To avoid the expense of solving the mixed-integer programs to full global optimality, we instead solve them only partially, extracting intervals containing the true cost-to-go from early termination of the branch-and-bound algorithm. These interval samples are used to weakly supervise the training of a neural net which approximates the true cost-to-go. Online, we use that learned cost-to-go as the terminal cost of a one-step model-predictive controller, which we solve via a small mixed-integer optimization. We demonstrate LVIS on piecewise affine models of a cart-pole system with walls and a planar humanoid robot and show that it can be applied to a fundamentally hard problem in feedback control-control through contact.