Sparse tree search optimality guarantees in POMDPs with continuous observation spaces

• Main Authors: Lim, Michael H, Tomlin, Claire J, Sunberg, Zachary N
• Format: Journal Article
• Language: English
• Published: 05-06-2023
• Subjects: Computer Science - Learning; Computer Science - Robotics; Computer Science - Systems and Control; Statistics - Machine Learning
• DOI: 10.48550/arxiv.1910.04332

Partially observable Markov decision processes (POMDPs) with continuous state
and observation spaces have powerful flexibility for representing real-world
decision and control problems but are notoriously difficult to solve. Recent
online sampling-based algorithms that use observation likelihood weighting have
shown unprecedented effectiveness in domains with continuous observation
spaces. However there has been no formal theoretical justification for this
technique. This work offers such a justification, proving that a simplified
algorithm, partially observable weighted sparse sampling (POWSS), will estimate
Q-values accurately with high probability and can be made to perform
arbitrarily near the optimal solution by increasing computational power.