A PAC algorithm in relative precision for bandit problem with costly sampling

• Published in: Mathematical methods of operations research (Heidelberg, Germany) Vol. 96; no. 2; pp. 161 - 185
• Main Authors: Friess, Marie Billaud, Macherey, Arthur, Nouy, Anthony, Prieur, Clémentine
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01-10-2022; Springer Nature B.V; Springer Verlag
• Subjects: Adaptive algorithms; Algorithms; Business and Management; Calculus of Variations and Optimal Control; Optimization; Estimates; Mathematics; Mathematics and Statistics; Operations research; Operations Research/Decision Theory; Optimization; Original Article; Probability distribution; Random variables; Sampling; Monte-Carlo estimates; Bandit algorithm; Probably approximately correct algorithm; Relative precision; Concentration inequalities; bounded random variables; relative precision; probably approximately correct algorithm; bandit algorithm
• ISSN: 1432-2994; 1432-5217
• DOI: 10.1007/s00186-022-00769-x

This paper considers the problem of maximizing an expectation function over a finite set, or finite-arm bandit problem. We first propose a naive stochastic bandit algorithm for obtaining a probably approximately correct (PAC) solution to this discrete optimization problem in relative precision, that is a solution which solves the optimization problem up to a relative error smaller than a prescribed tolerance, with high probability. We also propose an adaptive stochastic bandit algorithm which provides a PAC-solution with the same guarantees. The adaptive algorithm outperforms the mean complexity of the naive algorithm in terms of number of generated samples and is particularly well suited for applications with high sampling cost.