Benchmark-Driven Configuration of a Parallel Model-Based Optimization Algorithm

• Published in: IEEE transactions on evolutionary computation Vol. 26; no. 6; pp. 1365 - 1379
• Main Authors: Rehbach, Frederik, Zaefferer, Martin, Fischbach, Andreas, Rudolph, Gunter, Bartz-Beielstein, Thomas
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-12-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Algorithms; Benchmark testing; Benchmarking; Benchmarks; Computational modeling; Configurations; exploratory landscape analysis (ELA); Gaussian process; Linear programming; model-based optimization; Optimization; parallelization; Prediction algorithms; Predictive models; simulation
• ISSN: 1089-778X; 1941-0026
• DOI: 10.1109/TEVC.2022.3163843

This article introduces a benchmarking framework that allows rigorous evaluation of parallel model-based optimizers for expensive functions. The framework establishes a relationship between estimated costs of parallel function evaluations (on real-world problems) to known sets of test functions. Such real-world problems are not always readily available (e.g., confidentiality and proprietary software). Therefore, new test problems are created by Gaussian process simulation. The proposed framework is applied in an extensive benchmark study to compare multiple state-of-the-art parallel optimizers with a novel model-based algorithm, which combines ideas of an explorative search for global model quality with parallel local searches to increase function exploitation. The benchmarking framework is used to configure good batch size setups for parallel algorithms systematically based on landscape properties. Furthermore, we introduce a proof of concept for a novel automatic batch size configuration. The predictive quality of the batch size configuration is evaluated on a large set of test functions and the functions generated by Gaussian process simulation. The introduced algorithm outperforms multiple state-of-the-art optimizers, especially on multimodal problems. Additionally, it proves to be particularly robust over various problem landscapes, and performs well with all tested batch sizes. Consequently, this makes it well suited for black-box kinds of problems.