Identifying Properties of Real-World Optimisation Problems through a Questionnaire

• Main Authors: van der Blom, Koen, Deist, Timo M, Volz, Vanessa, Marchi, Mariapia, Nojima, Yusuke, Naujoks, Boris, Oyama, Akira, Tušar, Tea
• Format: Journal Article
• Language: English
• Published: 11-11-2020
• Subjects: Computer Science - Neural and Evolutionary Computing
• DOI: 10.48550/arxiv.2011.05547

Optimisation algorithms are commonly compared on benchmarks to get insight
into performance differences. However, it is not clear how closely benchmarks
match the properties of real-world problems because these properties are
largely unknown. This work investigates the properties of real-world problems
through a questionnaire to enable the design of future benchmark problems that
more closely resemble those found in the real world. The results, while not
representative as they are based on only 45 responses, indicate that many
problems possess at least one of the following properties: they are
constrained, deterministic, have only continuous variables, require substantial
computation times for both the objectives and the constraints, or allow a
limited number of evaluations. Properties like known optimal solutions and
analytical gradients are rarely available, limiting the options in guiding the
optimisation process. These are all important aspects to consider when
designing realistic benchmark problems. At the same time, the design of
realistic benchmarks is difficult, because objective functions are often
reported to be black-box and many problem properties are unknown. To further
improve the understanding of real-world problems, readers working on a
real-world optimisation problem are encouraged to fill out the questionnaire:
https://tinyurl.com/opt-survey