Selecting a Selection Procedure

• Published in: Management science Vol. 53; no. 12; pp. 1916 - 1932
• Main Authors: Branke, Jurgen, Chick, Stephen E, Schmidt, Christian
• Format: Journal Article
• Language: English
• Published: Linthicum, MD INFORMS 01-12-2007; Institute for Operations Research and the Management Sciences
• Series: Management Science
• Subjects: Allocative efficiency; Applied sciences; Approximation; Bayesian analysis; Budget allocation; Business statistics; Business studies; Efficiency metrics; Exact sciences and technology; Frequentism; Management science; Operational research. Management science; Opportunity costs; Research methods; Sampling; Sampling distributions; Selection; Simulation; Simulations; Statistical analysis; Statistical discrepancies; statistics; Studies; Bayes estimation; Stopping rule; Uncertain system; Statistical analysis; Probabilistic approach; simulation; Empirical method; Sampling; Modeling; Adaptive method; Optimization; statistics
• ISSN: 0025-1909; 1526-5501
• DOI: 10.1287/mnsc.1070.0721

Selection procedures are used in a variety of applications to select the best of a finite set of alternatives. "Best" is defined with respect to the largest mean, but the mean is inferred with statistical sampling, as in simulation optimization. There are a wide variety of procedures, which begs the question of which selection procedure to select. The main contribution of this paper is to identify, through extensive experimentation, the most effective selection procedures when samples are independent and normally distributed. We also (a) summarize the main structural approaches to deriving selection procedures, (b) formalize new sampling allocations and stopping rules, (c) identify strengths and weaknesses of the procedures, (d) identify some theoretical links between them, and (e) present an innovative empirical test bed with the most extensive numerical comparison of selection procedures to date. The most efficient and easiest to control procedures allocate samples with a Bayesian model for uncertainty about the means and use new adaptive stopping rules proposed here.