How Bayes factors change scientific practice

• Published in: Journal of mathematical psychology Vol. 72; pp. 78 - 89
• Main Author: Dienes, Zoltan
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-06-2016
• Subjects: Bayes factor; Confidence interval; Multiple comparisons; Null hypothesis; Planned vs post hoc; Stopping rule; Null hypothesis; Stopping rule; Bayes factor; Multiple comparisons; Planned vs post hoc; Confidence interval
• ISSN: 0022-2496; 1096-0880
• DOI: 10.1016/j.jmp.2015.10.003

Bayes factors provide a symmetrical measure of evidence for one model versus another (e.g. H1 versus H0) in order to relate theory to data. These properties help solve some (but not all) of the problems underlying the credibility crisis in psychology. The symmetry of the measure of evidence means that there can be evidence for H0 just as much as for H1; or the Bayes factor may indicate insufficient evidence either way. P-values cannot make this three-way distinction. Thus, Bayes factors indicate when the data count against a theory (and when they count for nothing); and thus they indicate when replications actually support H0 or H1 (in ways that power cannot). There is every reason to publish evidence supporting the null as going against it, because the evidence can be measured to be just as strong either way (thus the published record can be more balanced). Bayes factors can be B-hacked but they mitigate the problem because a) they allow evidence in either direction so people will be less tempted to hack in just one direction; b) as a measure of evidence they are insensitive to the stopping rule; c) families of tests cannot be arbitrarily defined; and d) falsely implying a contrast is planned rather than post hoc becomes irrelevant (though the value of pre-registration is not mitigated).
•Bayes factors would help science deal with the credibility crisis.•Bayes factors retain their meaning regardless of optional stopping.•Bayes factors retain their meaning despite other tests being conducted.•Bayes factors retain their meaning regardless of time of analysis.•The logic of Bayes helps illuminate the benefits of pre-registration.