Simulation-Based Power Analysis for Factorial Analysis of Variance Designs

• Published in: Advances in methods and practices in psychological science Vol. 4; no. 1
• Main Authors: Lakens, Daniël, Caldwell, Aaron R.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01-01-2021
• Subjects: open materials; ANOVA; hypothesis test; power analysis; sample-size justification
• ISSN: 2515-2459; 2515-2467
• DOI: 10.1177/2515245920951503

Researchers often rely on analysis of variance (ANOVA) when they report results of experiments. To ensure that a study is adequately powered to yield informative results with an ANOVA, researchers can perform an a priori power analysis. However, power analysis for factorial ANOVA designs is often a challenge. Current software solutions do not allow power analyses for complex designs with several within-participants factors. Moreover, power analyses often need
η
p
2
or Cohen’s f as input, but these effect sizes are not intuitive and do not generalize to different experimental designs. We have created the R package Superpower and online Shiny apps to enable researchers without extensive programming experience to perform simulation-based power analysis for ANOVA designs of up to three within- or between-participants factors. Predicted effects are entered by specifying means, standard deviations, and, for within-participants factors, the correlations. The simulation provides the statistical power for all ANOVA main effects, interactions, and individual comparisons. The software can plot power across a range of sample sizes, can control for multiple comparisons, and can compute power when the homogeneity or sphericity assumption is violated. This Tutorial demonstrates how to perform a priori power analysis to design informative studies for main effects, interactions, and individual comparisons and highlights important factors that determine the statistical power for factorial ANOVA designs.