Multivariate analysis of variance for multilevel data: a simulation study comparing methods

• Published in: The Journal of experimental education Vol. 90; no. 1; pp. 173 - 190
• Main Author: Finch, W. Holmes
• Format: Journal Article
• Language: English
• Published: Washington Routledge 02-01-2022; Taylor & Francis Inc
• Subjects: ANOVA/MANOVA; Education; Hierarchical Linear Modeling; HLM; Hypothesis Testing; multilevel; multivariate (classical); Multivariate Analysis; Simulation; Statistical Analysis; Structural equation modeling; Structural Equation Models
• ISSN: 0022-0973; 1940-0683
• DOI: 10.1080/00220973.2020.1718058

Multivariate analysis of variance (MANOVA) is widely used to test the null hypothesis of equal multivariate means across 2 or more groups. MANOVA rests upon an assumption that error terms are independent of one another, which can be violated if individuals are clustered or nested within groups, such as schools. Ignoring such nesting can result in Type I error inflation, biased parameter estimates, and underestimated standard errors. This simulation study compared two approaches for testing the MANOVA null hypothesis of no group mean differences, when data come from a multilevel structure, under a variety of conditions. Results indicated that the multilevel MANOVA method of Snijders and Bosker, as well as an approach based on multilevel structural equation modeling (SEM) controlled Type I error under most conditions. In addition, the SEM based method yielded slightly higher power than did the multilevel MANOVA, under some conditions. Implications for practice are discussed.