A Methodology for Zumbo's Third Generation DIF Analyses and the Ecology of Item Responding

• Published in: Language assessment quarterly Vol. 12; no. 1; pp. 136 - 151
• Main Authors: Zumbo, Bruno D., Liu, Yan, Wu, Amery D., Shear, Benjamin R., Olvera Astivia, Oscar L., Ark, Tavinder K.
• Format: Journal Article
• Language: English
• Published: Philadelphia Routledge 02-01-2015; Routledge, Taylor & Francis Group
• Subjects: Bias; Culture; Holistic Approach; International Programs; Item Response Theory; Program for International Student Assessment; Reading comprehension; Reading Tests; Regression (Statistics); Regression analysis; Statistical Analysis; Test Bias; Test Items
• ISSN: 1543-4303; 1543-4311
• DOI: 10.1080/15434303.2014.972559

Methods for detecting differential item functioning (DIF) and item bias are typically used in the process of item analysis when developing new measures; adapting existing measures for different populations, languages, or cultures; or more generally validating test score inferences. In 2007 in Language Assessment Quarterly, Zumbo introduced the concept of Third Generation DIF. In the current article we introduce a new methodology, latent class logistic regression, for Zumbo's Third Generation DIF, whose foundation is a novel ecological model of item responding. The ecological model and the new statistical methodology are introduced, and a proof-of-concept is provided, in the context of an example of an international reading test focusing on DIF due to testing language. The new DIF framework is described and contrasted with other methods, Mplus code is provided, and the new method is shown to have potential for application in assessment.