Hypothetical use of multidimensional adaptive testing for the assessment of student achievement in the programme for international student assessment

• Published in: Educational and psychological measurement Vol. 71; no. 3; pp. 503 - 522
• Main Authors: Frey, Andreas, Seitz, Nicki-Nils
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01-06-2011; Sage Publications; SAGE PUBLICATIONS, INC
• Subjects: Academic Achievement; Achievement Tests; Adaptive Testing; Bildungsforschung; Biological and medical sciences; Comparative Analysis; Computer Assisted Testing; Computerunterstütztes Verfahren; Conventional Testing; Deutschland; Educational psychology; Efficiency; Foreign Countries; Foreign students; Fundamental and applied biological sciences. Psychology; Germany; Item response theory; Item-Response-Theorie; Literacy; Measurement; PISA (Programme for International Student Assessment); PISA-Studie; Program for International Student Assessment; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Pupil and student. Academic achievement and failure; Schulleistungsmessung; Secondary School Students; Testen; Germany; multidimensional adaptive testing; Programme for International Student Assessment; item response theory; large-scale assessments; computerized adaptive testing; Human; Evaluation; Academic achievement; Item response theory; Psychometrics; Student; Test administration; Adolescent; Large scale; Computer aid
• ISSN: 0013-1644; 1552-3888; 1552-3888
• DOI: 10.1177/0013164410381521

The usefulness of multidimensional adaptive testing (MAT) for the assessment of student literacy in the Programme for International Student Assessment (PISA) was examined within a real data simulation study. The responses of N ¼ 14,624 students who participated in the PISA assessments of the years 2000, 2003, and 2006 in Germany were used to simulate MATwith different restrictions (unrestricted, treatment of link items, treatment of open items, content balance, unitwise item selection, all restrictions). Compared with conventional testing based on the booklet design of PISA 2006, unrestricted MAT increases measurement efficiency by 74% and reduces the average number of presented items from 55 to 26 without a loss in measurement precision. The incorporation of restrictions reduces the advantages of MAT. MAT is recommended for the assessment of newly introduced constructs but not for the assessment of the literacy domains in PISA. (Orig.).