Computer-based collaborative problem-solving assessment in Taiwan

• Published in: Educational psychology (Dorchester-on-Thames) Vol. 40; no. 9; pp. 1164 - 1185
• Main Authors: Kuo, Bor-Chen, Liao, Chen-Huei, Pai, Kai-Chih, Shih, Shu-Chuan, Li, Cheng-Hsuan, Mok, Magdalena Mo Ching
• Format: Journal Article
• Language: English
• Published: Dorchester-on-Thames Routledge 20-10-2020; Taylor & Francis Ltd
• Subjects: Achievement Tests; Adolescents; Collaboration; Collaborative problem solving (CPS); Computer Assisted Testing; computer-based assessment; Cooperative Learning; Foreign Countries; Gender Differences; Goodness of Fit; Grade 10; Grade 9; International Assessment; Item Response Theory; multidimensional item response theory; Multiple Choice Tests; Problem Solving; Secondary School Students; Skill Analysis; Students; Test Reliability; Taiwan
• ISSN: 0144-3410; 1469-5820
• DOI: 10.1080/01443410.2018.1549317

The current study explores students' collaboration and problem solving (CPS) abilities using a human-to-agent (H-A) computer-based collaborative problem solving assessment. Five CPS assessment units with 76 conversation-based items were constructed using the PISA 2015 CPS framework. In the experiment, 53,855 ninth and tenth graders in Taiwan were recruited, and a multidimensional item response analysis was used to develop CPS scales and represent the students' collaboration and problem solving performance. The results show that the developed H-A approach is feasible for measuring students' CPS skills, and the CPS scales are also shown to be reliable. In addition, the students' CPS performance scores are further explored and discussed under the PISA CPS framework.