Developing a construct-based assessment to examine students' analogical reasoning around physical models in Earth Science

• Published in: Journal of research in science teaching Vol. 49; no. 6; pp. 713 - 743
• Main Authors: Rivet, Ann E., Kastens, Kim A.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01-08-2012; Wiley-Blackwell
• Subjects: Analogical reasoning; Assessment; Cognitive Processes; Dynamic models; Earth Science; Grade 8; Grade 9; Learning; Logical Thinking; Lunar cycles; Motion; Pretests Posttests; Rasch Scaled Scores; Science Education; Science Process Skills; Scientific Concepts; Secondary School Science; Sophistication; Thinking Skills
• ISSN: 0022-4308; 1098-2736
• DOI: 10.1002/tea.21029

In recent years, science education has placed increasing importance on learners' mastery of scientific reasoning. This growing emphasis presents a challenge for both developers and users of assessments. We report on our effort around the conceptualization, development, and testing the validity of an assessment of students' ability to reason around physical dynamic models in Earth Science. Building from the research literature on analogical mapping and informed by the current perspectives on learning progressions, we present a three‐tiered construct describing the increasing sophistication of students' analogical reasoning around the correspondences and non‐correspondences between models and the Earth System: at the level of entities (Level 1), configurations in space or relative motion of entities (Level 2), and the mechanism or cause for observed phenomena (Level 3). Grounded in a construct‐centered design approach, we describe our process for developing assessments in order to examine and validate this construct, including how we selected topics and models, designed items, and developed outcome spaces. We present the specific example of one assessment centered on moon phases, which was administered to 164 8th and 9th grade Earth Science students as a pre/postmeasure. Two hundred ninety‐four responses were analyzed using a Rasch modeling approach. Item difficulties and student proficiency scores were calculated and analyzed regarding their relative performance with respect to the three levels of the construct. The analysis results provided initial evidence in support of the construct as conceived, with students displaying a range of analogical reasoning spanning all three construct levels. It also identified problematic items that merit further examination. Overall, the assessment has provided us the opportunity to better describe and frame the cognitive uses of models by students during learning situations in Earth Science. Implications for instruction and future directions for research in this area are discussed. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 713–743, 2012