How Much Is Enough? Evaluating Intervention Implementation Efficiently

• Published in: Assessment for effective intervention Vol. 44; no. 2; pp. 135 - 144
• Main Authors: Fritz, Ronda, Harn, Beth, Biancarosa, Gina, Lucero, Audrey, Flannery, K. Brigid
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01-03-2019; SAGE Publications and Hammill Institute on Disabilities; SAGE PUBLICATIONS, INC
• Subjects: Efficiency; Instructional Effectiveness; Interrater Reliability; Intervention; Kindergarten; Observation; Program Evaluation; Program Implementation; Reliability; Scores; Small Group Instruction; instruction; observation; interventions
• ISSN: 1534-5084; 1938-7458
• DOI: 10.1177/1534508418772909

This study investigated the use of brief observations to measure implementation of small group interventions using the Quality of Intervention Delivery and Receipt (QIDR) tool. Videos of 10-min segments representing the beginning, middle, and end of each 30-min intervention lesson were coded for implementation. Results indicated that (a) reliability can be achieved when observing a portion of a lesson; (b) QIDR scores obtained from 10-min segments are significantly correlated with the 30-min observations; and (c) post hoc analysis indicate the tool may be sensitive to changes in implementation quality at the intervention group level across time. Implications for research and practice are discussed, including the utility of shorter observations to increase the monitoring of intervention implementation to potentially improve outcomes.