Aftershocks of Chile’s Earthquake for an Ongoing, Large-Scale Experimental Evaluation

• Published in: Evaluation review Vol. 35; no. 2; pp. 103 - 117
• Main Authors: Moreno, Lorenzo, Treviño, Ernesto, Yoshikawa, Hirokazu, Mendive, Susana, Reyes, Joaquín, Godoy, Felipe, Del Río, Francisca, Snow, Catherine, Leyva, Diana, Barata, Clara, Arbour, MaryCatherine, Rolla, Andrea
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01-04-2011; SAGE PUBLICATIONS, INC
• Subjects: Adaptation, Psychological; Child; Child Welfare - statistics & numerical data; Child, Preschool; Childhood; Chile; Crisis Management; Data Collection - methods; Decision Making; Decisions; Design of experiments; Disaster relief; Disasters; Early Childhood Education; Earthquakes; Earthquakes - statistics & numerical data; Evaluation; Experiments; Foreign Countries; Formative Evaluation; Governance; Health technology assessment; Human development; Humans; Impact analysis; Impact Evaluation; Intervention; Latin America; Methodology (Data Collection); Natural Disasters; Performance Factors; Program Administration; Program Evaluation; Program Implementation; Program Termination; Relief Work - statistics & numerical data; Research Design; Social planning; Social Programs; Social service; Social work; Stress, Psychological; Trauma; Young Children; Chile; Latin America and the Caribbean; impact evaluation; external shocks; institutional factors; human development
• ISSN: 0193-841X; 1552-3926
• DOI: 10.1177/0193841X11400685

Evaluation designs for social programs are developed assuming minimal or no disruption from external shocks, such as natural disasters. This is because extremely rare shocks may not make it worthwhile to account for them in the design. Among extreme shocks is the 2010 Chile earthquake. Un Buen Comienzo (UBC), an ongoing early childhood program in Chile, was directly affected by the earthquake. This article discusses (a) the factors the UBC team considered for deciding whether to put on hold or continue implementation and data collection for this experimental study; and (b) how the team reached consensus on those decisions. A lesson learned is that the use of an experimental design for UBC insured that the evaluation’s internal validity was not compromised by the earthquake’s consequences, although cohort comparisons were compromised. Other lessons can be transferred to other contexts where external shocks affect an ongoing experimental or quasi-experimental impact evaluation.