Ofqual's Reliability Programme: a case study exploring the potential to improve public understanding and confidence

• Published in: Oxford review of education Vol. 39; no. 1; pp. 93 - 113
• Main Author: Newton, Paul E.
• Format: Journal Article
• Language: English
• Published: Oxford Routledge 01-02-2013; Taylor & Francis; Taylor & Francis Ltd
• Subjects: Accuracy; assessment; Case Studies; Confidence; Confidentiality; Educational Assessment; Educational evaluation; Educational policy; England; Error; Errors; Evaluation; Evaluation Research; Evidence; Examinations; Exit Examinations; Foreign Countries; Mass Media Effects; Ofqual; Public Opinion; Qualifications; Reliability; Secondary Education; Tests; United Kingdom (England)
• ISSN: 0305-4985; 1465-3915
• DOI: 10.1080/03054985.2012.760285

In May 2008, Ofqual established a two-year programme of research to investigate the nature and extent of (un)reliability within the qualifications, examinations and assessments that it regulated. It was particularly concerned to improve understanding of, and confidence in, this technically complex and politically sensitive phenomenon. The following article presents an account of this programme, from the perspective of one of its initiators, the author. It describes: the context prior to the programme, where little information on (un)reliability was routinely available to the public; the rationale for the programme, in terms of the tension between improving public understanding and the concomitant threat of decreasing public confidence; and ways in which aspects of the programme were constructed through media reports. It concludes with lessons learned from running the programme and with an extended discussion of the challenge of talking about reliability and error.