Developing a methodology to assess the impact of research grant funding: A mixed methods approach

• Published in: Evaluation and program planning Vol. 43; pp. 105 - 117
• Main Authors: Bloch, Carter, Sørensen, Mads P., Graversen, Ebbe K., Schneider, Jesper W., Schmidt, Evanthia Kalpazidou, Aagaard, Kaare, Mejlgaard, Niels
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01-04-2014; Elsevier Science Ltd
• Subjects: Additionality; Bibliometrics; Financing; Grants; Humans; Impact analysis; Interviews as Topic; Methodology; Mixed methods; Politics; Program Evaluation - economics; Program Evaluation - methods; Program Evaluation - statistics & numerical data; Public Policy - economics; Public Policy - trends; Qualitative Research; Reflection; Research policy; Research Support as Topic - economics; Research Support as Topic - standards; Research Support as Topic - statistics & numerical data; Socioeconomic factors; Additionality; Research policy; Grants; Mixed methods; Impact analysis
• ISSN: 0149-7189; 1873-7870
• DOI: 10.1016/j.evalprogplan.2013.12.005

•We develop and apply a mixed methods approach to evaluate the impacts of research grants.•We illustrate the strengths and weaknesses of the approach through a recent evaluation.•The interaction of quantitative and qualitative analyses complements and validates results.•Qualitative analysis show that quantitative career effects mainly due to enhanced recognition.•Benefits of small grants in terms publications reflect a number of qualitative aspects.
This paper discusses the development of a mixed methods approach to analyse research funding. Research policy has taken on an increasingly prominent role in the broader political scene, where research is seen as a critical factor in maintaining and improving growth, welfare and international competitiveness. This has motivated growing emphasis on the impacts of science funding, and how funding can best be designed to promote socio-economic progress. Meeting these demands for impact assessment involves a number of complex issues that are difficult to fully address in a single study or in the design of a single methodology. However, they point to some general principles that can be explored in methodological design. We draw on a recent evaluation of the impacts of research grant funding, discussing both key issues in developing a methodology for the analysis and subsequent results. The case of research grant funding, involving a complex mix of direct and intermediate effects that contribute to the overall impact of funding on research performance, illustrates the value of a mixed methods approach to provide a more robust and complete analysis of policy impacts. Reflections on the strengths and weaknesses of the methodology are used to examine refinements for future work.