Field trials of a novel toolkit for evaluating ‘intangible’ values-related dimensions of projects

• Published in: Evaluation and program planning Vol. 36; no. 1; pp. 1 - 14
• Main Authors: Burford, Gemma, Velasco, Ismael, Janoušková, Svatava, Zahradnik, Martin, Hak, Tomas, Podger, Dimity, Piggot, Georgia, Harder, Marie K.
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01-02-2013; Elsevier Science Ltd
• Subjects: Action research; Assessment; Business education; Civil society; Co-design; Data Collection - methods; Evaluation Utilization; Face validity; Group Processes; Humans; Indicators; Organizational change; Participatory evaluation; Peer generated; Process evaluation; Qualitative Research; Reproducibility of Results; Rigour; Success factors; Sustainable development; Systems Analysis; Utilisation-focused evaluation; Values; Participatory evaluation; Face validity; Utilisation-focused evaluation; Co-design; Peer generated; Action research; Values; Process evaluation; Indicators
• ISSN: 0149-7189; 1873-7870
• DOI: 10.1016/j.evalprogplan.2012.04.005

► We test a new toolkit for evaluating processes and outcomes related to values. ► Based on localisable indicators and mixed-methods evaluation. ► Combines participatory, process-based and utilisation-focused approaches to M&E. ► Useful in diverse civil society organisations, with potential for wider application. ► Enhances ‘process use’ benefits of evaluation.
A novel toolkit has been developed, using an original approach to develop its components, for the purpose of evaluating ‘soft’ outcomes and processes that have previously been generally considered ‘intangible’: those which are specifically values based. This represents a step-wise, significant, change in provision for the assessment of values-based achievements that are of absolutely key importance to most civil society organisations (CSOs) and values-based businesses, and fills a known gap in evaluation practice. In this paper, we demonstrate the significance and rigour of the toolkit by presenting an evaluation of it in three diverse scenarios where different CSOs use it to co-evaluate locally relevant outcomes and processes to obtain results which are both meaningful to them and potentially comparable across organisations. A key strength of the toolkit is its original use of a prior generated, peer-elicited ‘menu’ of values-based indicators which provides a framework for user CSOs to localise. Principles of participatory, process-based and utilisation-focused evaluation are embedded in this toolkit and shown to be critical to its success, achieving high face-validity and wide applicability. The emerging contribution of this next-generation evaluation tool to other fields, such as environmental values, development and environmental sustainable development, shared values, business, education and organisational change is outlined.