An AHP-based evaluation procedure for innovative educational projects A face-to-face vs. computer-mediated case study

• Published in: Omega (Oxford) Vol. 36; no. 5; pp. 754 - 765
• Main Authors: Garcia Melon, Monica, Aragones Beltran, Pablo, Gonzales Cruz, M. Carmen
• Format: Journal Article
• Language: English
• Published: Exeter Elsevier Ltd 01-10-2008; Elsevier; Elsevier Science Publishers; Pergamon Press Inc
• Series: Omega
• Subjects: AHP; AHP Group decisions Decision process Education; Applied sciences; Bewertung; Bildungswesen; Comparative analysis; Computer-assisted instruction; Computerunterstütztes Verfahren; Decision process; Decision theory. Utility theory; Education; Educational technology; Evaluation; Exact sciences and technology; Fallstudie; Group decisions; Innovations; Management; Multiple criteria decision making; Operational research and scientific management; Operational research. Management science; Project evaluation; Projektmanagement; Spanien; Studies; Technology application; United States; Universities and colleges; United States; Spain; AHP; Decision process; Education; Group decisions; Social decision; Computational geometry; Script; Innovation; Geometric mean; Analytic hierarchy process; Consensus; Teaching
• ISSN: 0305-0483; 1873-5274
• DOI: 10.1016/j.omega.2006.01.005

In this paper a procedure to evaluate proposals for Educational Innovation Projects is proposed. This methodology should help the Institute of Educational Sciences of the Politechnical University of Valencia to choose the best
Educational Project, the final aim being to provide the Administration with a stringent evaluation methodology, since the current evaluation methodology was found to be neither sufficiently objective nor systematic.
Since in the definition and evaluation of these Educational Projects diverse stakeholders are involved, the process has been approached as a MCDA carried out by a group of experts. Although a whole methodology is proposed, the paper has been focused on the weight assignment of the different criteria chosen by the experts.
The experts have been asked to act in two different ways:
in face-to-face meetings in which a consensus or compromise had to be reached, and
meetings at distance where the experts have given their individual judgements, which have been next combined using the geometric mean with the software EC 2000 [Expert Choice 2000 Team. Pittsburgh: Expert Choice, Inc.; 2001]. This procedure has allowed the authors to analyse the possible scenarios that the IES board team might come up against in the future. The main difference between the two ways of work is the dimension of physical space or the distance between the members of the evaluating team. This distance has a significant effect on the way team members relate to each other.