Peer review and competition in the Art Exhibition Game

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 113; no. 30; pp. 8414 - 8419
• Main Authors: Balietti, Stefano, Goldstone, Robert L., Helbing, Dirk
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 26-07-2016
• Subjects: Art exhibits; Competition; Competitive Behavior - physiology; Creativity; Games, Experimental; Humans; Interpersonal Relations; Peer review; Peer Review - methods; Photic Stimulation; Social Sciences; Surveys and Questionnaires; fairness; innovation; competition; creativity; peer review
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.1603723113

To investigate the effect of competitive incentives under peer review, we designed a novel experimental setup called the Art Exhibition Game. We present experimental evidence of how competition introduces both positive and negative effects when creative artifacts are evaluated and selected by peer review. Competition proved to be a double-edged sword: on the one hand, it fosters innovation and product diversity, but on the other hand, it also leads to more unfair reviews and to a lower level of agreement between reviewers. Moreover, an external validation of the quality of peer reviews during the laboratory experiment, based on 23,627 online evaluations on Amazon Mechanical Turk, shows that competition does not significantly increase the level of creativity. Furthermore, the higher rejection rate under competitive conditions does not improve the average quality of published contributions, because more high-quality work is also rejected. Overall, our results could explain why many ground-breaking studies in science end up in lower-tier journals. Differences and similarities between the Art Exhibition Game and scholarly peer review are discussed and the implications for the design of new incentive systems for scientists are explained.