The Sensitivity of Country Ranks to Index Construction and Aggregation Choice: The Case of Immigration Policy

• Published in: Policy studies journal Vol. 47; no. 3; pp. 647 - 685
• Main Authors: Bjerre, Liv, Römer, Friederike, Zobel, Malisa
• Format: Journal Article
• Language: English
• Published: Washington Blackwell Publishing Ltd 01-08-2019
• Subjects: Agglomeration; aggregation method; Arithmetic; Datasets; Empirical analysis; Extreme values; Immigration; Immigration policy; index building/construction; index country ranks; Indexes; Integration; integration policy; Policies; Ratings & rankings; Social sciences; Values
• ISSN: 0190-292X; 1541-0072
• DOI: 10.1111/psj.12304

How sensitive are country ranks to the aggregation function used in index construction? This paper tests whether different aggregation functions come to different results in regard to the ranking of countries. Indices within the field of immigration and integration policy are analyzed, yet, the results pertain to index building across the social sciences. The paper discusses three aggregation methods: the arithmetic mean, the geometric mean, and a noncompensatory/non‐linear aggregation function based on the Condorcet method. In the empirical part, these three aggregation functions are applied to the family indicators for the year 2010 of the Immigration Policies in Comparison (IMPIC) dataset, a new dataset which measures immigration policies’ restrictiveness, as well as to the eight policy strands of the Migrant Integration Policy Index for the year 2014. Results show that the methods react differently to extreme values and thus result in different rank orders in the middle range. In the politicized field of immigration and integration policies, country ranks play a crucial role and this is shown to have profound real‐world implications. The paper thus urges researchers to be reflective of the assumptions of different aggregation functions, as these lead to different results.
国家排名对构建指数时所使用的聚合函数有多敏感？本文测试了不同的聚合函数是否会给国家排名带来不同的结果。尽管本文的分析限定在移民和融合政策内，但其结果适用于整个社会科学领域的指数构建。本文讨论了三种聚合方法：算术平均值，几何平均值和基于孔多塞法（Condorcet method）的非补偿或非线性聚合函数。在实证部分，这三种聚合函数被应用于移民政策比较（IMPIC）数据集之中的2010年家庭指数。IMPIC是一个新的数据集，它衡量了移民政策的限制性以及2014年移民融合政策指数（MIPEX）中的八个政策链。结果表明，这三种聚合方法对极值的反应不同，从而导致了在中间范围内数据的排名有所不同。在移民和融合政策的政治化领域，国家排名发挥着至关重要的作用，并显示出了深刻的现实影响。因此，本文呼吁研究人员考虑假设不同的聚合函数进行实证检验，因为这些假设会带来不同的结果。.