An empirical comparison of the relative effects of rater response biases on three rating scale formats

• Published in: Journal of applied psychology Vol. 59; no. 3; pp. 307 - 312
• Main Authors: Burnaska, Robert F, Hollmann, Thomas D
• Format: Journal Article
• Language: English
• Published: Washington, etc American Psychological Association 01-06-1974; American Psychological Association, etc
• Subjects: Human; Rating Scales; Response Bias
• ISSN: 0021-9010; 1939-1854
• DOI: 10.1037/h0036536

Compared Smith-Kendall type behaviorally anchored scales for derived performance dimensions (Format 1), scales for the same dimensions but without the behavioral anchors (Format 2), and scales for dimensions selected on an a priori basis (Format 3) on the basis of susceptibility to rater response biases. Raters were 30 graduate students and ratees were 3 associate professors whom the raters had had in succession during their 1st year of graduate study. Leniency error and composite halo error were present in all ratings; there was no evidence of relative or absolute halo errors in any ratings. There was some evidence that the use of scales for derived dimensions reduced leniency error and increased the amount of variance attributable to ratee differences. The scale reliabilities of the 3 formats were also determined. A discussion of the feasibility of obtaining relatively independent scales for several job performance dimensions is included. (15 ref)