Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data

• Published in: IEEE transactions on visualization and computer graphics Vol. 25; no. 1; pp. 892 - 902
• Main Authors: Kale, Alex, Nguyen, Francis, Kay, Matthew, Hullman, Jessica
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-01-2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bars; Data visualization; Decision making; Decision theory; Encoding; hypothetical outcome plots; Judgments; Model accuracy; Modelling; Observers; psychometric functions; Public domain; Qualitative analysis; Sampling error; Statistical analysis; Statistical methods; Task analysis; Trends; Uncertainty; Uncertainty analysis; uncertainty visualization; Visualization
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2018.2864909

Animated representations of outcomes drawn from distributions (hypothetical outcome plots, or HOPs) are used in the media and other public venues to communicate uncertainty. HOPs greatly improve multivariate probability estimation over conventional static uncertainty visualizations and leverage the ability of the visual system to quickly, accurately, and automatically process the summary statistical properties of ensembles. However, it is unclear how well HOPs support applied tasks resembling real world judgments posed in uncertainty communication. We identify and motivate an appropriate task to investigate realistic judgments of uncertainty in the public domain through a qualitative analysis of uncertainty visualizations in the news. We contribute two crowdsourced experiments comparing the effectiveness of HOPs, error bars, and line ensembles for supporting perceptual decision-making from visualized uncertainty. Participants infer which of two possible underlying trends is more likely to have produced a sample of time series data by referencing uncertainty visualizations which depict the two trends with variability due to sampling error. By modeling each participant's accuracy as a function of the level of evidence presented over many repeated judgments, we find that observers are able to correctly infer the underlying trend in samples conveying a lower level of evidence when using HOPs rather than static aggregate uncertainty visualizations as a decision aid. Modeling approaches like ours contribute theoretically grounded and richly descriptive accounts of user perceptions to visualization evaluation.