Exploring data analysts' uncertainty reasoning strategies for effective uncertainty visualization design

Abstract

Despite its proven positive effects, visual data analysis rarely includes information about data uncertainty. Building on past research, we explore the hypothesis that effective uncertainty visualizations must support reasoning strategies that enable data analysts to utilize uncertainty information (‘uncertainty reasoning strategies’, UnReSt). Through this work, we seek to gain insights into the reasoning strategies employed by domain experts for incorporating uncertainty into their visual analysis. Additionally, we aim to explore effective ways of designing uncertainty visualizations that support these strategies. For this purpose, we developed a methodology involving online meetings that included think-aloud protocols and interviews. We applied the methodology in a user study with five domain experts from the field of epidemiology. Our findings identify, describe, and discuss the UnReSt employed by our participants, allowing for initial recommendations as a foundation for future design guidelines: uncertainty visualization should (i) visually support data analysts in adapting or developing UnReSt, (ii) not facilitate ignoring the uncertainty, (iii) aid in the definition of acceptable levels of uncertainty, and (iv) not hide uncertain parts of the data by default. We reflect on the methodology we developed and applied in our study, addressing challenges related to the recruiting process, the examination of an existing tool along with familiar tasks and data, the design of bespoke prototypes in collaboration with visualization experts, and the timing of the meetings. We encourage visualization researchers to adapt this methodology to gain deeper insights into the UnReSt of data analysts and how uncertainty visualization can effectively support them. The supplemental materials can be found at https://osf.io/s2nwf/.