Could this be Racially Biased? Displaying Training Data Diversity and Soliciting Feedback to Help Users Calibrate Trust in AI

Cheng Chen & S. Shyam Sundar

Training data diversity is important for ensuring group fairness in decisions made by AI algorithms. This study explores if displaying racial diversity in training data and labelers’ backgrounds on the interface will shape user trust in AI, and whether that will break down if the system shows racial bias in its performance. In a 2 (racial diversity in training data: presence vs. absence) × 2 (racial diversity in labelers’ backgrounds: presence vs. absence) × 2 (AI performance: racially biased vs. unbiased) × 2 (user feedback: yes vs. no) experiment, we discovered that the presence of racial diversity in training data and labelers’ backgrounds triggers the representativeness heuristic, which leads to higher expectations of AI fairness and accuracy as well as trust in AI. However, inviting user feedback lowers behavioral trust when there is no obvious bias in AI performance. Implications for communicating training data quality via explainable interfaces are discussed.

Is this AI trained on Credible Data? The Effects of Labeling Quality and Performance Bias on User Trust

Cheng Chen & S. Shyam Sundar

To promote data transparency, frameworks such as CrowdWorkSheets encourage documentation of annotation practices on the interfaces of AI systems, but we do not know how they affect user experience. Will the quality of labeling affect perceived credibility of training data? Does the source of annotation matter? Will a credible dataset persuade users to trust a system even if it shows racial biases in its predictions? We conducted a user study (N = 430) with a prototype of a classification system, using a 2 (labeling quality: high vs. low) × 4 (source: others-as-source vs. self-as-source cue vs. self-as-source voluntary action, vs. self-as-source forced action ) × 3 (AI performance: none vs. biased vs. unbiased) experiment. We found that high-quality labeling leads to higher perceived training data credibility, which in turn enhances users’ trust in AI, but not when the system shows bias. Practical implications for explainable and ethical AI interfaces are discussed.

What do Users Get Out of AI Technologies? Construction and Validation of an AI Gratifications Scale

Cheng Chen, Carlina DiRusso, Ruosi Shao, Hyun Yang, Michael Krieger, & S. Shyam Sundar

This study explores why people use AI-based tools, with a focus on identifying the major gratifications obtained by them. While prior studies have identified specific gratifications obtained by users of specific AI tools, the literature is yet to identify a universal set of gratifications that is common across most AI interfaces. As HCI designers increasingly treat AI as a distinct technology, it is important to identify a common set of factors that shape user experience of AI. With this goal, and guided by the Uses and Gratifications (U&G) framework, we conducted two survey studies (combined N = 1,264), which resulted in a reliable and valid 29-item AI Gratifications Scale, that loaded cleanly onto seven factors: anthropomorphism, privacy consciousness, informality, seamlessness, accuracy, fairness, and acknowledgment of limitations. This 7-factor gratifications scale was invariant across nine major AI functions. We discuss practical implications of these findings for designing and assessing human-AI interaction.