Trustworthy Generative AI Copilots for Data Analytics in Business Decision-Making

Project description

Organisations across all industries have recognised the value of using data to drive their business decisions. Traditionally, decision-makers without technical backgrounds had to rely on reports, dashboards, and other applications provided by their more tech-savvy colleagues. However, the advent of generative AI (GenAI) and large language models (LLMs) has democratised access to data analytics, and many companies have introduced GenAI analytics copilots with a natural language interface to help users of all skill levels to interact with data, extract insights, and become more data-driven in their decision-making.

Despite these promises, there are several challenges, not only related to the technical limitations of LLMs but also regarding users’ trust in the copilot’s responses during the interaction. Users’ tendency to either overtrust or undertrust often causes systematic over- or underreliance on the copilot’s responses, ultimately resulting in ineffective or inefficient decision-making. Our research project addresses this challenge by investigating the design of trustworthy analytics copilots and their impact on business decision-making among users without technical backgrounds.

Drawing on theories of mental models, we first explore the interplay between mental models and trust calibration in user–copilot interaction. These insights will then guide the design of trustworthy copilots that facilitate the development of shared mental models and help users adequately calibrate their trust in copilot responses. Finally, we empirically investigate how interactions with such analytics copilot influence the decision-making performance of users with different levels of data literacy.

Project team