A new study by UCL researchers reveals that AI systems amplify human biases, creating a feedback loop that can deepen user prejudices. The findings stress the importance of unbiased AI development.
Artificial intelligence systems tend to adopt and amplify human biases, leading users to become more prejudiced, according to a new study by researchers at University College London (UCL). This discovery points to a significant feedback loop where minor initial biases can increase human errors, potentially snowballing into more significant issues.
The study, published in Nature Human Behaviour, demonstrates real-world implications, indicating that people interacting with biased AI systems are more likely to underestimate women’s performance and overestimate the likelihood of white men holding high-status jobs.
“People are inherently biased, so when we train AI systems on sets of data that have been produced by people, the AI algorithms learn the human biases that are embedded in the data. AI then tends to exploit and amplify these biases to improve its prediction accuracy,” co-lead author Tali Sharot, a professor of cognitive science at UCL, said in a news release. “Here, we’ve found that people interacting with biased AI systems can then become even more biased themselves, creating a potential snowball effect wherein minute biases in original datasets become amplified by the AI, which increases the biases of the person using the AI.”
The researchers conducted experiments involving over 1,200 participants who interacted with various AI systems while completing different tasks.
In one experiment, an AI algorithm trained on a dataset of human responses displayed a bias toward judging faces as sad. Participants who later interacted with this AI system began to show a greater tendency to judge faces as sad, illustrating how the AI’s learned bias influenced human perceptions.
Similarly, participants assessing performance on tasks were more likely to overestimate men’s abilities after interacting with a gender-biased AI system.
Another part of the study involved the generative AI system Stable Diffusion, which amplified existing societal biases by overrepresenting white men as financial managers. Participants exposed to images generated by this AI were subsequently more inclined to select white men as likely financial managers.
“Not only do biased people contribute to biased AIs, but biased AI systems can alter people’s own beliefs so that people using AI tools can end up becoming more biased in domains ranging from social judgements to basic perception,” co-lead author Moshe Glickman, a research fellow in experimental psychology at UCL, said in the news release. “Importantly, however, we also found that interacting with accurate AIs can improve people’s judgements, so it’s vital that AI systems are refined to be as unbiased and as accurate as possible.”
Moreover, the study highlights that false beliefs about interacting with another human rather than AI resulted in participants internalizing biases to a lesser extent. This suggests that the perceived accuracy of AI plays a role in how deeply biases are adopted by users.
“Algorithm developers have a great responsibility in designing AI systems; the influence of AI biases could have profound implications as AI becomes increasingly prevalent in many aspects of our lives,” added Sharot.
The findings emphasize the pressing need for creating AI systems that are as unbiased and accurate as possible to mitigate their influence on human biases and ensure more equitable AI applications.