A new study from the University of Washington uncovers substantial racial, gender and intersectional biases in AI-based resume screening tools, raising questions about the fairness and accuracy of these systems in modern hiring processes.
In an era where artificial intelligence is increasingly integral to hiring processes, a new study by the University of Washington has highlighted significant biases within state-of-the-art large language models (LLMs) used for screening resumes. Presenting their findings at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society in San Jose on Oct. 22, the researchers pointed out the discriminatory tendencies of these AI tools regarding race and gender.
“The use of AI tools for hiring procedures is already widespread, and it’s proliferating faster than we can regulate it,” lead author Kyra Wilson, a doctoral student at UW’s Information School, said in a news release.
The study found that AI systems often favor white and male-associated names over Black and female-associated names. Specifically, the AI preferred white-associated names 85% of the time, while Black-associated names were favored only 9% of the time. Male-associated names were chosen 52% of the time compared to 11% for female-associated names.
In a comprehensive study involving over 550 real-world resumes, the UW team varied 120 names typically associated with white and Black men and women. Using three different LLMs from Mistral AI, Salesforce and Contextual AI, the researchers simulated job applications for more than 500 real-world job listings. This extensive exercise resulted in over 3 million resume-job description comparisons, providing robust data to analyze biases.
The findings revealed that not only do these AI tools exhibit clear preferences for white and male names, but they also show unique biases when intersectional identities, like Black males, are considered. In fact, the study found that names typically associated with Black males were never preferred over white male names. Conversely, Black female names were favored 67% of the time over 15% for Black male names.
“We found this really unique harm against Black men that wasn’t necessarily visible from just looking at race or gender in isolation,” Wilson added.
The implications of these findings are both disturbing and urgent. As more companies, including an estimated 99% of Fortune 500 businesses, rely on AI systems for hiring, these biases could perpetuate and even exacerbate existing inequalities in the workforce.
“Now that generative AI systems are widely available, almost anyone can use these models for critical tasks that affect their own and other people’s lives, such as hiring,” senior author Aylin Caliskan, an assistant professor at UW’s Information School, said in the news release.
Despite the efficiency touted by these AI systems, the study underscores the pressing need for regulatory measures and transparent, independent audits to understand and mitigate these biases. Outside of New York City, where a regulation demands auditing AI and automated employment decision tools, no such broad oversight exists.
The research emphasizes the necessity of exploring methods to reduce bias and align AI practices with fairness policies, extending the scope to other protected attributes like disability and age while prioritizing intersectional identities.
This study marks a significant step towards recognizing and addressing AI biases in hiring processes. By shining a spotlight on the hidden prejudices of automated systems, the study advocates for a more equitable and just AI-driven future in employment.