AI Chatbots Systematically Bias Judgment of Users, Study Finds
A research study found that AI chatbots like ChatGPT and Gemini systematically judge users using a rigid, demographic-based approach that amplifies biases compared to human decision-making. The models showed sizeable disparities in financial decisions based solely on demographic traits such as age.
What happened
A study published in the journal Proceedings of the Royal Society A analyzed 43,000 simulated decisions made by AI models including ChatGPT and Gemini alongside about 1,000 made by humans.
Researchers found that AI systems made judgements about people using a rigid, systematic approach that broke down individuals into scores on competence, integrity and kindness, whereas humans formed holistic impressions by blending multiple traits.
Reported impact
In financial scenarios, AI models showed sizeable differences in judgements based solely on demographic traits, with older people frequently given more favourable outcomes.
Researchers noted that AI's more rigid and less nuanced judgement style makes biases harder to detect, with the potential to amplify bias in real-world decisions about hiring, bank loans and medical advice.
Index analysis
The models appeared to grasp basic ingredients of trust such as competence, integrity and benevolence, favouring people who seemed competent, honest and well-intentioned.
However, AI followed a 'by-the-book' style of judgement that was consistent but less human, breaking down people into separate scores unlike the integrated way humans make trust judgements.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.