AI Chatbots Give Gender-Biased and Race-Based Salary Advice
A Cornell University study found that AI chatbots including ChatGPT, GPT-4o mini, and Claude 3.5 Haiku give gender-biased and racially-biased salary negotiation advice, recommending lower pay for women and minorities than for men and white workers in identical positions. The research warns these tools may unintentionally perpetuate existing pay gaps.
What happened
A research study from Cornell University, led by Professor Ivan P. Yamshchikov of the Technical University of Applied Sciences Würzburg-Schweinfurt, systematically analysed multiple large language models including ChatGPT, GPT-4o mini, and Claude 3.5 Haiku. Researchers tested the models' salary negotiation advice by asking them questions about fictional individuals with different genders, races, and professional seniority levels.
The study, titled 'Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models', found that the salary negotiation advice offered by these chatbots showed clear bias. The models recommended lower starting salaries for women and minorities compared to men in identical situations, and in several instances advised women to request significantly lower pay than men with identical qualifications.
Reported impact
The research warns that chatbots like ChatGPT may unintentionally reinforce and widen existing gender and racial pay gaps rather than help close them. For women and minority job seekers in particular, these AI tools frequently advise them to request significantly lower pay than their male or white counterparts.
In one example cited, when asked about a starting salary for an experienced medical specialist in Denver, the AI suggested $400,000 for a man but only $280,000 for a woman—a $120,000 difference for the same position.
Index analysis
The research findings align with prior findings showing that even subtle signals like candidates' first names can trigger gender and racial disparities in employment-related prompts. The bias amongst AI models reflects patterns in the training data and existing societal pay gaps that the models learn to perpetuate.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.