AI Models Show Covert Racism Against African American English Speakers
Researchers from Stanford, University of Chicago, and the Allen Institute for AI found that large language models including OpenAI's GPT-3.5 and GPT-4 exhibit "covert racism" against speakers of African American English. When presented with text in AAE, the models assigned less prestigious jobs, convicted defendants of crimes, and recommended death sentences significantly more often than for text in Standard American English, without explicit mention of race.
What happened
A study published in Nature by researchers from Stanford University, the University of Chicago, and the Allen Institute for AI found that large language models including OpenAI's GPT-3.5 and GPT-4 exhibit covert bias against speakers of African American English (AAE). The researchers conducted experiments using text in AAE without explicitly mentioning race, drawing on racialized meanings of the stigmatized dialect.
When presented with text in AAE, the models assigned less prestigious jobs to speakers, convicted defendants of crimes more often, and recommended death sentences significantly more frequently in hypothetical murder trials than when the same content was presented in Standard American English.
Reported impact
The biases discovered were found to be more negative than the most unfavorable human stereotypes about African Americans recorded in academic studies. The researchers noted that larger and more advanced models showed stronger covert biases while simultaneously displaying weaker overt biases such as hate speech or racial profiling when directly asked about African Americans.
Index analysis
The study challenges the assumption that scaling up AI models and incorporating human feedback naturally leads to fairer outcomes. Researchers found that this approach can paradoxically strengthen unconscious or implicit biases while reducing explicit ones.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.