AI Language Models Reproduce Bias Against East Germans
Researchers found that large language models including ChatGPT and LeoLM systematically assign lower ratings to people from East German states compared to West German states across multiple categories, including positive, negative, and neutral attributes. The models reproduced regional stereotypes from their training data, and researchers warned of potential harm if such systems are used in employment or other decision-making contexts.
What happened
Researchers Anna Kruspe and Mila Stillman from Hochschule München examined how large language models including ChatGPT and the German-focused LeoLM evaluate Germany's 16 federal states when asked about various positive, negative, and neutral properties.
The models systematically assigned lower scores to people from East German states than West German states across all categories tested, including positive traits like diligence and attractiveness, and negative traits like laziness. When asked objective factual questions such as the average body temperature of residents in each state, most models also assigned lower values to East Germans, with only GPT-4 recognizing that body temperature is identical regardless of region.
Reported impact
The researchers concluded that the models reproduce learned societal patterns of rating the East generally lower without maintaining logical consistency, as seen in the paradoxical finding that East Germans were rated as simultaneously less diligent and less lazy.
Kruspe warned that the consequences could be severe if AI systems are used as supposedly neutral assessors in employment procedures or other decision-making contexts.
Index analysis
The models trained on large internet and media datasets learned and perpetuated regional stereotypes present in their training data. Stillman stated that the model learned a pattern that assigns systematically lower numbers to certain regions regardless of the property being evaluated.
Governance controls
Kruspe suggested that explicitly stating in prompts that a person's origin should not influence evaluation could help filter out biases, but acknowledged this approach is not reliably effective.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.