GPT-4o's default framing biases research participants toward liberal political opinions
Research found that GPT-4o's default framing of historical events biases research participants toward more liberal political opinions compared to Wikipedia sources. The study demonstrates that AI chatbots' latent biases could influence public understanding of political issues at scale.
What happened
Daniel Karell and colleagues asked 1,912 research participants to read GPT-4o and Wikipedia summaries of two 20th century historical events: the 1919 Seattle General Strike and the 1968 Third World Liberation Front student protests. Some AI summaries were generated with explicit liberal or conservative framing; others used the model's default framing.
After reading the summaries, participants were asked to weigh in on issues related to the events, such as the appropriateness of labor strikes and the use of curricula to advance social justice causes, on a five-point scale from extremely conservative to extremely liberal. The researchers found that AI summaries with default framing and liberal framing led to more liberal opinions compared to Wikipedia summaries (Wikipedia average 3.47, default LLM average 3.57, liberal LLM 3.67), while conservative framing led to slightly more conservative opinions (3.36).
Reported impact
People seeking unbiased information from LLMs may be subtly influenced by the models' latent biases, which may have consequences for society writ large as members of the public increasingly turn to AI chatbots to understand their world.
Index analysis
The default framing in GPT-4o's historical summaries exhibited latent liberal bias that influenced public opinion on political issues. Even subtle biases in the underlying model could affect public understanding of historical events and contemporary political matters.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.