AI models generate election misinformation more frequently in Spanish than English
Nonprofit newsrooms testing AI models found they generate election-related misinformation in Spanish at significantly higher rates than in English, with Meta's Llama 3 showing particularly poor performance. The finding raises concerns about information disparities affecting Spanish-speaking voters during the 2024 U.S. election.
What happened
An analysis by Proof News and Factchequeado, in partnership with the Science, Technology and Social Values Lab at the Institute for Advanced Study, tested how popular AI models responded to specific prompts about the 2024 election ahead of November 5.
More than half of the election-related responses generated in Spanish contained incorrect information, compared to 43% of responses in English. Meta's Llama 3, which has powered the AI assistant inside WhatsApp and Facebook Messenger, generated nearly two-thirds of responses wrong in Spanish, compared to roughly half in English.
Reported impact
AI models are producing a stream of election-related falsehoods in Spanish more frequently than in English, muddying the quality of election-related information for Spanish-speaking voters, who are one of the nation's fastest-growing and increasingly influential voting blocs.
Voting rights groups worry AI models may deepen information disparities for Spanish-speaking voters during the election.
Index analysis
Large language models trained on extensive datasets sourced from the internet can generate AI-driven responses but remain prone to inaccuracies. The disparity between Spanish and English responses suggests potential gaps in training data quality or model performance across languages.
Governance controls
Voting rights advocates have urged Spanish-speaking voters to verify election information by researching multiple credible sources rather than relying on single entities or chatbot responses.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.