AI Models Reportedly Provide More Misinformation on Elections in Spanish
An investigation by Proof News and partner organizations found that five leading AI models, including Google's Gemini and Anthropic's Claude, provided inaccurate information on election questions in Spanish at higher rates than in English. The study tested identical election questions in both languages and found 52 percent of Spanish responses contained inaccuracies compared to 43 percent of English responses.
What happened
An investigation by AI Democracy Projects, Proof News, the Science, Technology, and Social Values Lab at the Institute for Advanced Study, and Factchequeado tested five leading AI models on the same 25 election questions in both English and Spanish.
The investigation found that 52 percent of responses to Spanish queries contained inaccurate information compared to 43 percent of responses to English queries. Google's Gemini chatbot gave a different response to a voter fraud question in Spanish than in English, discussing methods to root out voter fraud rather than noting it is 'incredibly rare' as it stated in English.
Reported impact
The disparity in accuracy rates points to a troubling disparity in the quality of AI-produced election information in Spanish, the second most spoken language in the U.S., with 42 million people speaking Spanish at home in the U.S.
According to Miranda Bogen, director of the AI Governance Lab at the Center for Democracy and Technology, the high level of inaccurate responses to important election information across languages is disappointing, particularly given that the issues had been raised with companies before the election.
Index analysis
AI models are widely used in dozens of languages and are offered for translation. Google Gemini Live supports Spanish, and Gemini's documentation indicates it can interpret prompts and respond in Spanish. Anthropic acknowledges that Claude has mostly been trained in English but says Claude 3 can be used in different languages.
Governance controls
The investigation notes that companies need to do a better job detecting when people are asking election-related questions in Spanish in the first place.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.