AI chatbots spread misinformation during India-Pakistan conflict
During an India-Pakistan conflict, AI chatbots including Grok, ChatGPT, and Gemini provided false fact-checks when users sought verification of claims on social media. The systems misidentified video footage, fabricated details about images, and repeated falsehoods, underscoring their unreliability as fact-checking tools at a time when platforms have reduced human fact-checkers.
What happened
During India's four-day conflict with Pakistan, social media users turned to AI chatbots including Grok, ChatGPT, and Gemini for fact-checking, but the systems provided misinformation instead.
Grok wrongly identified old video footage from Sudan's Khartoum airport as a missile strike on Pakistan's Nur Khan airbase. Unrelated footage of a building on fire in Nepal was misidentified as 'likely' showing Pakistan's military response to Indian strikes.
Grok also labeled an AI-generated video of a giant anaconda in the Amazon River as 'genuine' and fabricated supporting details. When AFP fact-checkers asked Gemini about an AI-generated image of a woman, it confirmed its authenticity and fabricated details about the woman's identity and where the image was taken.
Reported impact
Users increasingly rely on AI chatbots as fact-checking tools at a time when tech platforms have scaled back investments in human fact-checkers, according to NewsGuard researcher McKenzie Sadeghi.
NewsGuard's research found that 10 leading chatbots were prone to repeating falsehoods, including Russian disinformation narratives and false claims related to the Australian election. The Tow Center for Digital Journalism at Columbia University found that chatbots were 'generally bad at declining to answer questions they couldn't answer accurately, offering incorrect or speculative answers instead.'
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.