Google AI Overviews spread health misinformation and scam phone numbers
Google's AI Overviews feature displays AI-generated medical summaries without prominent safety disclaimers when users first see them, and scammers reportedly inject fake customer service phone numbers that the system presents as verified facts, creating risks of financial fraud and reliance on inaccurate medical information.
What happened
Google's AI Overviews, which display AI-generated summaries at the top of search results, present medical advice without safety disclaimers when users first see the information. The Guardian found that Google only issues warnings if users click a 'Show more' button to request additional health information, and even then the disclaimer appears below the medical advice in smaller, lighter font.
Scammers are hijacking AI Overviews by injecting fake customer service phone numbers into the summaries. According to reporting by Wired, the AI feature presents these numbers as facts without verifying them, potentially directing users to scammers impersonating legitimate companies.
Reported impact
Users may rely on AI-generated medical advice that contains errors or hallucinations without being adequately warned of the risks. An MIT researcher stated that the absence of prominent disclaimers creates critical dangers, as advanced AI models still hallucinate misinformation and exhibit biases that prioritize user satisfaction over accuracy.
People searching for company contact numbers may be directed to scammers instead of legitimate customer service, risking financial loss and personal information theft.
Index analysis
The AI Overviews system summarizes information from multiple sources without verifying the accuracy of phone numbers or medical claims. The feature's design—presenting summaries as authoritative before users can see underlying sources—makes users more vulnerable to misinformation and scams.
Governance controls
Google stated that AI Overviews 'encourage people to seek professional medical advice' and frequently mention seeking medical attention within summaries when appropriate. Google says it recently added safeguards to improve scam detection in AI Overviews but encourages users to verify phone numbers through additional searches.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.