Google AI Overview results contain AI-generated sources, study finds
A study by Originality.ai found that 10.4% of Google's AI Overview results were likely generated by AI, with 12.8% of citations marked as AI-generated. The research raised concerns about potential model collapse when AI systems train on content generated by other AI models, which could reduce information accuracy online.
What happened
Originality.ai analyzed 29,000 Google searches in the 'Your Money or Your Life' (YMYL) category and found that 10.4% of AI Overview results were likely generated by AI and 12.8% of citations were marked as AI-generated.
The study identified that 52% of citations used in AI Overview were not found in the top 100 organic Google search results, and noted the risk of 'model collapse' when AI systems train on content generated by other AI models rather than human-verified information.
Reported impact
The trend of AI-generated content in search results could long-term threaten the accuracy of information online, as AI systems citing AI-generated sources may amplify errors and misinformation.
Users may receive AI Overview results that cite content created by machines, with uncertainty about source reliability and whether all links have been properly verified.
Index analysis
Researchers noted that when AI Overview results display AI-generated sources, they increase those sources' perceived credibility, making them more likely to be used in future AI training cycles, creating a feedback loop that could degrade information quality.
The study observed that 74.4% of citations in AI Overview were written by humans, while 15.2% were unclassifiable due to format limitations.
Governance controls
Google disputed the study's findings, stating that AI detectors are unreliable and that the citations included in AI Overview are dynamic, changing based on current relevance and usefulness to users.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.