Instagram's content moderation AI fails to block explicit content from minors
Instagram's content moderation AI failed to block sexually explicit videos from reaching accounts registered as minors, according to a Folha de S. Paulo investigation. In tests of ten profiles, seven showed explicit content to a simulated minor account despite protection tools being enabled, and links associated with posts contained malware. Brazilian regulators are investigating whether Meta failed its legal obligation to prevent children from accessing illegal and pornographic content.
What happened
A Folha de S. Paulo investigation found that dozens of Instagram profiles displayed sexually explicit videos and pornographic content for at least three months, evading the platform's moderation systems and accumulating millions of views.
In tests with ten profiles distributing explicit material, seven displayed the content to an account registered as a minor under 16 years old, even with all protection tools activated. Only three showed age-restriction warnings.
Technology professionals identified malware in links associated with the posts, creating risks of device invasion and data theft in addition to exposure to inappropriate content.
Reported impact
Material reaches users registered as minors despite protection mechanisms being active.
The Instagram recommendation algorithm amplifies the reach of explicit content even when users do not follow the originating accounts.
Index analysis
Content creators use video overlay techniques to confuse automated detection systems, as the AI analyzes pixels for patterns that break when multiple video layers are stacked.
According to researchers, moderation models struggle when multiple media files are layered, creating confusion in detection, though systems improve while those seeking to evade them adapt accordingly.
Governance controls
Meta stated it proactively removed 96% of publications violating its policies against child sexual exploitation and 92% of publications with nudity between October and December 2025.
Since March, Meta automatically limits users aged 13 to 17 from accessing sensitive content. The profiles identified by the newspaper were forwarded for moderation and subsequently removed, according to the company.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.