Instagram's AI Safety Features Fail to Protect Teen Users
A research audit by child safety organizations and Northeastern University found that Instagram's safety features designed to protect teenage users were largely ineffective, with only 8 of 47 features working as intended and teenagers able to access harmful content and connect with inappropriate adults despite protections promised by Meta.
What happened
An audit by child safety advocacy organizations Molly Rose Foundation and Parents for Safe Online Spaces, supported by research from Northeastern University, evaluated Instagram's teen safety features announced by Meta over more than a decade. From 47 safety features tested, only 8 were rated as working effectively, while the remainder were found to be defective, unavailable, or substantially ineffective.
Reported impact
Researchers found that features designed to prevent teenagers from accessing content related to self-harm were easily bypassed with minor variations in written language. Anti-bullying message filters failed to activate even using identical phrases Meta had promoted in press releases. Features intended to redirect teenagers from harmful content never functioned during testing. Teen accounts could still connect with inappropriate unknown adults, view content related to suicide and self-harm, receive sexualized comments from adults without intervention, and be recommended search terms and accounts related to suicide, self-harm, eating disorders and illegal substances.
Index analysis
According to the report, 64 percent of safety tools or two-thirds of safety features were ineffective, and 17 percent posed risks of exposure to harmful content and abuse. The Instagram algorithm for users under age 13 displayed risky sexual behavior to gain engagement and recommended accounts of children as young as six years old.
Governance controls
Meta responded to the report by claiming its Teen Accounts feature provides significant protections and that the audit methodology was flawed. Spokesperson Andy Stone disputed the findings. The report was titled "Teen Accounts, Broken Promises" and was produced by organizations founded by parents who claimed their children died from effects of harmful content on Meta platforms.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.