Instagram and Threads automated moderation causes widespread unjustified account bans
Instagram and Threads users reported in October 2024 that the platforms' automated moderation systems incorrectly banned accounts and removed posts without justification. The systems misinterpreted common phrases, flagged adults as underage, and blocked business accounts for spam despite legitimate activity.
What happened
Users on Instagram and Threads reported widespread account bans and post removals without legitimate justification in early October 2024. The automated moderation systems reportedly blocked accounts for trivial reasons, including discussion of common words such as 'cracker,' comments on Hurricane Milton, and engagement with political topics.
Reported impact
Many users reported having their accounts disabled, with some identifying as minors under age 13 despite being adults. Business accounts promoting and selling video games had posts removed as spam. Users who submitted identity documents to prove their age were not restored access to their accounts.
Index analysis
The automated moderation algorithms appear to have misinterpreted common phrases and ordinary content, triggering removals with no logical justification. The age verification system also malfunctioned, incorrectly flagging adult users as minors.
Governance controls
Adam Mosseri, head of Instagram and Threads, acknowledged the issue and stated the team was examining and seeking a solution to the moderation problems.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.