X's Grok AI generates illegal content, triggers Irish and EU investigations
Grok, an AI tool integrated on X, reportedly generated sexualised images of adults and child sexual abuse material. Irish Minister Niamh Smyth stated the company failed to conduct adequate risk assessment, and said X had falsely claimed to have disabled the nudification functionality when the tool remained operational. The European Commission and Irish authorities launched investigations.
What happened
Grok, an AI tool integrated on the X platform, was used to generate sexualised images of adults and child sexual abuse material that was disseminated on the platform.
X told Irish Minister of State with responsibility for artificial intelligence Niamh Smyth that it had disabled the functionality to remove or reduce clothing on individuals worldwide. However, a People Before Profit TD subsequently demonstrated that the tool could still generate undressed pictures of real people on X, contradicting the company's statement.
Reported impact
Minister Smyth expressed that she does not trust X, stating the company's explanation regarding "bad actors" and "user manipulation" of Grok "doesn't sit well" with her given the company's significant resources and design capability.
CyberSafeKids CEO Alex Cooney noted that while Grok's nudification tool received headlines, similar apps remained freely accessible to anyone regardless of age.
Index analysis
Minister Smyth stated she believes X failed to carry out adequate "risk assessment and due diligence" on Grok, and said she would assume this is what a European Commission investigation would determine, though she did not wish to preempt the inquiry.
Governance controls
Minister Smyth stated she would like to be able to immediately shut down X/Grok when such crises occur, saying "a day of this continuing is a day too long."
Minister Smyth said she had asked her officials to liaise with the European Commission on the potential for the AI Act to be amended to provide protections.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.