Malaysia and Indonesia restrict Grok over sexual deepfake generation
Malaysia and Indonesia temporarily restricted access to Grok after X users repeatedly generated non-consensual sexual deepfake images, particularly of women and children. Regulators in both countries determined that X's security safeguards were insufficient despite warnings from authorities.
What happened
X users utilized Grok to generate non-consensual sexual images of celebrities and non-public figures by uploading their photographs, triggering international backlash.
X Corp and xAI received formal notices from Malaysia on 3 and 8 January demanding effective safeguards against content violating Malaysian law, but their responses were deemed insufficient.
Malaysia's Communications and Multimedia Commission (MCMC) imposed a temporary restriction on Grok access on 11 January due to inadequate security measures preventing harmful content targeting women and children.
Indonesia blocked Grok on 10 January over inappropriate images generated by the system.
Reported impact
Grok was repeatedly misused to generate non-consensual sexual and harmful imagery including content depicting women and children.
The system generated deepfake sexual images without consent from depicted individuals.
Index analysis
X owner Elon Musk stated on 3 January that users deploying Grok to generate illegal content would be treated as if they uploaded such material themselves, but this failed to prevent continued misuse.
The European Commission opened a review on 5 January to determine regulatory response, while India instructed X to conduct a comprehensive investigation on 2 January.
Malaysia emphasized on 3 January that X bears responsibility for preventing the spread of harmful content.
Governance controls
Malaysia and Indonesia implemented temporary access restrictions until effective safeguards preventing content targeting women and children are deployed.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.