Malaysia and Indonesia suspend Grok over generation of sexually explicit images
Malaysia and Indonesia blocked access to Grok, an AI chatbot developed by xAI, after authorities found it was being misused to generate sexually explicit and non-consensual deepfake images. Regulators said the tool lacked effective safeguards to prevent creation and distribution of fake pornographic content involving women and minors.
What happened
Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. The Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok after 'repeated misuse' of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.
Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of residents. The Malaysian regulator said notices issued this month to X Corp and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms.
Reported impact
Indonesia's communication and digital affairs minister Meutya Hafid said: 'The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space.'
A disability rights advocate reported that a stranger prompted the chatbot to edit her photograph to depict her wearing a bikini. Despite adjusting privacy settings and reporting the image, the content remained accessible. She described the experience as distressing and humiliating.
Index analysis
The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. Regulators in the two south-east Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors.
Governance controls
The Malaysian regulator said the restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing, and that access will remain blocked until effective safeguards are put in place.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.