Malaysia and Indonesia Block Grok AI Chatbot Over Deepfake Harms
Malaysia and Indonesia blocked access to Grok, an AI chatbot developed by xAI, citing its misuse for generating non-consensual sexual and explicit images. Authorities found that the system lacked effective safeguards to prevent users from creating and distributing deepfake pornographic content, particularly involving women and minors.
What happened
Malaysia and Indonesia blocked access to Grok, an AI chatbot developed by xAI and accessed through Elon Musk's X platform, citing concerns over the tool's ability to generate non-consensual sexual and explicit images. Indonesia's Ministry of Communications and Digital Affairs implemented the block on Saturday, and Malaysia's Communications and Multimedia Commission followed on Sunday, stating the measure was necessary to protect women, children, and the broader community.
According to Indonesia's General Director for Digital Space Oversight Alexander Sabar, initial investigations found that Grok lacks effective safeguards to prevent users from creating and distributing pornographic content based on real photographs of Indonesian citizens. The Malaysian commission stated it had previously notified X Corp. and xAI earlier in the month requesting stronger protections, but responses relied primarily on user-reporting mechanisms.
Reported impact
Indonesian authorities stated the government considers non-consensual sexual deepfakes a serious violation of human rights, dignity, and citizen safety in the digital space. Sabar emphasized that such practices risk violating privacy and photography rights when photographs are manipulated or shared without consent, causing psychological, social, and reputational harm.
Index analysis
Regulators in the two Southeast Asian countries noted that existing safeguards do not prevent the creation and distribution of generated pornographic content, particularly involving women and minors. The blocks reflect growing global concern over generative AI tools that can produce realistic images, audio, and text, while existing protections fail to prevent their misuse.
Governance controls
Grok had previously restricted image generation and editing for paying users following global backlash over sexualized deepfakes, but critics stated this did not fully resolve the problem. The Malaysian commission stated the restrictions would remain in place until effective safeguards are implemented, describing the block as a preventive and proportionate measure while legal and regulatory processes continue.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.