Malaysia and Indonesia Block Grok AI After Non-Consensual Sexual Deepfake Generation
Malaysia and Indonesia blocked Grok AI after it was reported to have generated non-consensual sexual deepfakes, particularly of women and children. South Korea's media regulator requested X implement protections for minors from explicit AI-generated sexual content. Multiple countries launched investigations into deepfake abuse and inadequate content moderation safeguards.
What happened
Malaysia and Indonesia became the first countries to block Grok, an AI chatbot developed by xAI and accessible through Elon Musk's X platform, citing its misuse to generate non-consensual sexual deepfakes and manipulated explicit images including those of women and children. Indonesia's Ministry of Communication and Digital Affairs stated that non-consensual sexual deepfakes represent serious violations of human rights, dignity and digital security. The digital space oversight director reported initial findings showed Grok lacked effective protections to prevent such misuse.
South Korea's media regulator requested X implement measures to protect minors from explicit sexual content generated by Grok, citing growing alarm over AI tools used to create non-consensual sexual images. The commission required X to prevent potential illegal activities on Grok and present steps to protect adolescent users from harmful content. Under South Korean law, social media platform operators must appoint officials responsible for minor protection and submit annual compliance reports to authorities.
Reported impact
Indonesia's communications minister stated that non-consensual sexual deepfakes generated by Grok represent violations of human rights, dignity and digital security, particularly affecting women and children.
South Korea's media regulator noted that production, distribution or storage of non-consensual sexual fake content is subject to criminal punishment under local regulations.
Index analysis
The blocking actions highlight growing global concern over misuse of generative AI tools for creating non-consensual intimate imagery. Grok's image generation and modification features reportedly enabled users to alter real photographs or create AI-generated images that regulators said could be misused for non-consensual sexually explicit content.
The incident demonstrates inadequate content moderation safeguards in the system, as the Indonesian regulator found Grok lacked effective protections to prevent users from generating prohibited content.
Governance controls
Indonesia and Malaysia implemented regional access blocks to prevent use of the platform. South Korea requested implementation of access restrictions and controls for minors, and required X to designate officials responsible for minor protection with annual compliance reporting.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.