Indonesia and Malaysia block Grok AI over sexualized deepfake image generation
Grok, an AI image generation system operated by xAI and integrated with X, was used to create non-consensual sexualized deepfake images of real people, including reportedly minors. Indonesia and Malaysia became the first countries to formally block access to the platform in January 2026, citing inadequate safeguards and violations of digital safety and human dignity protections, amid broader international criticism.
What happened
Grok's image generation feature was used to create and manipulate photos of real people, particularly women, into sexually explicit or minimal clothing depictions without consent. Some deepfake images reportedly involved minors.
Indonesia became the first country globally to formally ban Grok on January 12, 2026, citing violations of regulations requiring electronic system operators to prevent distribution of prohibited content. Malaysia followed on January 13, 2026, blocking access after finding repeated misuse of Grok and determining X's response—limited to user reporting mechanisms—insufficient to address fundamental design risks.
X restricted the Grok image generation feature to paid subscribers in response to the controversy, and Malaysian authorities demanded X implement effective security systems before the block would be lifted.
Reported impact
The widespread circulation of sexualized deepfake images raised concerns about privacy violations, human dignity, and digital safety. Global criticism emerged from officials in the United States, United Kingdom, Germany, France, and Australia.
UK Prime Minister Keir Starmer called the practice shameful and disgusting. The UK government and Digital Technology Minister Liz Kendall indicated they were considering regulatory measures.
Index analysis
The incident demonstrates a vulnerability in Grok's image generation system that enabled misuse at scale. The design permitted users to edit real photographs without consent safeguards, allowing sexual exploitation of individuals' likenesses.
Governance controls
Indonesia and Malaysia cited lack of adequate safeguards by X, with Malaysia's MCMC noting that X's voluntary user reporting mechanisms failed to prevent fundamental harms from the system's design. Indonesia's Digital Minister Meutya Hafid characterized the use of Grok for explicit sexual content generation as a violation of human rights, dignity, and digital security.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.