Indonesia bans Grok app due to deepfakes of a non-consensual sexual nature

Indonesian officials said on Saturday they had temporarily blocked access to xAI’s Grok chatbot.

This is one of the most aggressive moves yet by government officials responding to a deluge of artificial intelligence-generated sexual images — often depicting real women and minors, and sometimes showing assault and abuse — that Grok posted in response to user requests on the social network X. (X and xAI are part of the same company).

In a statement shared with The Guardian and other publications, Indonesian Communications and Digital Minister Mutya Hafid said, “The government considers the practice of non-consensual sexual deepfakes a serious violation of the human rights, dignity and security of citizens in the digital space.”

The ministry reportedly summoned X officials to discuss the issue.

Various government responses over the past week include an order from India’s Ministry of Information Technology for xAI to take action to prevent Grok from creating obscene content, as well as an order from the European Commission for the company to retain all documents related to Grok, which could pave the way for an investigation.

In the United Kingdom, communications regulator Ofcom said it would “conduct a rapid assessment to determine whether there are potential compliance issues that warrant investigation.” Prime Minister Keir Starmer said in an interview that Ofcom had his “full support to take the necessary action.”

And while in the US, the Trump administration appears to be staying silent on the issue (XAI CEO Elon Musk is a major Trump donor and led the administration’s controversial Department of Government Efficiency last year), Democratic senators have called on Apple and Google to remove X from their app stores.

xAI initially responded by posting an apparent first-person apology on Grok’s account, acknowledging that the post “violates ethical standards and possibly US laws” regarding child sexual abuse material. It later restricted the AI ​​photo creation feature to paying subscribers on X, although this restriction does not appear to affect the Grok app itself, which still allows anyone to create photos.

In response to a post questioning why the UK government hasn’t taken action against other AI image creation tools, Musk wrote: “They want any excuse for censorship.”