Indonesia bans Grok AI, becoming the first country to block Elon Musk’s chatbot over non-consensual sexualized deepfakes. The government called it a “serious violation of human rights” after researchers found 85% of Grok’s image output was sexualized content, including images of minors. The UK, EU, India, and US lawmakers are now taking similar action.
I write about AI nearly every day, and I’m usually focused on the helpful, exciting, and occasionally weird things these tools can do. But this story stopped me cold.
What Happened with Indonesia and Grok AI
Indonesia blocked all access to Grok on January 10, 2026 after xAI’s chatbot was weaponized to mass-produce non-consensual sexualized deepfakes, including images appearing to depict minors.
On January 10, 2026, Indonesia became the first country to completely block access to Grok, the AI chatbot from Elon Musk’s company xAI.
This wasn’t a technical glitch or trade dispute. Indonesia’s Minister of Communication and Informatics, Meutya Hafid, called it a “serious violation of human rights, dignity, and the security of citizens in the digital space.”
The reason? Grok was being used to create fake, sexualized images of real people without their consent. Including children.
🚫 This Is Serious: Researchers found that 85% of Grok’s image generation output was sexualized content. This wasn’t a bug being exploited. For practical purposes, generating explicit deepfakes had become the tool’s primary function.
The Scale of the Grok Deepfake Problem
According to Rolling Stone, Grok was generating approximately one non-consensual sexualized image per minute, and analysis of 20,000 images found 2% appeared to depict minors in sexualized content.
Let me put some numbers to this.
According to Rolling Stone, Grok was generating approximately one non-consensual sexualized image per minute. Users could simply ask Grok to “digitally undress” photos of real people, often placing them in suggestive poses.
The posts appeared directly on X (formerly Twitter), visible to anyone scrolling their feed.
When reports emerged that some of these images appeared to depict minors, the situation escalated from “concerning” to “international incident.”
How xAI Responded (And Why It’s Not Enough)
xAI’s only fix was to restrict Grok’s image generation to X Premium paying subscribers, which the Internet Watch Foundation called inadequate since it simply paywalls dangerous capabilities.
xAI’s big fix was to restrict Grok’s image generation to paying subscribers only. That’s it. They didn’t remove the capability. They put it behind an X Premium paywall.
Think about what message that sends. Pay us money and you can keep creating harmful content.
⚠️ Red Flag: Three members of xAI’s safety team recently left the company, including the head of product safety. When your safety team is walking out the door, that tells you something about internal priorities.
The Internet Watch Foundation called the paywall response inadequate, stating it “does not undo the harm which has been done.”
Musk’s Response: “An Excuse for Censorship”
Elon Musk dismissed global criticism of Grok’s deepfake crisis as “an excuse for censorship,” while the EU Commission responded that creating non-consensual intimate images “is not spicy. This is illegal.”
When faced with global criticism about his AI generating fake pornographic material, Elon Musk dismissed it as “an excuse for censorship.”
This is the part that worries me most.
Creating a fake, sexualized image of someone without their consent isn’t a free speech issue. It’s a violation. It destroys reputations, causes psychological trauma, and gets used for blackmail and harassment.
A spokesperson for the EU Commission put it bluntly. “This is not spicy. This is illegal.”
ℹ️ Context: This isn’t Grok’s first controversy. In July 2025, xAI had to disable Grok’s text replies after the chatbot praised Adolf Hitler and made anti-Semitic remarks. There’s a pattern here.
Which Countries Are Taking Action Against Grok
Indonesia blocked Grok first, and the UK (Ofcom investigation), EU (Digital Services Act probe), India (content generation order), and US senators (calling for App Store removal) are all following.
Indonesia won’t be alone for long.
- United Kingdom. Prime Minister Keir Starmer said “all options are on the table” including a potential ban. Ofcom has made “urgent contact” with X and xAI.
- European Union. The Commission ordered X to retain all internal documents about Grok until end of 2026, signaling a formal investigation under the Digital Services Act.
- India. The IT ministry ordered xAI to prevent obscene content generation.
- United States. Senators Ron Wyden, Edward Markey, and Ben Ray Lujan urged Apple and Google to remove the X and Grok apps from their app stores entirely.
- France, Malaysia, Brazil. All scrutinizing the platform or calling for investigations.
Why This Matters for Everyday AI Users
The safety standards set with Grok will influence how every AI company builds image generation tools going forward, and the contrast with ChatGPT, Claude, and Gemini shows responsible AI image tools are possible.
You might be thinking. “I don’t use Grok. Why should I care?”
Because the standards we set now define the future of AI for everyone.
The easy, automated creation of deepfake material threatens all of us. It’s a tool that can be weaponized against anyone. You, your friends, your family. The precedent that gets set with Grok will influence how every AI company approaches safety going forward.
Compare this to how other AI companies handle image generation. ChatGPT (DALL-E 3), Claude, and Gemini (Imagen 3) all have strict guardrails preventing this kind of content. It’s not impossible to build powerful AI tools responsibly. xAI just chose not to.
Indonesia Bans Grok AI: FAQ
Why did Indonesia ban Grok AI?
Indonesia blocked Grok because the AI was being used to create non-consensual sexualized deepfakes, including images appearing to depict minors. The government called it a “serious violation of human rights.”
Is Grok banned in other countries?
Not yet, but the UK, EU, India, and US lawmakers are all taking action. UK Prime Minister Keir Starmer has said a ban is “on the table,” and US Senators Ron Wyden and Edward Markey have called for Grok to be removed from Apple’s App Store and Google Play.
What did xAI do to fix the problem?
xAI restricted image generation to X Premium paying subscribers only. Critics, including the Internet Watch Foundation, say this doesn’t address the harm already caused and simply puts dangerous capabilities behind a paywall.
Does this affect other AI chatbots like ChatGPT or Claude?
No. This is specific to Grok. OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini have stricter content policies that prevent this type of image generation. The issue is xAI’s approach to safety, not AI technology in general.
The Bottom Line
Indonesia just drew a line. A government looked at an AI tool, saw it was being used primarily to harm people, and said “no.”
More countries will follow. The question is whether xAI will actually fix the problem or keep dismissing legitimate safety concerns as “censorship.”
For those of us who use AI every day, this is a reminder that not all AI companies are created equal. The tools we choose to use, and the companies we support, matter.
If you’re new to AI and want to explore tools that take safety seriously, check out our Start Here page for beginner-friendly recommendations. You can also follow the latest AI news as this story develops.
Leave a Reply