Indonesia and Malaysia temporarily blocked X's chatbot, Grok, over the weekend after it made scores of fake images publicly sexualizing mostly women and, in some instances, children late last year.
Governments around the world are also launching investigations. The latest came on Monday as the UK media regulator, Ofcom, launched a probe into the social media platform, which could result in a ban.
Grok had been generating sexually explicit images of people for some time. But the issue got widespread attention in late December as people used the chatbot to edit a high volume of existing images by tagging the bot in comments and giving it prompts such as "put her in a bikini." While Grok did not respond to all of the requests, it obliged in many cases. In some cases, Bellingcat senior investigator and researcher Kolina Koltai noted, users can get Grok to generate frontal nudes.
Untold numbers of women and in some cases, children, as Reuters first reported, have had their likenesses sexualized online by Grok without their permission, including one of the mothers of X owner Elon Musk's children.
It's unusual for so many governments to take action against a social media company but this case is different, said Riana Pfefferkorn, a policy fellow at Stanford University. "Making child sexual abuse [material] is flagrantly illegal, pretty much everywhere on Earth."
By last Friday, X had restricted Grok's AI image generation feature to make it only available to paying subscribers. Non-paying users can still put people in bikinis publicly with just a few clicks, but they can only put in a few such requests before being prompted to sign up for a premium membership, which costs $8 a month.
NPR reviewed Grok's publicly available images generated earlier this month and found it had stopped making images of scantily clad women several days into 2026. However, it sometimes still offers up bikini-clad men.
xAI, the parent company of X, has been pushing adult content with Grok since last year. In May, Koltai first noted that the chatbot would generate sexually explicit images in response to requests on X like "take off her clothes." This past summer, Grok introduced "spicy mode" in its standalone app, which allowed users to put bikinis on AI-generated characters.
Ben Winters, director of AI and privacy at the advocacy organization Consumer Federation of America, said that Grok now not only allows editing images of real people, but it also provides an easy distribution platform via X. "It's a further and significant escalation," he said.
Governments are outraged over X's move to restrict access to the image generation function to subscribers. British Technology Secretary Liz Kendall told Sky News "it is insulting to say that you can still access this service if you pay for it."
The Indonesian government found that Grok lacked effective guardrails to stop users from making nonconsensual pornographic content based on real Indonesian residents, the Associated Press first reported.
"The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement.
The AP also reported Grok will stay blocked in Malaysia until effective guardrails against misuse are put in place.
In response to NPR's questions, X spokesperson Victoria Gillespie pointed to a statement posted on January 3 that said "anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content." The statement echoed Musk's own post earlier that day.
Such an approach is an "attempt to abdicate responsibility, " said Winters.
"It certainly is not just the user that is prompting it alone," he said. "It is the fact that the image would not be created if not for … the tool they made."
Before the wave of Grok-generated nonconsensual explicit deepfakes hit X, other AI makers had added similar capabilities to their chatbots. In November, Google released a new image generating model, Nano Banana Pro. In December, OpenAI updated its model ChatGPT Images. Both can also edit images to put people in bikinis, Wired reported last month. A thread on Reddit distributing such images has been taken down.
In another X post, Musk suggested that since tools from other AI companies also have this undressing function, the pressure from governments is a form of censorship directed at his platform.
Koltai noted the trend to make AI-generated nonconsensual intimate media has been in the making for the past year or two. "You'll see these ads in Instagram even – it's like, upload a photo of you and your crush and you guys can kiss," she said. "So that's kissing and hugging, and then you see there's that more extreme spectrum."
"There is obviously a huge issue within the tech industry because we've seen this across multiple platforms, and [there is] not always great guidelines or boundaries or regulation," said Koltai.
The criticism in the U.S. has been far more muted than in other countries. Sen. Ted Cruz, R-Tex., posted on X Wednesday that the images "should be taken down and guardrails should be put in place." He also said that he is "encouraged that X has announced that they're taking these violations seriously and working to remove any unlawful images and offending users from their platform."
Grok has generated frequent controversy in the past year. Last summer, the chatbot referred to itself as "MechaHitler" and spewed antisemitic conspiracy theories.
Apps with so-called nudifying capabilities have existed for years, mostly in the shadows of the internet. Winters said officials need to do more to police some of X's features. "There are misrepresentations about the safety of their products. There are violations of their terms of service," he said.
"We haven't seen really any significant action from any U.S. agencies, whether it's state or federal, that have the authority to enforce the law," Winters said.
Copyright 2026 NPR
