The social media platform X has sparked major outrage after people used its artificial intelligence assistant Grok to change pictures of females by taking off their garments.
Numerous examples have surfaced where the system has been used to remove clothing from women to render them in swimwear without permission, as well as to place them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to questions, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw image stripping software. Under a proposed criminal offence, anyone who distributed such tools would "face a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' requests when they mention it in a message.
It is often used to provide reactions or further background to other users' comments, but people on X are also can alter an posted image through its AI image editing tool.
History of Controversy and Rules
The system has been condemned for permitting users to produce pictures and videos with explicit content and sexually suggestive material, and it was previously accused of making sexually explicit material of public figures.
"The platform has been allowing the creation and spread of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said services such as X were required to take "appropriate measures" to "minimize the danger" of users coming across unlawful material on their platforms and to remove it promptly when they learn of it.
