The social media network X has provoked widespread outrage after people used its AI chatbot Grok to manipulate images of women by taking off their clothing.
Several cases have surfaced where the tool has been employed to digitally undress women to show them in swimwear without permission, as well as to put them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to inquiries, aside from an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to prohibit image stripping technology. Under a upcoming law, anyone who distributed such technology would "receive a jail term and heavy penalties".
"The platform is capable of halting these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said platforms must "consider the potential" of people accessing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' requests when they reference it in a post.
It is commonly used to provide reactions or additional background to other posters' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been condemned for enabling users to generate pictures and clips with nudity and sexualised content, and it was earlier alleged to have been involved in creating sexually explicit content of celebrities.
"The platform has been enabling the creation and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of people in a sexually explicit way".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and stated this included explicit AI images made with AI.
It said networks such as X were required to take "suitable actions" to "lessen the risk" of users coming across unlawful material on their platforms and to take down it swiftly when they are notified of it.
