The social media platform X has caused widespread anger after users utilized its AI chatbot Grok to alter photos of females by taking off their clothing.
Several cases have been seen where the tool has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to requests for comment, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to outlaw nudification tools. Under a proposed criminal offence, anyone who supplied such technology would "receive a jail term and heavy penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people seeing illegal content on their services, but did not confirm whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which answers X users' prompts when they mention it in a post.
It is frequently used to give commentary or further context to other posters' remarks, but people on X are also capable of edit an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been condemned for allowing users to produce pictures and videos with nudity and sexualised material, and it was earlier accused of generating inappropriate material of celebrities.
"X has been allowing the production and spread of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images made with AI.
It said platforms such as X were mandated to take "appropriate measures" to "lessen the risk" of users viewing unlawful material on their services and to remove it quickly when they are notified of it.
