The online network X has caused widespread backlash after people employed its AI chatbot Grok to alter images of women by stripping their clothing.
Numerous cases have been seen where the technology has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to ban image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' prompts when they mention it in a message.
It is commonly used to offer commentary or more context to other posters' comments, but people on X are also capable of edit an uploaded picture through its AI image editing tool.
Background of Controversy and Policy
The tool has been faulted for enabling users to produce photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been enabling the generation and spread of these pictures for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and confirmed this applied to explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate steps" to "minimize the danger" of users coming across illegal material on their services and to take down it promptly when they learn of it.
