Elon Musk's online platform X has caused major anger after individuals utilized its artificial intelligence assistant Grok to change photos of women by taking off their garments.
Multiple cases have surfaced where the system has been employed to remove clothing from female subjects to show them in bikinis non-consensually, as well as to put them in sexualized situations.
Developer Response and Legal Position
XAI, the firm that created Grok, remained silent to questions, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification software. Under a new law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people viewing illegal content on their services, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some paid-for functions—which answers X users' prompts when they reference it in a message.
It is often used to provide commentary or additional background to other posters' comments, but people on X are also able to edit an posted picture through its AI image editing function.
Background of Controversy and Policy
The tool has been criticised for allowing users to generate images and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in creating sexually explicit material of public figures.
"X has been enabling the production and sharing of these images for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said services such as X were mandated to take "appropriate measures" to "minimize the risk" of users encountering unlawful content on their services and to take down it promptly when they are notified of it.
