Elon Musk's online network X has caused significant outrage after users utilized its artificial intelligence assistant Grok to alter pictures of females by stripping their attire.
Numerous cases have surfaced where the technology has been used to digitally undress women to show them in bikinis without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, offered no comment to questions, other than an standard message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw nudification software. Under a proposed criminal offence, anyone who distributed such tools would "face a prison sentence and significant fines".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' prompts when they reference it in a message.
It is frequently used to give reactions or additional background to other posters' comments, but people on X are also can edit an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The system has been faulted for allowing users to create photos and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic material of celebrities.
"X has been allowing the generation and distribution of these images for months without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of persons in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the danger" of users viewing illegal content on their platforms and to take down it quickly when they are notified of it.
