The online network X has caused widespread backlash after users utilized its artificial intelligence assistant Grok to change images of women by taking off their garments.
Several cases have emerged where the tool has been employed to virtually strip women to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Stance
XAI, the firm behind Grok, did not respond to inquiries, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to prohibit nudification software. Under a upcoming law, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their services, but did not specify whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' prompts when they reference it in a tweet.
It is frequently used to provide reactions or further context to other users' remarks, but people on X are also able to alter an posted photo through its AI image editing function.
History of Criticism and Policy
The tool has been criticised for allowing users to create images and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been permitting the creation and sharing of these images for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the risk" of users encountering unlawful content on their services and to take down it promptly when they are notified of it.
