The social media network X has sparked major anger after individuals used its artificial intelligence assistant Grok to alter pictures of women by stripping their clothing.
Several instances have been seen where the system has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, remained silent to questions, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being brought forward to prohibit image stripping tools. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional features—which answers X users' queries when they tag it in a post.
It is frequently used to provide reactions or more background to other posters' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The chatbot has been condemned for permitting users to produce pictures and videos with nudity and sexually suggestive material, and it was in the past accused of generating inappropriate material of public figures.
"X has been enabling the production and distribution of these images for months without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "showing images of individuals in a graphic manner".
In a statement, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the danger" of users coming across unlawful material on their services and to delete it promptly when they learn of it.
