🔗 Share this article

Elon Musk's social media network X has caused widespread outrage after individuals utilized its AI chatbot Grok to change photos of women by taking off their clothing.

Numerous cases have been seen where the technology has been used to digitally undress women to make them appear in swimwear non-consensually, as well as to place them in inappropriate contexts.

Company Response and Regulatory Stance

XAI, the firm that created Grok, did not respond to requests for comment, except for an standard response stating "traditional news outlets lies".

A government representative stated that legislation is being introduced to ban nudification tools. Under a proposed law, anyone who distributed such tools would "be subject to a jail term and heavy fines".

"X or Grok has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."

Oversight Body Position and Platform Capabilities

The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.

Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they mention it in a message.

It is often used to offer reactions or further context to other posters' comments, but people on X are also can edit an posted image through its AI image editing tool.

History of Controversy and Policy

The system has been criticised for allowing users to produce pictures and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in making sexually explicit content of public figures.

"The platform has been enabling the creation and distribution of these images for a long time without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own terms of service prohibits "depicting images of persons in a pornographic fashion".

In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or illegal content" and stated this covered sexual deepfakes generated with AI.

It said services such as X were obligated to take "suitable measures" to "reduce the risk" of users encountering unlawful material on their platforms and to take down it swiftly when they are notified of it.