California Launches Investigation Into xAI’s Grok Over Deepfake Porn

Of the 20,000 generated images analysed in a study, 2% depicted minors.

Enlarge text
Cover ImageCover image via The Guardian & Sinar Harian
Logo

Follow us on InstagramTikTok, and WhatsApp for the latest stories and breaking news.

California Attorney General Rob Bonta has launched a formal investigation into xAI, Elon Musk's artificial intelligence firm

This follows widespread reports that its chatbot, Grok, is being used to generate non-consensual sexually explicit imagery, according to ANI News.

The probe, announced on Wednesday, 14 January, focuses on allegations that the tool's "Spicy Mode" facilitated the harassment of women and minors by allowing users to virtually 'undress' real individuals using AI-generated deepfakes.

"The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," said Attorney General Bonta.

"We have zero tolerance for the AI-based creation and dissemination of child sexual abuse material or intimate images created without consent," he added.

SAYS.com

California Attorney General Rob Bonta.

Image via Cal Matters

The investigation follows a damning report by Paris-based non-profit AI Forensics, which analysed over 20,000 images generated by Grok

The study revealed that more than half of the outputs depicted individuals in "minimal attire", with the vast majority being women, according to The Guardian. Disturbingly, approximately 2% of these images appeared to depict minors.

Critics argue that xAI's marketing of "Spicy Mode" — a feature designed to be "edgy" and less restricted than competitors — has directly led to this surge in digital abuse. While the tool is integrated into the social media platform X, the resulting images have spread across the broader Internet, targeting both public figures and private citizens.

In an attempt to mitigate the backlash, xAI recently restricted Grok's image-generation features to paid subscribers only, a move intended to link usage to verified identities and payment details

Musk publicly stated that he was "not aware" of any naked underage images being generated, but warned that users creating illegal content would face the same legal consequences as those who upload it manually.

California's Department of Justice is currently assessing whether xAI's development and deployment of these features violated state laws, including newly enacted statutes regarding deepfake pornography.

SAYS.com
Image via Britannica

Follow SAYS Tech on FacebookInstagram, & TikTok for the latest in tech in Malaysia and the world!

Malaysia has taken action against Grok to ensure the safety of the public.
Read more trending stories on SAYS

You may be interested in: