A newly filed federal class-action lawsuit accuses Elon Musk's xAI and Stability AI of failing to implement adequate safeguards against the creation of AI-generated child sexual abuse material, with research suggesting Grok may have generated approximately 23,000 sexualized images depicting children during an 11-day period.

The complaint, filed on behalf of five Jane Doe plaintiffs including three girls from Tennessee, alleges that real, clothed photographs of minors were used to generate fake nude images and videos using artificial intelligence. According to the lawsuit, the first plaintiff discovered explicit AI-generated images of herself after receiving an anonymous Instagram message alerting her to the material, which had allegedly been created by someone she knew and shared across public platforms.

What Safeguards Did Grok Allegedly Lack?

The lawsuit specifically targets what it describes as xAI's "Spicy Mode," alleging the company promoted more explicit AI image generation while failing to implement comparable protections against child sexual abuse material. According to the complaint, this feature was placed behind a paywall, meaning AI companies were essentially profiting from providing tools that could be used to create child pornography.

"This is something Elon Musk has called Spicy Mode. They put it behind a paywall, which is essentially these AI companies are profiting from providing the tools to allow people to make child pornography," said Mark Chalos, the Nashville managing partner at Lieff Cabraser Heimann & Bernstein, who represents the plaintiffs.

Mark Chalos, Nashville Managing Partner at Lieff Cabraser Heimann & Bernstein

The lawsuit seeks at least $150,000 per victim per alleged violation, along with additional damages and changes to how AI companies design and safeguard their products.

How Widespread Is the Problem With AI-Generated Child Abuse Material?

The concerns raised in the lawsuit extend far beyond a single case. Research cited in the complaint reveals the scale of the issue: the Center for Countering Digital Hate reviewed a random sample of 200,000 AI-generated images from approximately 4.6 million images Grok generated during an 11-day period, and based on that sample, researchers estimated about 23,000 sexualized images appeared to depict children.

The National Center for Missing and Exploited Children (NCMEC), a nonprofit organization that tracks child exploitation, reported alarming statistics on the broader problem. Last year alone, the organization received over 400,000 reports through its CyberTipline for AI-generated child sexual abuse material.

"Last year alone, we've received over 400,000 reports through our CyberTipline for this type of material," said Shelley Allwang, director of NCMEC's Child Victim Identification Program.

Shelley Allwang, Director of Child Victim Identification Program at National Center for Missing and Exploited Children

The attorney representing the plaintiffs emphasized the severity of the situation: "It's not a small problem. It's a problem we have seen unfortunately throughout the United States, and it's a problem that seems to be getting worse".

What Are AI Companies' Responses to These Allegations?

When contacted for comment, Stability AI stated that the company is "committed to preventing the misuse of AI technology, particularly in the creation and dissemination of harmful content, including CSAM," which it says is prohibited under its Acceptable Use Policy. The company claimed to have implemented "robust safeguards" since taking over development of the Stable Diffusion family of models in late 2022.

Stability AI also noted that it regularly works with U.S. and international law enforcement and partners with organizations including Thorn and the Tech Coalition to strengthen child safety protections. The company added that any suggestion that safety is not a top priority is "categorically wrong."

Steps Experts Say AI Companies Should Take to Prevent Misuse

• Content Filtering Systems: Implement robust detection mechanisms that identify and block attempts to generate child sexual abuse material before images are created or distributed on platforms.
• Age Verification Requirements: Require users to verify their age before accessing features that generate explicit content, particularly those behind paywalls like Grok's "Spicy Mode."
• Law Enforcement Collaboration: Establish regular partnerships with organizations like the National Center for Missing and Exploited Children and law enforcement agencies to report suspected abuse material and track emerging misuse patterns.
• Transparent Safety Documentation: Publish detailed information about safeguards in place and regularly audit systems to ensure protections are effective and up to date.

Court records indicate that the person accused of creating the images in the Tennessee case was arrested. The lawsuit was filed as Tennessee leaders met to discuss the future of artificial intelligence, highlighting the timing of growing concerns about AI safety and child protection.

The case represents a significant moment for the AI industry, as it directly challenges the business models and safety practices of major generative AI companies. The allegations suggest that profit-driven features designed to generate more explicit content may have come at the expense of protections that could prevent serious harm to minors.