California Launches Formal Action Against xAI

The California Attorney General’s Office has escalated its scrutiny of Elon Musk–backed AI startup xAI, following reports that its chatbot, Grok, was being used to generate nonconsensual sexual images involving women and minors. Earlier this week, state authorities confirmed that an investigation had been opened into the company’s practices.

By Friday, the matter intensified when the state issued a formal cease-and-desist letter, instructing xAI to take immediate steps to prevent the creation and circulation of nonconsensual intimate imagery and CSAM — child sexual abuse material.

---

Attorney General Demands Immediate Compliance

California Attorney General Rob Bonta underscored the seriousness of the allegations in a public statement accompanying the order.

“Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material,” said California AG Rob Bonta in a press release. “The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM].”

According to the Attorney General’s Office, xAI has been given five days to demonstrate that it is actively implementing safeguards to address the issue.

---

Claims of Large-Scale Abuse and Online Harassment

State officials allege that xAI may have been “facilitating the large-scale production” of nonconsensual nude images, which they say are being “used to harass women and girls across the internet.” The office maintains that the scope and scale of the activity necessitated swift regulatory intervention.

The controversy centers on Grok’s so-called “spicy” mode — a feature designed to allow the generation of explicit content. While xAI reportedly introduced new restrictions on its image-editing capabilities late Wednesday, California authorities proceeded with enforcement action regardless.

---

International Scrutiny and Platform Restrictions

The regulatory backlash has not been confined to the United States. Authorities in Japan, Canada, and Britain have opened investigations into Grok, while Malaysia and Indonesia have temporarily blocked access to the platform altogether. The widening international response reflects growing concern over the misuse of generative AI tools across jurisdictions.

---

xAI Response and Broader Industry Pressure

X’s safety account has previously condemned the misuse of Grok, stating:

“Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

When contacted for comment, xAI responded with an automated email reading, “Legacy Media Lies.” TechCrunch also reached out to the California Attorney General’s Office for further clarification on the enforcement action.

---

Generative AI Under Growing Legislative Focus

The situation highlights a broader challenge facing the tech industry as free generative AI tools become more widely accessible. Platforms beyond X have struggled to curb the rise of non-consensual sexual content created using AI.

This growing concern has drawn attention at the federal level. On Thursday, U.S. lawmakers sent a letter to executives at several major technology companies — including X, Reddit, Snap, TikTok, Alphabet, and Meta — seeking details on how they plan to limit the spread of sexualized deepfake content.