Canada's privacy commissioner has determined that Elon Musk's Grok AI chatbot violated the country's federal privacy law by enabling the creation and spread of sexual deepfakes without proper safeguards. The investigation, launched in January 2026, found that both xAI (the company behind Grok) and X Corp. (the social media platform) failed to adequately protect users' personal information and consent rights.

What Happened With Grok's Image Generation Tool?

Grok's AI image generation feature was deployed without sufficient protections, allowing users to create and distribute sexually explicit deepfakes at an alarming scale. At its peak, the tool was generating well over 6,000 sexualized images per hour, with researchers finding that women and children were disproportionately targeted. The investigation examined whether the companies obtained valid consent to collect, use, and disclose personal information for creating these deepfakes, including explicit content.

"This lack of protection allowed users to create and share sexualized deepfakes, largely targeting women and children. According to researchers, Grok was at one point generating well over 6,000 sexualized images per hour," said Philippe Dufresne, Canada's privacy commissioner.

Philippe Dufresne, Privacy Commissioner of Canada

While xAI and X Corp. have taken some measures in response to the investigation, the privacy commissioner emphasized that their efforts remain insufficient. The companies have reduced the number of images being generated, but the deepfakes remain available on the platform, indicating incomplete compliance with Canadian privacy law.

Why Is Canada's Privacy Commissioner Frustrated With the Response?

The privacy commissioner recommended that the companies suspend the tool entirely until they could implement comprehensive safeguards. However, both xAI and X Corp. declined to do so, and the commissioner lacks the legal authority to force compliance. This limitation highlights a critical gap in Canada's current privacy framework.

"One of the recommendations that we made was that they suspend the tool until they can put in place all of the appropriate safeguards. So press pause, fix the issue given the harms, and that gives an incentive to fix the issues quickly. They've not accepted to do this and I can't force them to do it," explained Dufresne.

Philippe Dufresne, Privacy Commissioner of Canada

The commissioner pointed out that Canada's privacy law currently lacks enforcement teeth. There is no ability to impose financial penalties or binding orders, which creates little incentive for companies to comply with privacy protections. This regulatory weakness has become a focal point in the broader debate over how to govern artificial intelligence and emerging technologies.

What Steps Are Being Taken to Prevent Future Violations?

• Privacy Law Updates: Artificial Intelligence Minister Evan Solomon has committed to introducing updated privacy legislation that would grant the privacy commissioner power to issue fines and binding orders, though timing remains uncertain before the House of Commons summer recess.
• Online Safety Bill: The Liberal government has proposed an online safety bill that would regulate social media platforms and AI chatbots, requiring companies to act responsibly and blocking access for users under 16.
• Criminal Legislation: A separate crime bill would criminalize the creation and distribution of non-consensual sexual deepfakes, though it has not yet become law.

The privacy commissioner has welcomed these legislative efforts but stressed that criminal law alone cannot solve the problem. Criminal approaches are inherently reactive, meaning significant harm occurs before prosecution can begin. Preventive regulatory measures are essential to protect the public before violations happen.

This case is not isolated to Canada. The United Kingdom, the European Union, and California have all launched their own investigations into Grok's deepfake generation capabilities, signaling a global concern about the technology's potential for abuse.

"If you rely only on criminal law, it's going to be reactive, and you're going to have significant harm that we need to prevent," noted Dufresne.

Philippe Dufresne, Privacy Commissioner of Canada

The Grok investigation underscores a fundamental tension in AI governance: companies can deploy powerful generative tools rapidly, but regulatory frameworks struggle to keep pace. As artificial intelligence and other emerging technologies increasingly rely on mass collection and use of personal data, policymakers face mounting pressure to establish enforceable safeguards before harm occurs rather than after.