Are Grok’s Fake Nudes Breaking Canadian Laws? A Tyee Explainer

The flood of artificial-intelligence-created sexualized and nearly naked images of women and children on X, formerly known as Twitter, over the last month is an attempt to intimidate them into staying off the internet, says law professor Suzie Dunn.

“People are saying — and I think it’s well-intentioned, but terrible messaging — ‘Girls, women, don’t put photos of yourself on the internet,’” said Dunn, who is an assistant professor of law and interim director of the Law and Technology Institute at Dalhousie University.

“The idea that we can’t safely put everyday clothed photos of ourselves on the internet — without being at risk of being sexualized without our consent — really adds to this experience that women and girls already have where they don’t feel safe on the internet because of sexism and misogyny.”

The images are manipulated to produce a convincingly real depiction of someone, which is known as a “deepfake,” and shared by users of X’s Grok generative AI app. Users have prompted Grok to alter existing or create new images of both real and fake women, youth and children, put them in sexualized positions, remove most of their clothing and even put blood, bruises and semen-like fluids on their bodies.

While Grok, like many other generative AI programs, has safeguards to prevent the creation of fully nude images of real people or child sexual abuse material images, users have created workarounds like requesting clear tape over a person’s genitals and nipples, or that they wear fishing-wire bikinis, for example.

Elon Musk, X’s owner and the world’s richest man, initially responded to the phenomenon by making jokes.

What followed was weeks of the proliferation of what could be considered child sexual abuse materials under Canadian law, and non-consensual intimate image sharing of adults under B.C. law.

“[This] shows that there’s a real lack of care for women and children on that platform, and shows that there’s a real need for more consistent regulation that holds the platforms and the AI companies to account, not just the users,” Dunn said.

X has since restricted access to Grok image creation to its paid subscribers, and stopped the sexualized manipulation of real people’s images, but that has not stopped the criticism.

Emily Laidlaw, the Canada Research Chair in cybersecurity law and an associate law professor at the University of Calgary, noted that even images of fake people could affect real individuals.

“There is fan fiction pornography out there of, say, Buffy the Vampire Slayer. It looks like Sarah Michelle Gellar,” she said. “We’re in this grey area now.”

The federal government has said a legal response is coming in the form of Bill C-16.

But Laidlaw told The Tyee that, unlike B.C.’s own Intimate Images Protection Act, the proposed federal legislation would not cover deepfakes or nearly nude images of adults. With some tweaks, the former Bill C-63, or Online Harms Act, would still be a great start for holding companies to account and protecting consumers in Canada.

“What we want is consumer protection legislation that holds platforms accountable to have to manage the risks of harm of their services,” she said.

“And I think a very clear and obvious safety measure would be to not permit the use of this AI to create intimate images of others, whether it’s criminal or not.”

As well, the Online Harms Act included implementing a “pause” to allow a government regulatory body to assess whether reported content violates the law or not, and to reinstate materials that are incorrectly flagged for removal.

The Tyee interviewed Dunn and Laidlaw to get answers to our lingering questions about this issue: What do current and proposed federal and provincial laws say about the creation and sharing of fake, non-consensual sexualized images of adults and kids online? What is the recourse for Canadians whose images are being altered to sexualize or expose them? And is X, the U.S.-based social media platform that has done little to stop the creation and sharing of these images, breaking existing Canadian laws?

Fake images, real harm

The images may be fake, but the harms caused by creating and sharing them without consent are very real for the people being victimized, Dunn said.

“Research has shown that the psychological impact... can be very similar to what we see when people’s actual intimate images are shared,” she said.

“One of the challenges that comes up with this type of harm is that it’s ongoing. Every time someone views this content, there’s another experience of harm.”

With Grok in particular, images can quickly be created with just a few keystrokes, and the images are immediately publicly available on X, increasing the likelihood that more people will see and download the non-consensual sexualized images.

“[Victims] often have to pay to get those images removed,” Dunn said, including hiring people to search for the image online and demand their removal from the platforms hosting them.

Dunn has seen a few of the Grok-created sexualized images of adults and said that, unlike with AI-generated photos from the recent past, it is hard to tell that these images are fake.

“There is still a touch of glossiness to it,” she said, adding some of the prompts, such as requesting a person be depicted wearing a bikini made of dental floss, affect the appearance of authenticity.

“But overall, on a quick glance, it looks very realistic.”

Under existing Canadian law, creating and sharing fake images sexualizing children — even if the children depicted are fake — is illegal and harmful, said Dunn.

“We did see people charged [and convicted] with creating child sexual abuse material using things like Photoshop. So this type of harm has been recognized for a long time,” she said, adding that even searching for sexualized images of children and teenagers on Grok may be illegal.

“That could potentially cross into accessing and possessing child sexual abuse material.”

Images of children don’t even need to be fully nude for them to be considered child sexual abuse material under federal law.

But when it comes to the non-consensual creation and distribution of fake semi- or nearly nude photos of adults, what the law covers depends on which province you live in.

British Columbia’s Intimate Images Protection Act is one of the most comprehensive civil statutes protecting people from the non-consensual distribution of their intimate images, Dunn said, because it covers nearly nude, as well as fake and altered, images.

But Canada’s existing federal legislation and statutes don’t protect adults in this way. The closest we have come, both Dunn and Laidlaw say, was the Online Harms Act. But that proposed legislation failed to become law before Parliament was prorogued for the spring 2025 election.

What about the new federal legislation?

Evan Solomon, federal minister of artificial intelligence and digital innovation, recently announced Bill C-16, described as “an Act to amend certain Acts in relation to criminal and correctional matters (child protection, gender-based violence, delays and other measures).” The legislation would cover non-consensual deepfake images of adults like those created by Grok.

But Dunn said criminalizing deepfakes fills only a “small gap” in the vast gulf between what victims need and what Canadian law currently provides.

“It basically is including deepfaked images onto the definition of intimate images as currently defined, which includes images where a person’s sexual organs are exposed or they’re engaged in sexually explicit activity,” she said.

That’s not what is happening in the Grok images, which again are not fully but only partially or nearly naked images of people in sexually suggestive poses, not engaging in sexually explicit acts.

“I don’t know that this [legislation] would be interpreted to actually include this situation,” Dunn said.

Because the number of potential offenders sharing a single image could number in the hundreds, Dunn said, there could also be logistical barriers to enforcing the law. And both criminal and civil proceedings can take a long time, she added.

That is why the proposed Online Harms Act was more on the mark, she said, because it put the responsibility on the websites hosting these images to have them removed within 24 to 48 hours of receiving a complaint.

Dunn anticipates any law will face constitutional challenges regarding people’s right under freedom of expression to create deepfake images, because people have already tried that defence regarding child sexual abuse material.

“The courts have recognized that creating child sexual abuse material is a form of expression,” Dunn said.

“But in balancing the rights of children and the need to protect vulnerable populations with the freedom of expression rights of the person creating this content, the balance supports protecting children.”

What do victims need?

The Tyee reached out to both the BC RCMP and the Vancouver Police Department to ask whether they have received reports of non-consensual image creation and sharing on X. The Tyee also asked how the volume of these complaints has changed over the past few months, if at all. And we asked what a person victimized by these non-consensual images should do to get help.

The Tyee also asked if their department was still posting information related to crime and public safety on X and, if yes, why.

The Vancouver Police Department did not acknowledge or respond to our request or followup email.

The BC RCMP sent our request to national headquarters, whose spokesperson Robin Percival provided answers nearly five days after our deadline.

Percival said RCMP have received complaints about suspected child sexual abuse and exploitation material and non-consensual image sharing on X, but couldn’t say more as the investigation is ongoing.

“Users are encouraged to report content containing depictions of those under the age of 18, real or not, to the service provider, the police and/or Cybertip.ca, Canada’s cybertip line for online child sexual abuse and exploitation,” Percival wrote in an email to The Tyee, adding Cybertip.ca processed close to 4,000 deepfake images and videos of kids and youth in the past year.

Fake images are used to extort young people into sending money or providing their own sexual images or videos, she wrote. As well, Pervical said research has shown viewing child sexual assault materials can lead to real-life assaults.

As the technology improves, she wrote, it is becoming more difficult to tell real children from fake images and videos.

X is a signatory to Canada’s Voluntary Principles to Counter Online Child Sexual Exploitation and Abuse, she added, asking social media providers to “proactively detect, disrupt, and report” child sexual abuse matierals.

The RCMP continues to post on its X account “to provide timely information to Canadians on matters of public safety,” Percival said, adding its posts are not an endorsement of X’s platform or decisions.*

More protections are needed for adult victims

While there is already an established criminal law protecting children and youth under 18 whose images are altered to be sexualized and/or nearly nude and shared online, Dunn wants to see more protections for adult victims.

When it comes to images of adults, Dunn said chronic offenders should face civil or criminal processes that hold them to account.

But, she added, research has found that what victims want more than the law getting involved is to have their images taken off the internet as quickly as possible.

This is where the process established under B.C.’s Intimate Images Protection Act is effective, Dunn said, because they have both a victim support line and a BC Civil Resolution Tribunal web form for claiming damages under $5,000 and receiving a protection order.

“That has a fast-track option where, if it’s deemed that your image does fit within the statutes, you can get an order to get the content taken down off the website and deleted by the person who created it,” Dunn said. She added that victims don’t need to hire a lawyer to file a claim.

While Dunn wants governments to recognize deepfake sexualized photos as causing serious harm, she is more cautious about a criminal law approach to punish image creators, noting a recent Ontario court ruling did not find the sharing of digitally altered images to be a crime.

“Looking at the history of sexual violence in Canada, criminal responses haven’t necessarily reduced sexual violence in Canada,” Dunn said. “So I don’t know that criminalizing deepfakes is actually going to reduce the proliferation of deepfakes.”

Instead she wants to see governments hold online platforms like X to account for creating and sharing non-consensual sexualized deepfake images of adults and kids in Canada.

Consequences for social media companies

Unfortunately, under existing Canadian law, there is very little recourse for holding online platforms to account, said Laidlaw.

“It's the whole reason why online harms legislation was being proposed,” she said.

The only current avenue for accountability, Laidlaw added, is the one Canada’s privacy commissioner, Philippe Dufresne, announced last week: he expanded his ongoing investigation into X Corp.’s use of private data to train its AI model. Dufresne’s investigation will now also cover the Grok deepfakes.

But the privacy law angle is a narrow one, she added, focusing on whether users’ private data has been illegally accessed by X. And at the moment, it is not clear whether manipulating a photo that was consensually posted online is considered a privacy violation, Laidlaw said.

“That’s the big question right now. There was a big investigation of Clearview AI, and the privacy commissioners held that scraping of public data to create a database for using biometrics to basically allow law enforcement to check if some image matched that database... was a privacy violation,” she said.

That finding is currently being tested in court. While the B.C. Supreme Court ruled Clearview AI’s facial recognition technology did violate provincial privacy legislation, Alberta’s court ruled Clearview AI’s data scraping was protected by the right to free expression.

“I think [the Alberta court ruling] is a problematic interpretation, because it brings up exactly the issue we’re in now, which is the idea that if these companies are using this AI and they’re scraping this data, are we now giving them, basically, super protection around that activity?” Laidlaw said.

“It’s an unnecessarily broad interpretation of the right to freedom of expression.”

During a U.S. lawsuit against Character.ai, the company argued it was the public’s First Amendment right to receive information from a chatbot, before it settled with the family of a 13-year-old boy who died by suicide after talking to the chatbot.

“We have a looming challenge going on in this entire space,” Laidlaw said.

The original Online Harms Act did not mention chatbots, however, and a reintroduced version of the bill would need to be updated to include them, Laidlaw added.

Unlike Canada, the United Kingdom has already passed its own Online Safety Act to protect children and underage youth, in particular, from the potentially harmful aspects of online access to adult content like pornography or eating-disorder content. Last week the U.K. government announced more legislation would be coming to address Grok’s sexualized deepfakes in particular.

However, the United Kingdom’s current legislation has been criticized for blocking more than children’s access to pornography and self-harm content. Critics have warned that everyone lacking ID or unwilling to share it could lose access to news about ongoing wars and genocides, music streaming and social media sites.

“One of the hot questions is going to be age assurance mechanisms. There wasn’t anything specific in [the Online Harms Act] — they kind of said there should be a special duty to kids and that includes age assurance. But that was it; they left it to be dealt with by a regulator,” Laidlaw said.

“And that’s smart, because a regulator can create something more nuanced.”

Canada is also currently considering its own age verification legislation, Bill S-209, described as “An Act to restrict young persons’ online access to pornographic material.” That bill is facing its own criticism for potential overreach.

“The big question is whether it scopes in [X] or Google search, stuff like that,” Laidlaw said.

“There’s a reason why often these laws narrowly target nudity and don’t scope in more sexualized but harmful stuff, where someone might be creating bikini or underwear images of a person. Because there’s all kinds of contexts where we do share those images and it's perfectly fine.”

This would put the onus on social media companies like X to return to moderating posts, which they and other platforms like Meta have moved away from in recent years, Laidlaw added.

“[X] gutted their trust and safety team, and they would need to bring that back for Canada,” she said. Otherwise, Laidlaw warned, Canada will be “totally reliant on the whims of corporations when it comes to their commitment to trust and safety.”

* Story updated on Jan. 22 at 6:17 p.m. to include a response from the RCMP.