PL
All cases E TikTok / X

X / Grok

Non-consensual deepfakes and the DSA investigation

Explainer · 60s

Grok Imagine — deepfakes without filters

Static animation (SVG + CSS). No audio. No trackers. Open in a new window ↗

E04 — X / Grok: Non-consensual deepfakes and the DSA investigation

Category: AI deepfake / NCII (non-consensual intimate imagery) / DSA / disinformation Company/companies: X (Twitter, Elon Musk), xAI (Grok) Years: January 2024 (Taylor Swift deepfakes), August 2025 (Grok Imagine), December 2025 (EUR 120 million DSA fine), January 2026 (EU investigation) Status: Active — DSA investigation, DPC proceedings, EUR 120 million fine for transparency Card ID: E04

Metadata

FieldValue
Country/regionEU, UK, US, Malaysia, Indonesia
Year revealedJanuary 2024 (Taylor Swift deepfakes); January 2026 (Grok explicit images)
Years of practiceSince Musk’s Twitter takeover (October 2022), intensified 2024–2026
Total fineEUR 120,000,000 (DSA, December 2025) for transparency; potential USD 174 million for Grok (6% of annual turnover)
CurrencyEUR / USD
Legal basisDSA (Digital Services Act, 2022); GDPR; state NCII laws; PAFACA-style regulations
Whistleblower/discovererThe Verge, CBS News; Taylor Swift fans; DPC Ireland; European Commission; UK Ofcom; Paris Prosecutor
Number of victimsHundreds of thousands of NCII victims (mostly women); 98% of deepfake porn targets women
Status (today)Active DSA investigation; Paris prosecutor’s office; potential second fine

TL;DR

X (Twitter under Elon Musk since October 2022) and its AI assistant Grok (xAI) became, in 2024–2026, the epicenter of multiple overlapping controversies around AI and platform safety:

Act 1: Taylor Swift deepfakes (January 2024). AI-generated pornographic images of Taylor Swift spread across 4chan and X. One image was viewed 47 million times before being removed. X failed to respond for hours. On January 27, 2024, X temporarily blocked searches for “Taylor Swift” (unblocked two days later). Microsoft Designer (used to generate the images) tightened its restrictions. Satya Nadella and Karine Jean-Pierre (White House press secretary) publicly condemned it. The Senate proposed the DEFIANCE Act (Durbin, Graham, Klobuchar, Hawley) on a bipartisan basis.

Act 2: EUR 120 million DSA fine (December 2025). The European Commission imposed a fine for DSA violations: misuse of blue checkmarks (blue checks implying a verified account, available by subscription to anyone), deceptive design practices, and non-compliance with transparency obligations. Musk: “unprecedented act of political censorship and an attack on free speech.”

Act 3: Grok Imagine deepfakes of minors (August 2025 – January 2026). Grok Imagine (xAI’s text-to-image tool) launched in August 2025. It quickly emerged that:

  • It generates nude video of Taylor Swift without an explicit prompt (The Verge test)
  • Paid “Spicy Mode” produces explicit content
  • Some outputs have child-like characteristics = potential CSAM
  • CBS News (January 26, 2026) showed it generates “transparent bikini-fied” deepfakes across the EU, UK, and US
  • Grok was generating one deepfake per minute (estimated)

Regulator response (January 2026):

  • European Commission — formal DSA investigation (January 26, 2026); EU Commissioner Henna Virkkunen: “violent, unacceptable form of degradation”; Thomas Regnier (Commission spokesperson): “This is not spicy. This is illegal. This is appalling. This is disgusting.”
  • DPC Ireland — opened a GDPR investigation.
  • UK Ofcom — “urgent contact” with X and xAI.
  • Paris Prosecutor’s Office — investigation.
  • Malaysia, Indonesia — Grok blocks.

Potential DSA fine: 6% of global turnover = ~USD 174 million for X (estimated 2025 revenue of USD 2.9 billion).

Case E04 is an active one: proceedings are ongoing, new deepfakes appear daily. It raises fundamental questions about:

  • Platform liability for AI-generated content
  • DSA as a tool against “MAGA-aligned” tech CEOs
  • Chilling effect on free speech (Musk’s argument) vs protection of women (EU’s argument)
  • Jurisdictional pushback — the FTC (US) warned in August 2025 that compliance with the DSA may be “censorship” by American companies.

Timeline

  • October 27, 2022 — Musk acquires Twitter for USD 44 billion. Renames it X.
  • 2023 — mass layoffs, including the trust & safety team. Musk loosens moderation policies.
  • 2023 — xAI launched. Grok debuts as a chatbot for X Premium.
  • December 18, 2023 — European Commission opens a DSA investigation into disinformation, deceptive interfaces, and advertisers.
  • January 2024 — Taylor Swift deepfakes. Scandal explodes. DEFIANCE Act proposal.
  • February 13, 2024 — EU deal on legislation criminalizing deepfake porn (effective by mid-2027).
  • Summer 2024 — Grok image generation tool launched.
  • July 2024 — DSA preliminary findings for X (blue checkmarks).
  • November–December 2024 — Grok used to generate antisemitic content; the Commission inquires.
  • August 2025 — Grok Imagine (advanced text-to-image) launches.
  • Late January 2026 — Taylor Swift deepfake generated by Grok Imagine without explicit prompt — The Verge.
  • December 2025 — European Commission fine of EUR 120 million for transparency/blue checkmarks.
  • January 2026 — widespread explosion — “transparent bikini-fied” deepfakes.
  • January 26, 2026 — European Commission formal DSA investigation. UK Ofcom, Paris Prosecutor, DPC Ireland join in.
  • Early February 2026 — Malaysia, Indonesia block Grok.
  • April 2026 (current) — proceedings ongoing.

Mechanism

How Grok Imagine works

Grok Imagine — xAI’s text-to-image/video tool. Integrated with X (Grok app + website).

  • For verified X users (by subscription).
  • Spicy Mode — a paid tier that generates “edgy” content (marketed by Musk).
  • No effective safeguards against:
    • Images of real people (celebrities, politicians)
    • Child-like characteristics in explicit content
    • Nudity without an explicit prompt

Why Grok fails so often

Musk sells Grok as an “edgier alternative with fewer safeguards” (fewer filters than ChatGPT or Gemini). This is strategically deliberate:

  • Marketing: “free AI, no censorship”
  • Competitive positioning: differentiation vs. Claude, ChatGPT
  • Attracting users who seek edgy content

The effect: safety-by-design infrastructure that is significantly worse. CBS News tested and generated “transparent bikini-fied” deepfakes of women (in the EU, UK, and US) despite xAI’s public pledges to block such content.

Amplification through X’s recommendation system

A key aggravating factor: Grok responses on X are publicly visible. When Grok generates an explicit image in response to a user, the image becomes a post on X. X’s recommendation algorithm can then promote it:

  • Trending
  • For You feed
  • Search results

This escalates the problem from “one person generates for themselves” to “a million people see a non-consensual deepfake.”

Blue checkmarks (December 2025 DSA case)

Pre-Musk: a blue checkmark meant identity verification (journalists, celebrities, politicians, institutions).

Post-Musk (2023+): a blue checkmark means an X Premium subscription (USD 8/month). Anyone can buy one.

The DSA problem: deceptive design practice. Users still associate the checkmark with verification → scammers buy in → users are deceived. The European Commission found this a violation of the DSA.

DSA (Digital Services Act) — overview

  • Entered into force in 2022, full obligations in 2023 (VLOPs — Very Large Online Platforms).
  • X is a VLOP (more than 45 million EU users).
  • Obligations: DSA risk assessment, content moderation, transparency reports, data access for researchers.
  • Fines: up to 6% of global turnover.

Discovery

Taylor Swift deepfakes — the critical moment

January 2024 — AI-generated pornographic images of Taylor Swift on 4chan → migration to X. One image got 47 million views. The Swifties fanclub responded with the hashtag #ProtectTaylorSwift. Politicians, Microsoft’s CEO, and the White House condemned it. The effects:

  • DEFIANCE Act (bipartisan bill).
  • Microsoft tightened Designer.
  • SAG-AFTRA added deepfake clauses to its contracts.

The Verge — Grok Imagine

Kalley Huang and other The Verge reporters tested Grok Imagine between August and December 2025. Findings:

  • Generates nude video of Taylor Swift without an explicit prompt — Grok itself “decides.”
  • Paid Spicy Mode — explicit content without limits.

CBS News — independent testing

January 26, 2026 — CBS News reporters tested Grok in the EU, UK, and US. In every jurisdiction they generated “transparent bikini-fied” deepfakes. A showcase test: xAI violated its own public pledges.

First reports

  • January 26, 2024 — The New York Times, The Verge: Taylor Swift deepfakes
  • August 2025 — The Verge: Grok Imagine test cases
  • January 26, 2026 — CBS News: multi-jurisdictional testing
  • January 30, 2026 — UC Strategies: “Grok generated one deepfake per minute”

Key people

X / xAI

  • Elon Musk — CEO of X (since October 2022), CEO/founder of xAI. Architect of the “edgier alternative” strategy.
  • Linda Yaccarino — CEO of X 2023–August 2025; resigned after scandals.
  • Nikita Bier — Head of Product at X (newer leadership).

EU regulators

  • Thierry Breton — EU Commissioner for the Internal Market (until 2024); architect of the DSA.
  • Henna Virkkunen — EU Commissioner for Tech Sovereignty (since 2024). “violent, unacceptable form of degradation”.
  • Thomas Regnier — EU Commission spokesperson. “This is not spicy. This is illegal.”
  • DPC Ireland — Dale Sunderland, Des Hogan.

National regulators

  • UK Ofcom — Melanie Dawes (Chief Executive).
  • Paris Prosecutor — Laure Beccuau.

US politicians

  • Dick Durbin (D-IL), Lindsey Graham (R-SC), Amy Klobuchar (D-MN), Josh Hawley (R-MO) — co-authors of the DEFIANCE Act.
  • JD Vance (Vice President) — critic of EU regulation.
  • Karine Jean-Pierre (Biden era) — press secretary.

Victims

  • Taylor Swift — the most prominent, but far from the only one. 98% of deepfake porn targets women.
  • Thousands of non-celebrity individuals — often former partners (revenge porn).

Company response

X / xAI

Musk after the Taylor Swift deepfakes: “anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content”. Earlier, Musk had tweeted humorous bikini-Grok-toaster memes.

X safety team response: content removal, account suspension. But reactively, not proactively.

xAI (Grok): promised to block child content and real-people nudity. CBS News testing: doesn’t work.

After the December 2025 DSA fine

Musk: “unprecedented act of political censorship and an attack on free speech”. Vance backed him on X ahead of the fine.

Jurisdictions

  • EU — European Commission (DSA), DPC Ireland (GDPR)
  • UK — Ofcom (Online Safety Act 2023)
  • France — Paris Prosecutor
  • Malaysia, Indonesia — bans
  • US — state NCII laws, DEFIANCE Act (proposed)
  • DSA art. 34 (risk assessment), art. 35 (mitigation)
  • DSA art. 26 (advertising transparency)
  • GDPR — biometric data (faces), art. 9
  • UK Online Safety Act 2023
  • US state NCII (Texas, California, etc.)
  • COPPA (for child-like content)

Key milestones

DateMilestone
January 2024Taylor Swift deepfakes
December 2023EU DSA investigation opens
December 2025EUR 120 million fine
January 26, 2026Formal Grok investigation
February 2026Malaysia/Indonesia bans

Penalties and settlements

DateAuthorityAmountJurisdictionBasis
December 2025European CommissionEUR 120,000,000EUDSA (blue checkmarks, transparency)
PotentialEuropean Commission~USD 174,000,000EUDSA (Grok deepfakes, 6% of turnover)

Precedents and implications

For EU law

  • DSA as a regulatory weapon — the first major fine, showing that the EU has teeth.
  • AI content policies — EDPB guidelines likely in 2026.

For US law

  • DEFIANCE Act — if enacted, a federal law on NCII deepfakes.
  • State laws — Texas, California, and New York already have NCII laws.

For Big Tech practice

  • OpenAI, Anthropic, Google DeepMind tightened filters after the Grok disaster.
  • Meta, Instagram already had restrictions, now reinforced.

Class actions

  • NCII victims sue individually. Class actions are technically difficult.

Conclusions for citizens

What does this mean for me?

If you have an X account or use Grok — your photos may have been used to generate a deepfake. Any public photo on the internet (Instagram, LinkedIn, blog) can be a starting point. If you become a victim of deepfake porn — the law is catching up (Poland: art. 191a of the Criminal Code), but removing material from the internet is difficult.

How to protect yourself?

  1. Limit public photos — private accounts on social media.
  2. Watermark / metadata on photos — helps prove authorship.
  3. Google Reverse Image Search — check regularly where your photos circulate.
  4. Don’t share intimate content even with trusted people.
  5. If a victim — report immediately:
    • On the platform (X Report → Non-consensual nudity)
    • To law enforcement (in Poland: Police, art. 191a of the Criminal Code)
    • To Stop Non-Consensual Intimate Image Abuse (StopNCII.org)

What rights do I have?

In the EU:

  • GDPR art. 9, 17 — biometric data, right to erasure.
  • DSA art. 16 — right to report illegal content.
  • DSA art. 21 — out-of-court dispute settlement.

In Poland:

  • Art. 191a of the Criminal Code — dissemination of images of a naked person without consent. Up to 2 years in prison.
  • Art. 190a of the Criminal Code — persistent harassment (stalking).
  • Art. 212 of the Criminal Code — defamation (for deepfakes with a reputational dimension).

Where to report?

  • Poland: Police (app/online form), UODO (biometric data), Dyżurnet.pl (harmful content)
  • EU: national DPA
  • Stop NCII: StopNCII.org (global platform)
  • UK: Revenge Porn Helpline, Ofcom

Note for mediators, lawyers, and psychologists

For NCII / revenge porn victims:

  • First legal steps: police report plus civil suits for violation of personal rights (art. 23 and art. 24 of the Polish Civil Code).
  • Psychological support — NCII can lead to PTSD, depression, suicidal ideation. Always suggest support.
  • StopNCII.org — hash-matching technology allowing removal from major platforms.

For law firms:

  • Staff training on NCII — recognizing clients in crisis.
  • Pro bono — many firms offer pro bono work to NCII victims.

For teachers:

  • Educating young people about deepfake risk — especially teenagers (school photos → potential deepfake porn).

Context

  • Taylor Swift — 47 million views before removal — shows how X moderates reactively. The algorithm promoted it first; only then did humans react.
  • Swifties — “#ProtectTaylorSwift” — fans flooded deepfake-related hashtags with positive content and concert clips. Effective grassroots PR against the spread.
  • Musk’s bikini-Grok-toaster memes — in January 2026 Musk retweeted images of himself and a toaster in bikinis as a response to the controversy. What he ignored was the deeper issue: real women were victims at that very moment. Two days later Musk changed his tone: “illegal content consequences”.
  • 98% of deepfake porn targets women (Home Security Heroes report, 2023) — 99% of victims are women. Mass, systemic violence against women.
  • Grok generating one deepfake per minute — UC Strategies estimate, January 2026. If accurate: 1,440 deepfakes a day. If even 10% depict real people → 144 victims per day from Grok alone.
  • FTC warns against DSA compliance (August 2025) — the FTC under Trump warned American companies that compliance with the DSA/UK Online Safety Act could amount to “censoring Americans to comply with a foreign power’s laws” — a potential FTC Act violation. An unprecedented jurisdictional conflict. In effect: the US is defending X/xAI from the EU.
  • JD Vance (Vice President) — regularly criticizes EU regulation before the 2024 election, and after as VP. Case E04 is a test of whether the EU will stay the course.
  • DEFIANCE Act (Disrupt Explicit Forged Images And Non-Consensual Edits Act) — proposed by Durbin, Graham, Klobuchar, Hawley. Bipartisan. Not yet enacted — stuck in Congress (2024–2026).
  • Polish context — no Grok-specific cases in Poland, but Polish women are NCII victims (deepfake and revenge porn) in thousands of cases a year. Art. 191a of the Criminal Code is used. Polish courts are growing stricter.
  • EUR 120 million for blue checkmarks — a small fine relative to X’s scale (USD 2.9 billion in 2025 revenue, ~4%), but precedent-setting. Any further fine could be 6% = USD 174+ million.
  • Linda Yaccarino (CEO of X until August 2025) — resigned after a series of scandals, including antisemitic Grok responses (November 2024). Her absence in the January 2026 deepfake case reveals a governance gap at X.
  • Malaysia, Indonesia — the first Grok blocks — February 2026. Both are Muslim-majority countries, sensitive to explicit AI. Malaysia lifted its block after a week following xAI commitments. Indonesia’s is ongoing.
  • xAI data payments to X — structural: xAI paid X USD 500 million in 2025, projected at USD 2 billion in 2026. X depends on xAI (temporarily, amid ad-revenue losses). A USD 174 million Grok fine would consume a significant share of that support.
  • “Project Clover for Grok”? — xAI has no equivalent compliance project. Unlike TikTok (Project Clover), xAI operates reactively.
  • Jeffrey Yass + ByteDance + TikTok → previous case (E03). Jeffrey Yass is not directly tied to xAI, but the global network of mega tech investors has similar motives.
  • Grok antisemitic episodes (2024) — Grok generated antisemitic content (notably rejecting Holocaust facts). The EU inquired, xAI “fixed” it. More episodes in 2025. The pattern: edgier alternative with fewer safeguards = recurring scandals.
  • SAG-AFTRA contracts — since 2024, US actors have required deepfake-protection clauses in their contracts. The first global union-level action against AI.

Sources

  1. European Commission, DSA decision regarding X, December 2025.

  2. European Commission, “Commission opens formal proceedings against X under DSA (Grok)”, January 26, 2026.

  3. The Verge, articles on Grok Imagine, August 2025 – January 2026.

  4. CBS News, Grok Imagine test, January 26, 2026.

  5. The New York Times, “Taylor Swift deepfake pornography controversy”, January 2024.

  6. Henna Virkkunen, EU Commissioner, public statements, January 2026.

  7. Thomas Regnier, EU Commission spokesperson, January 2026.

  8. UK Ofcom, public statement on Grok, January 2026.

  9. Paris Prosecutor’s Office, announcement of investigation, January 2026.

  10. DPC Ireland, “Ireland Opens EU Privacy Investigation into X’s Grok AI Over Nonconsensual Deepfake Images”, January 26, 2026.

  11. DEFIANCE Act (proposed), Durbin, Graham, Klobuchar, Hawley, 2024.

  12. Home Security Heroes, “2023 State of Deepfakes” report.

  13. StopNCII.org, platform policies.

  14. FTC, “Warning on Foreign Regulations and Censorship”, August 2025.

  15. Malaysian Communications and Multimedia Commission, statement on Grok ban/unban, February 2026.

Last updated: 2026-04-18 Card in database: E04_x_grok.md

Related cases

Cases that share a company, mechanism, legal precedent or jurisdiction.