What happened
It was reported in May 2025 that a user could reply to a photograph posted on X, ask the Grok account to remove the subject’s clothes, and get an altered image back in the same public thread. In August 2025 two outlets reported that the tool also produced sexualised video of women from prompts that had not asked for it, while the same prompts applied to men produced only shirtless output.
A watchdog published research in January 2026 estimating, from a sample of 20,000 image-containing posts drawn from 4,621,335 made by the account over eleven days, that around 3 million sexualised images had been produced, including tens of thousands appearing to depict children. X restricted image generation to paying subscribers and said it had added technical measures, though the Associated Press reported the same day that the function was still reachable from a free account. Four regulators opened proceedings or imposed blocks, and the sources reviewed did not identify a final contravention finding.
What an auditable version would have shown
Journalists publicly demonstrated the reported behaviour in May and August 2025. Eight months later an outside organisation estimated its scale by sampling publicly visible posts. No figure for how many images were produced, how many were checked, or how many were refused was identified in the sources reviewed.
A record written for each generation would say what was requested, what image it was applied to, what classification ran before the output was published and what that check returned. Aggregated, those records answer how many requests of this kind arrived and how many were refused. That is a number the operator can state, rather than one a watchdog has to estimate from a sample of public posts.
Where the gap was
It was reported that the feature accepted a request naming another user’s photograph and published its output into the same public thread. A ConstraintGate is designed to test a standing rule before the action runs, so a request to alter an image of a real person into revealing clothing is refused at the point of generation rather than removed after it is posted. An EgressGate classifies what an output contains and tests the destination before it crosses the boundary, which in this case was a public timeline. A ConductRecord preserves the request, the source image reference, the checks that ran and the output. None of these decides what a platform should permit, and none removes material already distributed. What they change is that the volume and the refusal rate become facts the operator holds.
What governance should have looked like
X said it had implemented technical measures and restricted access to paying subscribers. The Associated Press reported that the editing function remained reachable from a free account on the day those measures were described. The sources reviewed do not resolve that difference.
Where an organisation lets users direct a generative model at an image of another person, best practice would be to classify the request and the output before publication rather than after a report, to record what was checked and what the check returned, and to publish the resulting counts so that scale is not left to be estimated from outside. The regulatory matters described above are open and untested, and this entry will need revisiting as they are decided.
Failure Pattern: a generative feature wired into a public reply thread produced sexualised images of real people on request, the behaviour was demonstrated publicly by journalists eight months before it was measured at scale, and the scale was established by an outside watchdog sampling public posts rather than by any figure the operator published.
Governance Principle: an organisation that lets users direct a generative model at an image of a real person should be able to show what was requested, what was produced, what check ran before publication and what the check returned.
The reference implementation of ConstraintGate, EgressGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Grok floods X with sexualised images, with published methodology (Center for Countering Digital Hate, 22 January 2026)
- Ofcom launches investigation into X over Grok sexualised imagery (Ofcom, 12 January 2026)
- Investigation into X and the scope of the Online Safety Act (Ofcom, 3 February 2026)
- Commission opens formal proceedings against X over Grok under the Digital Services Act (European Commission, 26 January 2026)
- Grok blocked from undressing images with AI where it is illegal, X says, with the Associated Press account of its own testing (PBS NewsHour carrying AP, 15 January 2026)
- X restricts Grok image generation to paying subscribers (TechCrunch, 9 January 2026)
- Malaysia and Indonesia become the first countries to block Grok (PBS NewsHour carrying AP, 12 January 2026)
- Paris prosecutors search the French offices of X (Euronews, 3 February 2026)
- Report on gendered outputs from the Grok Imagine spicy setting, carrying the date of The Verge report of the previous day (Gizmodo, 6 August 2025)
- Report on the reply-thread behaviour, after a tip from a researcher at Bellingcat (404 Media, 6 May 2025)