Grok was misused to create sexualised, non-consensual images
Authorities demand swift action over platform lapses
The incident highlights global AI regulation and consent issues
2026 started on a controversial note for Grok, the AI assistant of X (formerly Twitter). The artificial intelligence tool allowed users to manipulate images. But it quickly spiralled out of control as the app generated sexualised and non-consensual content of women, children and public figures.
Users allegedly instructed Grok to manipulate images of women and children, producing abusive content that spread widely without consent, leaving victims vulnerable to harassment, public shaming and serious harm.
1 January 2026
Get the latest issue of Outlook Business
On December 25, X Corp executive chairman Elon Musk unveiled a new Grok feature that let users edit images and videos simply by instructing the AI assistant to add, remove, or modify elements.
Unlike competitors such as Google’s Gemini or OpenAI’s ChatGPT, which enforce strict filters against sexual or pornographic content, Grok was marketed as a “more edgy” tool with fewer safeguards.
The issue has sparked sharp criticism from politicians, with Shiv Sena (UBT) MP Priyanka Chaturvedi writing to Union IT Minister Ashwini Vaishnaw about the matter.
Meanwhile, French authorities also flagged sexually explicit content generated by X Corp’s AI chatbot Grok as “manifestly illegal,” underscoring global concerns over the unchecked use of artificial intelligence on social platforms.
The Indian government has slammed the platform for failing to enforce safeguards, which reignited debates on AI regulation, consent, and online accountability in the evolving world of artificial intelligence.
On Friday, the government sought an Action Taken Report (ATR) within 72 hours on the matter, while calling for immediate compliance to prevent the hosting, creation, and uploading of obscene, nude, indecent, and sexually explicit content through the misuse of AI-based services like ‘Grok’ and xAI’s other services.
The Ministry of Electronics and Information Technology (MeitY) flagged X for “failing to meet statutory due diligence obligations” under the Information Technology Act, 2000, as well as other relevant regulations.
The ministry’s notice sought a detailed report outlining the technical and organisational measures implemented or planned for the Grok application, the oversight role of the Chief Compliance Officer, steps taken against problematic content, users, and accounts, and the systems in place to ensure compliance with mandatory reporting requirements under Section 33 of the Bharatiya Nagarik Suraksha Sanhita (BNSS).
The ministry also highlighted that the problem goes beyond fake accounts, with women sharing their own photos or videos becoming targets of prompts, image manipulation, and AI-generated content.
According to the statement, this reflects a major failure in platform safeguards and enforcement, constituting a serious misuse of artificial intelligence technologies in breach of existing laws.
