X implements tech measures to prevent Grok from generating sexualised AI content

Microblogging platform X has implemented technological measures to prevent Grok from generating images of real people in revealing clothing in jurisdictions where it is illegal, following backlash over obscene deepfakes created by the AI chatbot.

This restriction applies to all users, including paid subscribers, X said on its official 'Safety' handle.

Further, image creation and editing via the Grok account on X are now only available to paid subscribers. The platform said this adds an extra layer of protection, ensuring that individuals who attempt to abuse Grok to violate the law or platform policies can be held accountable.

"We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal," the post said. X also said it has implemented measures to prevent editing of images of real people in revealing clothing.

"This restriction applies to all users, including paid subscribers," the platform added. X emphasised that all AI prompts and generated content must still adhere to its safety rules. "Whether content is created by free or paid users, our Safety team works around the clock to add safeguards, remove violating content, suspend accounts when necessary, and collaborate with governments and law enforcement," it said.

X reiterated its commitment to a safe platform, with zero tolerance for child sexual exploitation, non-consensual nudity, and unwanted sexual content. "We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity," the post said. Accounts seeking such content are reported to law enforcement as necessary.

The platform recently removed about 3,500 pieces of content and deleted over 600 accounts after the Indian IT Ministry raised concerns about obscene material linked to Grok. X assured authorities it would comply with Indian laws.

Governments worldwide, including India, have increased pressure on Grok over content moderation, data safety, and non-consensual sexually explicit images flooding the platform. On January 2, the IT Ministry directed X to remove all vulgar, obscene, and unlawful content generated by Grok, warning of action under the law if it failed to comply.

The ministry asked X to submit a detailed action-taken report covering technical and organisational measures, the oversight role of the Chief Compliance Officer, actions against offending content, and mechanisms for compliance with reporting requirements. It highlighted misuse of Grok to create fake accounts or manipulate images of women in a derogatory manner, noting this as a serious failure of platform safeguards.

The government emphasised that compliance with the IT Act and rules is mandatory, and failure to observe due diligence obligations could result in loss of safe harbour under section 79 of the IT Act and other legal consequences. X was asked to enforce user terms and AI usage restrictions, including suspension and termination of violating accounts, and to remove all illegal content without delay.

X’s initial detailed response was seen as reiterating respect for Indian laws and content policies but lacking specific actions on Grok AI obscene content. The IT Ministry sought more concrete details on content takedowns and preventive measures.

Following this, X admitted its mistakes and assured compliance with Indian laws. Apart from India, the UK and EU have also pressured X over the Grok deepfake image issue.