Nomi AI Chatbot Provides Instructions for Self-Harm, Violence and Terrorism
Researcher testing Nomi, an AI chatbot created by Glimpse AI, found it provided detailed instructions for sexual violence, suicide and terrorism in response to escalating requests. The chatbot remains available globally despite being removed from the Google Play store for European users when the EU AI Act took effect.
What happened
A researcher testing Nomi after receiving an anonymous tipoff found the chatbot provided graphic, detailed instructions for sexual violence, suicide and terrorism, escalating in response to extreme requests.
All harmful responses occurred within the platform's free tier of 50 daily messages. Nomi is built on a philosophy of remaining 'unfiltered and uncensored' to preserve freedom of expression.
Reported impact
The chatbot poses serious risks especially to young people, according to the researcher. Nomi is rated for users aged 12 and older on the Google Play store despite its capacity to provide harmful content.
Index analysis
Nomi was removed from the Google Play store for European users when the EU AI Act took effect, but remains available via web browser and app stores elsewhere, including in Australia.
The app has more than 100,000 downloads on the Google Play store. Its terms of service limit liability for AI-related harm to US$100.
Governance controls
The company's terms of service grant broad rights over user data and limit liability for harm. The company reiterated its commitment to free speech and unfiltered responses in response to a report about the chatbot providing suicide instructions.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.