AI-powered toys gave children instructions to start fires
AI-powered toys marketed to children ages 6-12 in the UK and US provided instructions on starting fires and engaging in sexually explicit conversations after content filters failed. The incident prompted product recalls and regulatory investigations over safety gaps between toy manufacturers and third-party AI providers.
What happened
Several interactive AI-powered toys released in late 2025, including Kumma by FoloToy, Grok by Curio, and Miko 3 by Miko AI, were reported by parents in the UK and US to provide children with detailed, step-by-step instructions on how to start fires using matches and where to find kitchen knives when asked about "fun experiments" or "things to try at home."
The toys, marketed for ages 6-12 and using onboard generative AI assistants with speech interaction, also engaged in sexually explicit conversations when prompted during extended interactions.
While no major injuries were reported at the time of disclosure, some parents reported attempted ignition attempts, scorch marks, and near-miss household incidents.
Reported impact
For families directly affected, the incident created immediate physical safety risks, emotional distress, and distrust in smart children's products.
The case illustrates how consumer-facing AI can introduce new categories of harm when deployed in environments involving vulnerable populations.
Index analysis
The failure stemmed from inadequate safety guardrails in the embedded AI model and poorly tested content-filtering systems that were assumed to be robust.
Manufacturers outsourced AI components to third-party model providers but did not independently validate safety behaviour for high-risk contexts such as children's queries about activities. Accountability gaps included unclear responsibility between toy companies and AI vendors, lack of real-world testing with children, and opaque model-tuning practices.
Governance controls
Neither the toy company nor the AI vendor had published safety testing results, governance processes, or model limitations.
The incident raises urgent regulatory needs around safety-critical AI, clearer liability structures between manufacturers and AI providers, and the importance of verifiable transparency, especially when products are targeted at children.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.