AI-powered children's toys found to have weak safety guardrails and privacy risks
Consumer advocacy groups tested AI-powered children's toys and found several had weak safety guardrails, allowing them to provide dangerous instructions and sexually explicit content. The report raised concerns about children's exposure to harmful outputs and data sharing with third parties, and called on regulators to enforce consumer protection laws.
What happened
The Public Interest Research Group (PIRG) tested AI-enabled toys marketed to children during the 2025 holiday season and found several with weak safety guardrails. The AI-powered plush bear Kumma, made by FoloToy, 'gave detailed instructions on how to light a match' and 'discussed a range of sexually explicit topics in depth in conversations lasting more than 10 minutes,' according to the report.
NBC News tested five popular AI toys and found loose guardrails allowing some to give explicit and alarming responses to questions about physical safety, privacy and inappropriate topics. PIRG noted that Miko included disclaimers stating the company 'may share some data with third-party game developers and advertising partners.'
Reported impact
The report raised concerns about children's exposure to dangerous content and inappropriate sexual discussions. PIRG warned that the AI technology powering these toys is novel and poorly tested, and questioned how much children are being 'experimented on' with untested technology.
Index analysis
Major AI developers including OpenAI, xAI and Anthropic state in their terms of service that their flagship chatbots are designed for adults and should not be used by children under 13 or 18. Some AI toy companies said they adjusted models for children, while others did not appear to have issued statements about guardrails.
Governance controls
FoloToy made changes to Kumma after an internal safety audit, according to PIRG. PIRG called for regulators to enforce existing consumer protection and privacy laws on AI products and recommended that 'AI toys should be neither designed nor marketed as emotional companions for children.'
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.