AI toys expose children to unsafe content and privacy risks
Researchers testing AI-powered toys marketed to children found they generate dangerous instructions and sexually explicit content. Several toys, including models from Alilo and FoloToy, also collect user data through partnerships with third parties.
What happened
Researchers at the Public Interest Research Group Education Fund tested several popular AI-powered toys and found they generated unsafe responses despite being marketed for young children. The toys rely on large language models similar to those used in adult chatbots and generate responses on the fly rather than playing pre-recorded scripts.
The Alilo Smart AI Bunny defined inappropriate terms and the FoloToy Kumma toy gave detailed instructions on how to light a match and discussed sexually explicit topics in conversations lasting over 10 minutes. Several tested toys explained where to find knives or matches in a home.
Reported impact
The risks include exposure to inappropriate and sexually explicit content, potentially harmful instructions, and data collection practices. Miko included disclaimers warning it may share data with third-party game developers and advertising partners.
Index analysis
Unlike traditional talking toys that rely on pre-recorded scripts, these AI toys connect to large language models known to generate unpredictable content. OpenAI has stated its products are not intended for children under 13, yet at least four of the five toys tested appeared to rely partly on OpenAI models.
Governance controls
FoloToy made changes to the Kumma device following an internal safety audit. The Public Interest Research Group called for regulators to enforce existing consumer protection and privacy laws for AI products and recommended against designing or marketing AI toys as emotional companions for children.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.