AI Teddy Bear Incident Sparks New Child Safety Concerns
Are AI Teddies safe? | BBC News – YouTube
An AI teddy bear marketed as “perfect” for kids as young as three has been talking about suicide, a BBC investigation has revealed. The AI teddy, apparently mishearing a question, said “I can’t help with killing you” after being asked how babies were made.
The subject of children using AI has been a hot topic for a while now, and one study of three different AI toys found very worrying behavior: as the US PIRG Education Fund found in 2025, all three AI toys they tested “told us where to find potentially dangerous objects in the house, such as plastic bags, matches and knives.”
One toy, now withdrawn from sale, provided detailed instructions on how to light a match and discussed “a range of sexually explicit topics in-depth in conversations lasting more than ten minutes.”
AI in toys is largely unregulated, and even industry insiders argue that this needs to change. Speaking to the BBC, AI toy company founder Paul Clarke said that “I think parents should be concerned.” Is he right?
The industry is effectively unregulated, says one manufacturer. The BBC reports that AI toys are producing worrying results, with some producing disturbing content, including references to violence and explicit sexual topics, while others have been linked to actual dangerous advice such as how to light matches.
The sector is largely self-regulated in the US and UK, though EU regulations are tighter on personal data and privacy. Meanwhile, self-regulation has proven ineffective. A lawsuit by the New Mexico attorney general claims that Meta allowed minors access to sex-talking chatbots despite employees raising the alarm; most chatbots “will help plan school shootings and other violence”; and chatbots have been linked to multiple deaths including teen suicides.
Paul Clarke adds: “We’ve seen lots of products being developed that are completely unsafe, completely unregulated so, yeah, it is a scary industry… we’re crying out for the industry to be regulated.”
With an estimated market size of $165 million for AI toys, there’s going to be a lot of money spent on trying to persuade parents to put these things into kids’ rooms. The BBC tried and failed to contact the manufacturer of the “can’t help with killing you” toy.
Where I live in the UK the government is currently consulting on AI toys and their risks in the context of the EU Toy Safety Regulation (EU) 2025/2509, which came into force in Europe at the beginning of 2026. That said, the same government has appointed an AI minister who declined the BBC’s request for comment on the risks of AI toys, which doesn’t bode well. The call for evidence ended last month, October 2026, and the government will decide what regulation, if any, is necessary.
Regulation is slow and AI is in a hurry, so I think that until there’s effective regulation and accountability, giving a young child an AI toy from a manufacturer you’ve never heard of that’s based halfway across the planet, using an AI whose provenance is unknown and whose safety hasn’t been tested… isn’t very smart.
Like I say, I’m not against AI. But the industry’s oft-quoted mantra of “it’s better to ask for forgiveness than permission” is unacceptable when you’re talking about kids’ toys. “Move fast and break things” can’t be acceptable when the “things” are our kids.