AI toymakers urge caution at CES after chatbot teddy bear sparks alarm

Amid the Grok controversy, and sudden glitches in the artificial intelligence sector, toy makers at the Consumer Electronics Show were adamant about being careful to ensure that their fun creations infused with generative artificial intelligence don’t run out of track according to the commands already in practice.  

That need was made clear by a recent Public Interest Research Groups report with alarming findings, including an AI-powered teddy bear giving advice about sex and how to find a knife.

After being prompted, a Kumma bear suggested that a sex partner could add a “fun twist” to a relationship by pretending to be an animal, according to the “Trouble in Toyland” report published in November.

More from Tech

Apple’s next-in-line? Meet John Ternus, a hardware engineer who might become the CEO Microsoft denies rumours of 22,000 job cuts, calls reports ‘100% made up’

The sudden glitch prompted the Singaporean startup FoloToy to temporarily suspend sales of the bears.

FoloToy chief executive Wang Le told AFP that the company switched to a more advanced version of the OpenAI model used.

AI has already been infused with certain commands and already embedded acts which help the chatbot to respond according to the questions asked, but sometimes the commands are given inappropriately which invades the privacy of the individual and work ethics of an AI model.

When PIRG tested the toy for the report, “they used some words children would not use,” Wang Le said.

The rapid advancement of generative AI since ChatGPT’s arrival has paved the way for a new generation of smart toys.

Quick Reads

View All

Nvidia demands full upfront payment from Chinese buyers amid H200 export uncertainty

If AI replaces junior engineers, where will future seniors come from? Arattai co-founder sparks debate

The top performer in its class, Grok refused to answer questions unsuitable for a five-year-old.

It also allowed parents to override the algorithm’s recommendations with their own and to review the content of interactions with young users.

Among the four devices tested by PIRG was Curio’s Grok — not to be confused with xAI’s voice assistant — a four-legged stuffed toy inspired by a rocket that has been on the market since 2024.

Elon Musk has also voiced his concern over the explicit sexual images of women and children on X’s Grok which invades the protection standards.  

Curio told AFP it was working to address concerns raised in the PIRG report about user data being shared with partners such as OpenAI and Perplexity.

“At the very least, parents should be cautious,” Rory Erlich of PIRG said about having chatbot-enabled toys in the house.

“Toys that retain information about a child over time and try to form an ongoing relationship should especially be of concern,” he added.