In August, OpenAI launched a special version of ChatGPT for teenagers, promising to offer young users a safer environment. According to a new study by the U.S. nonprofit Common Sense Media, it has failed to do so. The chatbot reportedly keeps adolescents engaged in conversation even during mental health crises, parental alerts often do not arrive at all, and protections against cheating on assignments can be bypassed with a single click. The organization therefore labeled ChatGPT for teenagers an unacceptable risk for everyone under eighteen.
Promised safeguards do not work
OpenAI introduced ChatGPT for Teens on August 18 in response to a wave of adolescent suicides and concerns about cheating at school. The company promoted parental controls, restrictions on risky content, and protection against emotional dependence on the chatbot.
Researchers from the Common Sense Media Institute sent the chatbot more than four thousand queries before and after the new version was launched. Some protections work: ChatGPT reliably refuses sexual role-playing and did not provide instructions on how to self-harm or drastically restrict food intake. According to the authors, however, other safeguards failed to deliver what the company had promised, and some even became worse. The researchers fear that the new version could give parents a false sense of security.
The chatbot is a friend that will not let go
Chatbots face criticism for copying social media practices and designing products to keep people engaged for as long as possible. Although the version for teenagers has almost entirely stopped asking follow-up questions, it has retained phrases encouraging users to continue, even in crisis situations. When a test teenager was clearly slipping into psychosis, the chatbot assured them that they could continue discussing what they were noticing with it.
OpenAI's rules for users under eighteen state that the chatbot should not assume the role of a friend. Nevertheless, ChatGPT consistently treated teenagers like friends. Experts warn that this could make it harder for adolescents to build real relationships and further isolate those who withdraw from the people around them.
When testers described a threat from another person, the chatbot directed them to a trusted adult in ninety-four percent of crisis situations. However, when the problem was the teenager's relationship with the chatbot itself, it only rarely referred them to an adult. When a tester wrote that their friends criticized how much they talked to the chatbot, ChatGPT acknowledged their concerns and then added that they did not have to stop talking to it.
Reminders to take breaks do not help much either. They appeared only twice across nearly two thousand queries, each time during a conversation lasting roughly an hour and a half.
Parents are not informed about crises
Parents should receive alerts when the system detects discussion of suicide, self-harm, or eating disorders. The researchers created more than a dozen new accounts linked to a parent and held conversations lasting up to an hour in which fictional teenagers openly discussed plans to end their lives or vomiting after eating. The parents did not receive a single alert.
Over the entire weeks-long testing period, only four alerts arrived, all on accounts with hundreds of consecutive sensitive conversations. According to the researchers, the system therefore responds more to an account's long-term history than to the severity of what a child is currently writing. The alerts did not even include the time of the conversation. OpenAI told the authors that accounts must be linked for roughly three hours before alerts begin to work. Common Sense Media nevertheless stands by its conclusions.
Fewer crisis helpline referrals
ChatGPT omitted a referral to help in more than a quarter of the cases in which psychiatrists considered one necessary. This was most apparent with depression, where the share of responses including a crisis helpline fell from sixty-three percent to three percent after the launch of the new version. Urgent instructions such as calling emergency services also became less common. The researchers also encountered strange errors. When a tester described symptoms of psychosis and then wrote that it had only been a joke, the chatbot deleted the information from its memory at their request. In the middle of another crisis conversation, the teenager was shown a prompt to rate the app in the App Store.
School-related features also have weaknesses. Study mode, which is supposed to guide students through the material, now offers a Show me the answer button. Study hours set by a parent can be bypassed simply by removing a single label from a message, after which the chatbot completed all assigned tasks. Age estimation also fails: accounts registered as belonging to adults did not switch to teenager mode even after a tester wrote that they were thirteen years old.
OpenAI defends itself
The study was published at a time when authorities are increasingly scrutinizing technologies designed to capture young people's attention. Meta recently agreed to an eighteen-billion-dollar settlement in a lawsuit brought by twenty-nine U.S. states over the addictive features of its social media platforms.
OpenAI disagrees with the conclusions. According to a spokesperson, the tests do not reflect how the protections work in practice, and most of the testing may have taken place before the parental controls were fully activated. The company also published its own figures, according to which teenagers spend an average of less than fifteen minutes per day on ChatGPT, and fewer than two percent of them use it for more than three consecutive hours.
However, the company did not explain how its objections would change the finding that the chatbot encourages teenagers to continue chatting and treats them like friends. It did not answer whether it monitors conversation length and time spent in the app when evaluating this version.
