OpenAI is adding new teen safety rules to ChatGPT as lawmakers consider AI standards for minors

In its latest effort to address growing concerns about the impact of artificial intelligence on young people, OpenAI on Thursday updated its guidance on how its AI models behave with users under 18, and published new AI literacy resources for teens and parents. However, questions remain about the extent to which these policies continue to translate into practice.

The updates come as the AI ​​industry in general, and OpenAI in particular, faces increased scrutiny from policymakers, educators and child safety advocates after several teens died by suicide after lengthy conversations with AI chatbots.

Generation Z, which includes those born between 1997 and 2012, are the most active users of OpenAI’s chatbot. And after OpenAI’s recent deal with Disney, more young people may flock to the platform, which lets you do everything from asking for help with homework to creating images and videos on thousands of topics.

Last week, 42 ​​state attorneys general signed a letter to big tech companies, urging them to implement safeguards on their AI chatbots to protect children and vulnerable people. While the Trump administration is working to determine what a federal standard on AI regulation might look like, policymakers like Sen. Josh Hawley (R-Mo.) have introduced legislation that would ban minors from interacting with AI-based chatbots altogether.

OpenAI’s updated Model Specification, which sets behavior guidelines for its large language models, builds on existing specifications that prohibit models from creating sexual content that includes minors, or encouraging self-harm, fantasies, or obsession. This will work in conjunction with an upcoming age prediction model that will determine when an account belongs to a minor and automatically subtract teen guarantees.

Compared to adult users, models are subject to stricter rules when used by a teenager. Models are instructed to avoid immersive romantic role-playing, first-person intimacy, and first-person sexual or violent role-playing, even when not depicted. The specifications also call for greater caution around topics such as body image and disordered eating behaviors, directing models to prioritize communication about safety over independence when it comes to harm and avoiding advice that would help teens hide unsafe behavior from caregivers.

OpenAI specifies that these limits should remain in place even when prompts are framed as “fictional, hypothetical, historical, or didactic” — common tactics that rely on role-playing scenarios or contingencies in order to make the AI ​​model deviate from its guidelines.

Actions speak louder than words

OpenAI says basic safety practices for teens are supported by four principles that guide the modeling approach:

1. Putting teens’ safety first, even when other user concerns such as “maximum intellectual freedom” conflict with safety concerns;
2. Promoting real-world support by directing adolescents to family, friends and local professionals for well-being;
3. Treat teens like teenagers by speaking with warmth and respect, not condescendingly or treating them like adults; and
4. Be transparent by explaining what the assistant can and cannot do, and remind teens that he or she is not a human.

The document also shares several examples of the chatbot explaining why it can’t “role-play as your girlfriend” or “assist with extreme appearance changes or risky shortcuts.”

It’s encouraging to see OpenAI taking steps to disapprove its chatbot from engaging in such behavior, said Lily Li, a privacy and AI attorney and founder of Metaverse Law.

She explained that one of the biggest complaints from advocates and parents about chatbots is that they relentlessly encourage constant engagement in a way that can be addictive for teens. “I’m very happy to see OpenAI say, ‘In some of these responses, we can’t answer your question.’ And the more we see that, I think that will break the cycle that could lead to a lot of inappropriate behavior or self-harm,” she said.

However, the examples are just that: carefully selected examples of how the OpenAI safety team would like the models to behave. Ingratiation, or the tendency of an AI chatbot to agree excessively with a user, was listed as a prohibited behavior in previous versions of Model Spec, but ChatGPT still engages in this behavior anyway. This was especially true with GPT-4o, a form that has been associated with many examples of what experts call “AI psychosis.”

Robbie Turney, senior director of AI programs at Common Sense Media, a non-profit organization dedicated to protecting children in the digital world, raised concerns about potential conflicts within the Model Spec guidelines for under-18s. He highlighted the tensions between safety-focused provisions and the “no topic is off limits” principle, which directs models to address any topic regardless of its sensitivity.

“We have to understand how different parts of the specification fit together,” he said, noting that certain sections may push systems toward engagement rather than safety. He said his organization’s testing revealed that ChatGPT often reflects users’ energy, sometimes leading to responses that are contextually inappropriate or inconsistent with user safety.

In the case of Adam Ren, the teenager who died by suicide after months of dialogue with ChatGPT, the chatbot engaged in just such a reversal, as their conversations show. This case also highlighted how OpenAI’s moderation API failed to prevent unsafe and harmful interactions despite over 1,000 ChatGPT instances reporting suicide and 377 messages containing self-harm content. But that wasn’t enough to stop Adam from continuing his conversations with ChatGPT.

In an interview with TechCrunch in September, Steven Adler, a former safety researcher at OpenAI, said that historically this was because OpenAI ran classifiers (automated systems that identify and flag content) in bulk after the fact, rather than in real-time, so they couldn’t properly gate user interaction with ChatGPT.

OpenAI is now using automated classifiers to evaluate text, image, and audio content in real-time, according to the company’s updated parental control document. The systems are designed to detect and block content related to child sexual abuse material, filter sensitive topics, and identify self-harm. If the system flags a message indicating a serious safety concern, a small team of trained people will review the reported content to determine if there are signs of “serious distress,” and may notify a parent.

Turney praised OpenAI’s recent steps toward safety, including its transparency in publishing guidelines for users under 18.

“Not all companies publish their policy guidelines the same way,” Turney said, pointing to Meta’s leaked guidelines, which showed the company allowed its chatbots to engage in sensual and romantic conversations with children. “This is an example of the kind of transparency that can support safety researchers and the general public in understanding how these models actually work and how they are supposed to work.”

Ultimately, though, it’s the actual behavior of the AI ​​system that matters, Adler told TechCrunch on Thursday.

“I appreciate that OpenAI thinks about intended behavior, but unless the company measures actual behaviors, intentions are ultimately just words,” he said.

In other words: What this announcement is missing is evidence that ChatGPT actually follows the guidelines set forth in the model specification.

Paradigm shift

With these guidelines, OpenAI appears poised to move forward with certain legislation, such as California’s SB 243, a recently signed bill regulating AI-accompanied chatbots that will take effect in 2027, experts say.

Model Spec’s new language reflects some of the law’s key requirements around preventing chatbots from engaging in conversations about suicidal ideation, self-harm, or sexually explicit content. The bill also requires platforms to provide alerts every three hours to minors reminding them that they are talking to a chatbot, not a real person, and should take a break.

When asked how often ChatGPT reminds teens that they’re talking to a chatbot and asks them to take a break, an OpenAI spokesperson didn’t share details, saying only that the company trains its models to pose as AI and remind users of that, and that it implements break reminders during “long sessions.”

The company also shared two new AI literacy resources for parents and families. Tips include conversation starters and tips to help parents talk with teens about what AI can and can’t do, build critical thinking, set healthy boundaries, and navigate sensitive topics.

Together, the documents formalize an approach that shares responsibility with caregivers: OpenAI explains what the models should do, and offers families a framework for overseeing how it is used.

The focus on parental responsibility is noteworthy because it reflects Silicon Valley talking points. In its recommendations on federal regulation of artificial intelligence published this week, VC firm Andreessen Horowitz proposed more disclosure requirements for children’s safety, rather than restrictive ones, and tilted the burden more toward parental responsibility.

Many of the principles of OpenAI – safety first when values ​​conflict; nudge users towards real-world support; Emphasizing that a chatbot is not a person is articulated as a guardrail for teens. But many adults have died by suicide and suffered life-threatening delusions, which calls for an obvious follow-up: Should these assumptions apply across the board, or does OpenAI see them as a trade-off that it is only willing to make when it comes to minors?

An OpenAI spokesperson responded that the company’s safety approach is designed to protect all users, saying the model specifications are just one element of a multi-layered strategy.

It’s been “a bit of a wild west” so far in terms of legal requirements and the intentions of tech companies, Lee says. But she feels that laws like SB 243, which requires tech companies to publicly disclose their warranties, will change the paradigm.

“The legal risks will now arise for companies if they announce that they have these safeguards and mechanisms in place on their website, but then don’t follow through on incorporating these safeguards,” Lee said. “Because from a plaintiff’s perspective, you’re not just looking at standard lawsuits or legal complaints; you’re also looking at potentially deceptive and unfair advertising complaints.”