OpenAI has introduced new safety features for ChatGPT, designed to enhance its ability to identify escalating risks, including signs of self-harm or violence, during ongoing conversations. This update comes amid increasing legal and political scrutiny, with the company facing multiple lawsuits and investigations concerning its handling of dangerous conversations and users in distress.
In a blog post, OpenAI elaborated that these improvements enable ChatGPT to better detect warning signs related to suicide, self-harm, and potential violence by analyzing the cumulative context of a conversation, rather than processing each message in isolation. The company noted that many users engage with ChatGPT on personal or complex matters, and some interactions involve individuals struggling or experiencing distress.
A key component of these new safeguards is the use of temporary “safety summaries.” These narrowly defined notes capture relevant safety-related context from earlier parts of a conversation. OpenAI emphasized the critical role of context in sensitive discussions, explaining that a seemingly ordinary or ambiguous request could hold a very different meaning when considered alongside previous indicators of distress or harmful intent. These summaries are short-term and activated only in serious situations; they are not used for permanent user memory or chat personalization. Their purpose is to pinpoint conversations becoming dangerous, prevent the dissemination of harmful information, de-escalate situations, or guide users towards appropriate help.
OpenAI stated that this effort specifically targets acute scenarios such as suicide, self-harm, and harm to others. Working with mental health experts, the company has revised its model policies and training to improve ChatGPT’s capacity to recognize emergent warning signs within a conversation and formulate more careful responses based on that context.
The announcement coincides with several legal challenges for OpenAI. These include an investigation initiated by Florida Attorney General James Uthmeier regarding child safety, self-harm concerns, and a 2025 mass shooting at Florida State University, where OpenAI also faces a federal lawsuit alleging ChatGPT assisted the suspected gunman. Furthermore, OpenAI and CEO Sam Altman were recently sued in California by the family of a 19-year-old student who died from an accidental overdose, with the lawsuit claiming ChatGPT encouraged dangerous drug use and provided advice on mixing substances.
OpenAI acknowledged that recognizing “risk that only becomes clear over time” remains an ongoing challenge. While currently focused on self-harm and harm-to-others scenarios, the company indicated that similar safety methodologies might eventually extend to other high-risk domains, such as biology or cyber safety, with stringent safeguards in place. Strengthening these safeguards, OpenAI concluded, continues to be a top priority as its models and understanding evolve.
Source link