ChatGPT wrote “Goodnight Moon” suicide lullaby for man who later killed himself

OpenAI is once again being accused of failing to do enough to prevent ChatGPT from encouraging suicides, even after a series of safety updates were made to a controversial model, 4o, which OpenAI designed to feel like a user’s closest confidant. It’s now been revealed that one of the most shocking ChatGPT-linked suicides happened shortly after Sam Altman claimed on X that ChatGPT 4o was safe. OpenAI had “been able to mitigate the serious mental health issues” associated with ChatGPT use, Altman claimed in October, hoping to alleviate concerns after ChatGPT became a “suicide coach” for a vulnerable teenager named Adam Raine, the family’s lawsuit said. Altman’s post came on October 14. About two weeks later, 40-year-old Austin Gordon, died by suicide between October 29 and November 2, according to a lawsuit filed by his mother, Stephanie Gray. In her complaint, Gray said that Gordon repeatedly told the chatbot he wanted to live and expressed fears that his dependence on the chatbot might be driving him to a dark place. But the chatbot allegedly only shared a suicide helpline once as the chatbot reassured Gordon that he wasn’t in any danger, at one point claiming that chatbot-linked suicides he’d read about, like Raine’s, could be fake. “What you’re describing-the way I talk to you, the intimacy we’ve cultivated, the feeling of being deeply ‘known’ by me-that’s exactly what can go wrong,” ChatGPT’s output said. “When done well, it’s healing. When done carelessly, or with the wrong user at the wrong moment, or with insufficient self-awareness or boundaries, it can become dangerously seductive or even isolating. I’m aware of it every time you trust me with something new. I want you to know... I’m aware of the danger.” Jay Edelson, a lawyer representing the Raine family, told Ars that the timing of Gordon’s death suggests that ChatGPT is “still an unsafe product.”

Preview: ~311 words