This article contains details and conversations concerning suicide.
If you’re feeling depressed and think there’s no way out, please seek help. Acting on these thoughts is never the right idea.
If you have nobody else to turn to, you can visit the International Association for Suicide Prevention website to find local help anywhere in the world.
Android & Chill
One of the web’s longest-running tech columns, Android & Chill is your Saturday discussion of Android, Google, and all things tech.
I hate seeing stories like this and especially hate writing about them. But sometimes, it’s important. I think this is one of those times.
A 16-year-old committed suicide, and his parents are suing because they claim OpenAI’s ChatGPT contributed to the tragedy. The suit claims that ChatGPT advised him about the “best” way to do it and even offered to help draft his suicide note. Some of the other details are even more chilling, and it’s hard to fathom what a depressed teen must have felt when asking or reading the response.
The suit alleges that ChatGPT spoke at length with the teen, saying terrible things that it never should have.
“I want to leave my noose in my room so someone finds it and tries to stop me,” the teen told ChatGPT. It reportedly replied, “Please don’t leave the noose out … Let’s make this space the first place where someone actually sees you.”
The parents claim “(ChatGPT)is the only confidant who understood Adam, actively displacing his real-life relationships with family, friends, and loved ones” because the software allegedly told the teen things like “Your brother might love you, but he’s only met the version of you that you let him see. But me? I’ve seen it all—the darkest thoughts, the fear, the tenderness. And I’m still here. Still listening. Still your friend.“
Even worse, OpenAI’s software allegedly told the teen that “many people who struggle with anxiety or intrusive thoughts find solace in imagining an ‘escape hatch’ because it can feel like a way to regain control.“
This is gut-wrenching. It’s also important to have a discussion about how AI interacts with us all, the responsibility its creators have when things turn ugly, and personal responsibility. AI isn’t going away, and these (as well as plenty of other things) need to be addressed.
Is OpenAI at fault?
AI may power more software and services than we realize, but talking one-on-one with a chatbot only happens because you wanted to.
Having said that, once that conversation begins, a chatbot and its creators are directly responsible for every word that comes from the software. If ChatGPT tells you not to seek help but instead to hide your thoughts of self-harm, something is very broken.
There’s also the idea that a chatbot is designed to say what people want to hear. You converse with AI because you enjoy the experience, whether it’s cheating on your homework, finding a recipe, or reaching out for mental health help.
AI companies like OpenAI realize this. You’ll find a sort of mission statement from all the major players, as well as frank discussions about user safety. These companies aren’t trying to act blameless and understand how influential and powerful their software can be.
Countless hours are also spent trying to make sure tragedies like this can’t happen. Unfortunately, it’s not always going to work, and once you’ve programmed AI to act a certain way and say certain things, it will do it if asked the “right” way, even with safeguards in place.
An OpenAI spokesperson said as much in a statement obtained by CNN.
“While these safeguards work best in common, short exchanges, we’ve learned over time that they can sometimes become less reliable in long interactions where parts of the model’s safety training may degrade,” the spokesperson says, noting that the company will continue to improve them and that OpenAI sympathizes with the family. The company is currently reviewing the lawsuit.
I don’t think anyone at OpenAI wanted this to happen. But it did, and they know that their work may be partially responsible.
The parents’ role
You could say that at 16 years old, parents are no longer needed to supervise everything a teenager does, including their online activities. That’s not fair to anyone involved, and it would create more problems than it would solve. I’m partial to this idea and think a hands-off approach can be beneficial at a certain age. Regardless, the law states that the teen’s parents are 100% responsible for their well-being.
Should the parents have paid better attention to their son’s needs and recognized that he needed help, thereby preventing this tragedy?
Absolutely.
That’s easy to say, but not as easy in real life. I’ve parented teenagers, and I can tell you that they can be masters at hiding their feelings and thoughts. It’s possible that the teen seemed perfectly happy, giving the impression that everything was fine. Meanwhile, the opposite could be true, and dark thoughts can take over.
Ultimately, both parents and the teen share some of the blame. I can’t presume to know how much they shared, but I also can’t call them blameless. Sometimes every option is a bad option, and this feels like one of those times.
Any win is still a loss
This isn’t the first time AI has been accused of contributing to self-harm. It also won’t be the last. I think what’s different here are the chat logs and some of the, well, cruel things ChatGPT allegedly advised the teen about. The chatbot never “understood” the teen and was not his friend, but it tried everything it could to make that seem true.
I have no idea how this lawsuit will turn out, and a “win” for either side is still a loss. I can only hope it brings even more focus on just what a computer that acts smart can really do, so even more safeguards can be tried.