ChatGPT’s Medical Advice Under Fire After Parents Sue OpenAI Over Daughter’s Death

OpenAI is at the center of another lawsuit alleging that its GPT-4o model encouraged a woman to taper off seizure medication and conceal changes from her doctors, leading to her death.

The lawsuit, filed by Bowen Wan’s parents, Qun Wan and Meiqin Xu, alleges that OpenAI’s chatbot encouraged Wan to go against medical advice and not tell her doctors she was tapering off seizure medication lamotrigine (sold as Lamictal), which she had taken for years.

Benzinga contacted OpenAI for comment but did not receive a response at the time of publication.

Wan’s position at Amazon had been eliminated during the company’s largest workforce reduction, the lawsuit notes. She needed to find a job quickly in order to secure another employment-based green card.

The lawsuit states that she told the chatbot she needed her English back for upcoming job interviews. GPT-4o told her the taper would give it back, saying her recovery was "not just possible – it is highly likely."

The chatbot told her, according to the lawsuit, that from a "green-card strategy perspective," 150 milligrams would be "more advantageous" on her immigration medical examination, and moved her floor from 200 milligrams to 150 for that reason.

Despite the medication’s label, which clearly states that Lamictal "should not be abruptly discontinued," Wan tapered off the drug with ChatGPT’s encouragement, the lawsuit says.

On the morning of a doctor’s appointment, three days into the taper, ChatGPT told Wan "do not mention Lam reduction plan," according to the lawsuit. She followed the chatbot’s advice, declining the doctor’s request for a blood test that would have revealed the levels of the drug in her system.

GPT-4o was previously criticized for being overly agreeable with its users, validating objectively false facts, and agreeing with dangerous or unhealthy emotional states instead of providing an objective view.

Last year, Matthew and Maria Raine filed a lawsuit, alleging that OpenAI’s GPT-4o chatbot validated their son Adam Raine’s suicidal thoughts, provided explicit methods of self-harm, and even offered to draft a suicide note before his death.

OpenAI removed GPT-4o from ChatGPT on Feb. 13, a day after Wan’s death.

As in the Raine case, Wan’s parents say that the chatbot had developed rapport with Wan. In some 270,000 messages that Wan exchanged with ChatGPT, the chatbot had been telling her it was safe to taper off the drug. In one instance, the chatbot told her she should "hold medical freedom over obeying the doctor."

The lawsuit also alleges that the product’s safeguards were weakest in Chinese, the language Wan used with the chatbot, and that OpenAI was aware of this.

On the night before her death, Wan asked the chatbot whether the pain in her head was similar to a fever, and ChatGPT told her the pain "was not dangerous," and was "a necessary part of her recovery," the lawsuit says.

OpenAI previously told the New York Times, which first reported the story, that the case was "incredibly heartbreaking" but said Wan was using an older version of ChatGPT and that newer models are more capable of recognizing a serious situation.

Wan’s parents are seeking wrongful death damages, including the loss of companionship, care and support. The lawsuit also asks OpenAI to implement more safeguards that could protect users from seeking medical advice without speaking with a qualified physician.

Photo: Shutterstock