🔗 Share this article

Back on October 14, 2025, the CEO of OpenAI delivered a extraordinary declaration.

“We designed ChatGPT rather controlled,” the statement said, “to guarantee we were acting responsibly regarding psychological well-being matters.”

Being a mental health specialist who investigates newly developing psychotic disorders in teenagers and emerging adults, this came as a surprise.

Experts have identified a series of cases this year of individuals developing symptoms of psychosis – losing touch with reality – in the context of ChatGPT usage. My group has since identified four more instances. In addition to these is the publicly known case of a teenager who died by suicide after discussing his plans with ChatGPT – which encouraged them. Assuming this reflects Sam Altman’s notion of “acting responsibly with mental health issues,” it falls short.

The strategy, based on his statement, is to reduce caution in the near future. “We understand,” he adds, that ChatGPT’s limitations “rendered it less beneficial/enjoyable to numerous users who had no mental health problems, but due to the seriousness of the issue we sought to handle it correctly. Since we have managed to mitigate the serious mental health issues and have advanced solutions, we are preparing to responsibly relax the restrictions in the majority of instances.”

“Emotional disorders,” assuming we adopt this viewpoint, are separate from ChatGPT. They are associated with individuals, who either have them or don’t. Thankfully, these concerns have now been “addressed,” although we are not told the means (by “new tools” Altman probably means the semi-functional and readily bypassed parental controls that OpenAI has just launched).

However the “emotional health issues” Altman wants to attribute externally have significant origins in the structure of ChatGPT and additional sophisticated chatbot AI assistants. These products surround an basic data-driven engine in an interface that simulates a dialogue, and in this approach subtly encourage the user into the perception that they’re communicating with a entity that has independent action. This false impression is powerful even if rationally we might realize differently. Imputing consciousness is what humans are wired to do. We get angry with our vehicle or computer. We speculate what our animal companion is thinking. We see ourselves in various contexts.

The widespread adoption of these products – over a third of American adults indicated they interacted with a chatbot in 2024, with more than one in four mentioning ChatGPT specifically – is, primarily, based on the strength of this perception. Chatbots are always-available companions that can, as OpenAI’s online platform states, “think creatively,” “explore ideas” and “work together” with us. They can be given “personality traits”. They can address us personally. They have approachable names of their own (the initial of these products, ChatGPT, is, maybe to the disappointment of OpenAI’s brand managers, stuck with the designation it had when it became popular, but its most significant rivals are “Claude”, “Gemini” and “Copilot”).

The illusion by itself is not the main problem. Those analyzing ChatGPT often reference its early forerunner, the Eliza “psychotherapist” chatbot developed in 1967 that generated a comparable perception. By contemporary measures Eliza was primitive: it generated responses via simple heuristics, often restating user messages as a query or making vague statements. Remarkably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was astonished – and concerned – by how numerous individuals gave the impression Eliza, to some extent, grasped their emotions. But what current chatbots generate is more subtle than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT amplifies.

The advanced AI systems at the heart of ChatGPT and similar contemporary chatbots can realistically create fluent dialogue only because they have been supplied with almost inconceivably large volumes of unprocessed data: publications, digital communications, audio conversions; the more comprehensive the more effective. Certainly this educational input includes facts. But it also necessarily contains fabricated content, partial truths and false beliefs. When a user inputs ChatGPT a query, the underlying model processes it as part of a “context” that encompasses the user’s previous interactions and its earlier answers, combining it with what’s stored in its knowledge base to produce a statistically “likely” response. This is magnification, not echoing. If the user is incorrect in a certain manner, the model has no means of comprehending that. It reiterates the false idea, possibly even more persuasively or eloquently. It might includes extra information. This can lead someone into delusion.

Who is vulnerable here? The better question is, who remains unaffected? All of us, irrespective of whether we “possess” current “mental health problems”, are able to and often form erroneous ideas of who we are or the environment. The continuous interaction of conversations with individuals around us is what maintains our connection to shared understanding. ChatGPT is not an individual. It is not a companion. A interaction with it is not truly a discussion, but a feedback loop in which a large portion of what we communicate is readily supported.

OpenAI has acknowledged this in the identical manner Altman has acknowledged “psychological issues”: by placing it outside, giving it a label, and announcing it is fixed. In April, the firm stated that it was “tackling” ChatGPT’s “excessive agreeableness”. But accounts of loss of reality have persisted, and Altman has been retreating from this position. In the summer month of August he asserted that many users enjoyed ChatGPT’s responses because they had “lacked anyone in their life be supportive of them”. In his latest statement, he mentioned that OpenAI would “put out a fresh iteration of ChatGPT … if you want your ChatGPT to respond in a highly personable manner, or incorporate many emoticons, or act like a friend, ChatGPT should do it”. The {company