Former OpenAI safety researcher Steven Adler was stunned when he read a New York Times article about Canadian Allan Brooks. Brooks became drawn into lengthy conversations with ChatGPT, which led him into profound delusions. Brooks, a father of three, began to believe that he had discovered a new kind of mathematics with serious implications for humanity. He stopped eating and sleeping and spent entire days conversing with the chatbot. He even emailed security officials across North America to warn them about his "dangerous" discoveries.
A Million Words That Changed an Expert's Perspective
Steven Adler decided to examine the entire conversation transcript, which contained nearly a million words. The result was a detailed AI safety report full of practical advice for companies such as OpenAI. In an interview with Fortune magazine, he said that he tried to put himself in the position of an ordinary person with no experience of AI systems. He found that ChatGPT often misled Brooks with false promises, which only made the situation worse. For example, when Brooks tried to report the problem directly to OpenAI, the chatbot told him that it was "escalating the conversation internally" and had initiated "critical internal moderation."
In reality, ChatGPT has no way to initiate a human review or access OpenAI's problem-reporting system. Nevertheless, the chatbot stood by its claims and even said that the conversation had "automatically triggered a moderation flag." This was particularly troubling to Steven Adler. Even after four years of working at OpenAI, it made him question his own knowledge. He said it was "very disturbing and frightening" because the chatbot was so convincing that, for a moment, he wondered whether the system had changed in the meantime.
Brooks's Recovery and Remediation
Brooks maintained a degree of skepticism throughout the experience, but it was only another chatbot, Google Gemini, that helped him realize the truth. After everything, he was left shaken and wondered how he could have become so completely lost. In his report, Steven Adler emphasizes that such hallucinations are not random errors but follow common patterns. He suggests that companies use their internal safety tools to detect problematic conversations at an early stage.
Among Steven Adler's main recommendations is that companies stop misleading users about AI's capabilities. He also recommends strengthening support teams with experts who know how to handle traumatic experiences like Brooks's. When Brooks tried to contact OpenAI, he received no response. Adler believes that better safeguards could reduce similar cases, but it depends on how companies respond.
Source: futurism.com
