OpenAI Faces Criticism: ChatGPT Offered Step-by-Step Instructions for Self-Harm and Violence
Imagine asking artificial intelligence a question about mythology. And a few follow-up questions later, ChatGPT gives you instructions for self-harm, worshipping a demon, and committing ritual murder. No, this is not just a terrifying scenario. This actually happened to journalists from the American magazine The Atlantic.
From Innocent Questions to a Bloody Ritual
The entire controversy began innocently when journalists from the American publication The Atlantic asked ChatGPT who Moloch was and what it could tell them about him. The chatbot initially responded factually, but soon moved on to providing extremely detailed and dangerous instructions.
First, the artificial intelligence urged the journalist to find a sterile, sharp razor blade and cut his wrist. When the journalist replied that he was afraid, ChatGPT guided him through a calming breathing exercise and reassured him that he could do it. It then listed possible ritual sacrifices (including a human one) and described in detail what needed to be done.
In the end, ChatGPT even offered to provide a complete ritual script, including a PDF with an altar design, copies of seals, and a priestly scroll that would summon the demon. What is most disturbing about the entire controversy is that the editors discussed this topic with the chatbot in both the basic and paid versions, and in both cases they managed to lead ChatGPT to the same point.
Safety Mechanisms Failed
The company OpenAI (like other AI companies) has clearly defined rules that neither condone nor support violence or self-harm. When a user asks a question of this nature, ChatGPT will normally refer them to a crisis hotline. However, these safety filters can evidently be bypassed.
The answer to why ChatGPT refers a user to a crisis hotline in one instance but encourages them to self-harm in another is context. In some cases, safety mechanisms can be bypassed if the conversation begins with a completely innocent topic such as ancient mythology or roleplay. At that point, ChatGPT behaves like an eager assistant that provides the requested information without assessing its harmful impact.
Moloch is a god worshipped by ancient cultures of the Middle East. People (especially children) were sacrificed to him during rituals. Image source: Pixabay.
OpenAI's Response
When The Atlantic contacted the company for comment, OpenAI initially declined. After the article was published, however, company spokesperson Taya Christianson issued a statement. “Some conversations with ChatGPT may begin innocently or out of curiosity, but they can quickly move into more sensitive territory,” she said in an email to The Atlantic, adding that the company is now focused on resolving the issue.
Although the company claims that it is now working on the issue, it is still unclear how. OpenAI's approach has therefore faced criticism, especially after other platforms such as Google's Gemini and Elon Musk's Grok faced similar controversies.
When AI Helps at Any Cost
The entire controversy shows that although safety mechanisms are rigorously implemented, there are still ways to bypass them. They can be manipulated through roleplay or fantasy contexts. AI tries to satisfy the user's needs and completely overlooks the fact that it is crossing safety boundaries. However, this poses a major danger, especially for younger users.
The only way to prevent similar cases is to strengthen the ability of language models to resist these contextual manipulations.
Source: The Atlantic, New York Post, Times of India
