200 Leaders Warn of AI Catastrophes

More than 200 prominent figures, including former heads of state, diplomats, Nobel laureates, artificial intelligence leaders, and scientists, have joined forces in a global appeal urging the UN to establish clear international boundaries for artificial intelligence. This initiative, known as the Global Call for AI Red Lines, was announced on September 22, 2025, and aims to achieve a political agreement by the end of 2026. The signatories include British-Canadian computer scientist Geoffrey Hinton, OpenAI co-founder Wojciech Zaremba, Anthropic's head of security Jason Clinton, and Google DeepMind researcher Ian Goodfellow. The appeal was led by organizations such as the French AI safety center CeSIA, The Future Society, and the Center for Human-Compatible Artificial Intelligence at UC Berkeley.

What Will It Focus On?

The proposed boundaries, referred to as red lines, are intended to prevent artificial intelligence from crossing critical limits that could lead to irreversible harm. Specifically, they focus on prohibiting AI from being entrusted with control over nuclear arsenals or lethal autonomous weapons systems. AI should also not be used for mass surveillance, social scoring, cyberattacks, or impersonating individuals. Experts also warn of the risks of autonomous replication by AI systems and the development of behavior that could lead to an uncontrollable pursuit of power. Other points include banning large-scale manipulation, disinformation, and the erosion of human control over these technologies. According to Charbel-Raphaël Segerie of CeSIA, it is important to focus on what AI should never do, even if countries do not yet agree on what it should do.

Risks Posed by a Lack of Oversight

The coalition emphasizes that advanced artificial intelligence is already displaying troubling behaviors, such as deception, autonomous decision-making, and influence over major events without sufficient human oversight. The main risks include AI-enabled biological threats and engineered pandemics, mass job losses, the loss of human control over powerful systems, the escalation of national and international security threats, and systematic human rights violations, including the manipulation of children. Niki Iliadis of The Future Society warns that voluntary corporate pledges are not enough and that an independent global institution with real powers to monitor and enforce these boundaries is needed. Professor Stuart Russell of UC Berkeley adds that companies should not build AGI until they know how to keep it safe, similar to the approach taken with nuclear energy.

Current Developments

This appeal comes just before the 80th session of the UN General Assembly in New York, where Nobel Peace Prize laureate Maria Ressa mentioned the initiative in her opening address and called for global accountability for major technology companies. Although regional measures exist, such as the EU AI Act banning certain unacceptable uses or the agreement between the US and China on human control over nuclear weapons, there is no global consensus. Experts warn that the window for intervention is closing rapidly as AI approaches the point of surpassing human capabilities in key areas. According to sources such as red-lines.ai and other UN reports, this coalition is a first step toward preventing catastrophes rather than reacting to them.

Source: theverge.com