Artificial Intelligence Is Starting to Improve Itself—Experts Warn of Losing Control!

Artificial Intelligence Is Starting to Improve Itself—Experts Warn of Losing Control!

Ondřej Barták
Ondřej Barták
Entrepreneur and Programmer
29. 1. 2026
4 minutes reading · 18 views
Artificial Intelligence Is Starting to Improve Itself—Experts Warn of Losing Control!

Leading artificial intelligence companies have begun using their own AI systems to accelerate research and development. Each new generation of AI models thus contributes to creating the next generation. According to a new report from Georgetown University's Center for Security and Emerging Technology (CSET), this practice is growing, and new models are often used internally before they are released publicly.

DeepMind CEO Demis Hassabis confirmed during an interview at Axios House Davos that Google is actively exploring whether models can "continue learning in the wild after you finish training them." OpenAI CEO Sam Altman said during a livestream last year that OpenAI would build a "true automated AI researcher" by March 2028.

Recursive self-improvement is key

The technical term for this approach is "recursive self-improvement," and it is considered a key technique that could sustain rapid AI progress. Richard Socher, CEO of You.com, is founding a new startup focused specifically on this area. The startup is raising hundreds of millions of dollars in funding, which could value the company at approximately CZK 108 billion.

"AI is code, and AI can code," Socher said. "And if you can close that loop properly, you could actually automate the scientific method to essentially help humanity."

Using their own systems

The CSET report reveals that technical staff at leading AI companies spend a large portion of their time using AI tools to assist with their work. Anthropic describes its technical teams' intensive use of AI tools in its public materials. "New data scientists on our Infrastructure team give Claude Code their entire codebase so they can quickly become productive. Claude reads the codebase's CLAUDE.md files, identifies the relevant ones, explains data pipeline dependencies, and shows which upstream sources feed into dashboards," Anthropic states.

During incidents, the security engineering team gives Claude Code stack traces and documentation to track the flow of control through the codebase. Problems that typically take 10-15 minutes of manual scanning are now resolved three times faster.

In July 2025, CSET held an expert workshop that brought together participants from leading AI companies, government, academia, and civil society. The workshop revealed fundamental disagreements among experts about the future of AI research automation. Some expect rapid progress toward a high degree of automation and highly advanced capabilities, while others expect slower progress that will reach its ceiling much sooner. The report identifies several possible scenarios, ranging from an "intelligence explosion" to a situation in which automation encounters insurmountable obstacles.

Demis Hassabis in Davos on the Axios stage.
DeepMind CEO Demis Hassabis in Davos on the Axios stage.

Main risks

The CSET report identifies two main ways in which increasing automation of AI research could heighten societal risks. First, by reducing humans' ability to understand and control AI research. Second, by reducing the time available for people to navigate rapidly improving AI capabilities.

As AI plays a larger role in research workflows, human oversight of AI research processes would likely decline. If AI systems were to contribute significantly to AI research, the research process would likely produce fewer human-understandable outputs, with less time for human review. Faster AI progress resulting from the automation of AI research would make it more difficult for people, including researchers, executives, policymakers, and the public, to notice, understand, and intervene as AI systems develop increasingly impressive capabilities.

The CSET report emphasizes that policymakers currently lack reliable visibility into the automation of AI research and are overly dependent on voluntary disclosures from companies. The authors propose greater transparency, targeted reporting, and updated safety frameworks, while warning that poorly designed mandates could have the opposite effect. Currently, anyone seeking access to empirical evidence about AI research automation is heavily dependent on voluntary disclosures from leading AI companies. While companies choose to publish some data relevant to AI research automation, it tends to be incomplete.

Warning of civilization-level risks

Anthropic CEO Dario Amodei warns in a 38-page essay of the imminent "real danger" that superhuman intelligence will cause civilization-level harm without smart and rapid intervention. "I believe we are entering a rite of passage, both turbulent and inevitable, that will test who we are as a species," Amodei writes. "Humanity is about to be handed almost unimaginable power, and it is profoundly unclear whether our social, political, and technological systems have the maturity to wield it."

Although experts disagree about their likelihood, there are possible scenarios in which AI research becomes highly automated, the pace of AI research accelerates dramatically, and the resulting systems pose extreme risks. This calls for preventive measures now.

Advertisement

Content created with help from UpTier.

SEO and GEO on autopilot. UpTier’s multi-agent systems write and optimize content for search engines and AI answers.

Discover UpTier ↗

Category:AI
Did you enjoy this article?
Discover more interesting posts on our blog
Back to blog

Related posts

OpenAI cracks down on child abuse. ChatGPT will block accounts for a full dayOpenAI cracks down on child abuse. ChatGPT will block accounts for a full day
OpenAI is strengthening child protection: a new system can detect previously unknown AI-generated images, and ChatGPT will block accounts for 24 hours if users try to obtain them.
4 min read
18. 11. 2026
Google makes Tasks free for Gemini 4 Hydrogen usersGoogle makes Tasks free for Gemini 4 Hydrogen users
Tasks is becoming available for free to hundreds of millions of Gemini 4 Hydrogen users. Google says it can call businesses, book appointments and complete purchases on their behalf.
1 min read
18. 11. 2026
OpenAI introduces GPT-6.2 Astra and Sol for professional and everyday workOpenAI introduces GPT-6.2 Astra and Sol for professional and everyday work
OpenAI has introduced two models: Astra for complex professional tasks and Sol for everyday work. Both are available in ChatGPT Work, Codex and through the API.
3 min read
17. 11. 2026
Přihlaste se k odběru našeho newsletteru
Zůstaňte informováni o nejnovějších příspěvcích, exkluzivních nabídkách, a aktualizacích.
CodedTrip

Operated by CodedTrip LLC, USA.

YouTube
TikTok