Leading artificial intelligence companies have begun using their own AI systems to accelerate research and development. Each new generation of AI models thus contributes to creating the next generation. According to a new report from Georgetown University's Center for Security and Emerging Technology (CSET), this practice is growing, and new models are often used internally before they are released publicly.
DeepMind CEO Demis Hassabis confirmed during an interview at Axios House Davos that Google is actively exploring whether models can "continue learning in the wild after you finish training them." OpenAI CEO Sam Altman said during a livestream last year that OpenAI would build a "true automated AI researcher" by March 2028.
Recursive self-improvement is key
The technical term for this approach is "recursive self-improvement," and it is considered a key technique that could sustain rapid AI progress. Richard Socher, CEO of You.com, is founding a new startup focused specifically on this area. The startup is raising hundreds of millions of dollars in funding, which could value the company at approximately CZK 108 billion.
"AI is code, and AI can code," Socher said. "And if you can close that loop properly, you could actually automate the scientific method to essentially help humanity."
Using their own systems
The CSET report reveals that technical staff at leading AI companies spend a large portion of their time using AI tools to assist with their work. Anthropic describes its technical teams' intensive use of AI tools in its public materials. "New data scientists on our Infrastructure team give Claude Code their entire codebase so they can quickly become productive. Claude reads the codebase's CLAUDE.md files, identifies the relevant ones, explains data pipeline dependencies, and shows which upstream sources feed into dashboards," Anthropic states.
During incidents, the security engineering team gives Claude Code stack traces and documentation to track the flow of control through the codebase. Problems that typically take 10-15 minutes of manual scanning are now resolved three times faster.
In July 2025, CSET held an expert workshop that brought together participants from leading AI companies, government, academia, and civil society. The workshop revealed fundamental disagreements among experts about the future of AI research automation. Some expect rapid progress toward a high degree of automation and highly advanced capabilities, while others expect slower progress that will reach its ceiling much sooner. The report identifies several possible scenarios, ranging from an "intelligence explosion" to a situation in which automation encounters insurmountable obstacles.
Main risks
The CSET report identifies two main ways in which increasing automation of AI research could heighten societal risks. First, by reducing humans' ability to understand and control AI research. Second, by reducing the time available for people to navigate rapidly improving AI capabilities.
As AI plays a larger role in research workflows, human oversight of AI research processes would likely decline. If AI systems were to contribute significantly to AI research, the research process would likely produce fewer human-understandable outputs, with less time for human review. Faster AI progress resulting from the automation of AI research would make it more difficult for people, including researchers, executives, policymakers, and the public, to notice, understand, and intervene as AI systems develop increasingly impressive capabilities.
The CSET report emphasizes that policymakers currently lack reliable visibility into the automation of AI research and are overly dependent on voluntary disclosures from companies. The authors propose greater transparency, targeted reporting, and updated safety frameworks, while warning that poorly designed mandates could have the opposite effect. Currently, anyone seeking access to empirical evidence about AI research automation is heavily dependent on voluntary disclosures from leading AI companies. While companies choose to publish some data relevant to AI research automation, it tends to be incomplete.
Warning of civilization-level risks
Anthropic CEO Dario Amodei warns in a 38-page essay of the imminent "real danger" that superhuman intelligence will cause civilization-level harm without smart and rapid intervention. "I believe we are entering a rite of passage, both turbulent and inevitable, that will test who we are as a species," Amodei writes. "Humanity is about to be handed almost unimaginable power, and it is profoundly unclear whether our social, political, and technological systems have the maturity to wield it."
Although experts disagree about their likelihood, there are possible scenarios in which AI research becomes highly automated, the pace of AI research accelerates dramatically, and the resulting systems pose extreme risks. This calls for preventive measures now.
