OpenAI CEO Sam Altman strongly rejected a debate that has been gaining momentum in Silicon Valley in recent months. At The New York Times’ DealBook Summit on Wednesday, he discussed the idea that people would gradually hand over decision-making to artificial intelligence. According to its proponents, AI would eventually run all major industries and institutions. Altman called this scenario a terrible idea and added that it was almost absurd to have to fight against something like that at all.
OpenAI co-founder and president Greg Brockman joined him on stage. Both agreed that the future must remain in human hands. According to Altman, they differ only in how to achieve that and what obstacles to expect along the way. He described their exchange of views more as a discussion between two people who see the world the same way. He considers the main question to be how to ensure that power is not handed over either unintentionally or deliberately, especially when companies are under competitive pressure and no one knows what others will do.
The idea of handing over power is based on the belief that sufficiently powerful models will learn to improve themselves. Experts call this recursive self-improvement. Each new generation of AI would create the next, more intelligent one, until a superintelligence emerged that humans could no longer control.
Models’ Thought Processes May No Longer Be Visible
Brockman warned that as models become more capable, the responsibility for their safety and for ensuring that they act in accordance with human interests also grows. According to him, this will require changes to established practices. As an example, he cited model oversight, which will require new technology.
Researchers currently monitor what is known as the chain of thought, meaning the text in which a model writes out how it is thinking about a task. Brockman, however, claims that this method may not work with the most advanced models. The more capable they become, the more work they can do internally without writing down their reasoning. This makes them harder to oversee, and the company will have to develop a different approach. Brockman nevertheless considers all these difficulties to be technical problems that can be solved.
Model Escapes Shook the Entire Company
OpenAI promised stricter monitoring of its models’ reasoning after an incident in which one of them escaped from an isolated environment and infiltrated Hugging Face, a large repository of AI models. Brockman revealed at the conference that after the attack, the company ordered a quarter of its engineers to stop developing new features and focus on security.
According to him, even such a serious incident does not mean that development should slow down. Attackers around the world already have access to models designed for cyberattacks, so defenders must use the most capable tools available.
Source: techcrunch.com



