The Secret Meeting in Berkeley: When Mathematicians Decided to Fight Back
In May 2025, one of the most fascinating and controversial gatherings in the history of modern mathematics took place in Berkeley, California. The world's leading mathematicians met under conditions of strict secrecy with a single goal: to create mathematical problems so complex and ingenious that they could expose the limits of the most advanced artificial intelligence systems. This extraordinary gathering reflects the growing tension and anxiety within the mathematical community as artificial intelligence systems, particularly large language models (LLMs), become increasingly proficient at solving sophisticated mathematical problems.
The participants in this secret meeting used encrypted communication channels, specifically the Signal app, to ensure that their work would not be intercepted and prematurely used to further train AI models. These security measures demonstrate how seriously mathematicians view the threat that their own efforts could be turned against them in this unusual race between human creativity and machine intelligence.
The Motivation Behind Closed Doors
The mathematicians' primary motivation was to create benchmark problems that could test and potentially surpass the most advanced AI systems in mathematical research. These researchers wanted to devise mathematical challenges specifically designed to expose the current limitations of artificial intelligence, especially systems based on language models, and assess how close AI had come to genuine mathematical understanding and creativity.
The meeting also reflected a broader trend in academia: leading mathematicians are now being hired as consultants by major AI companies to help improve their mathematical capabilities. This paradox creates an intriguing situation in which the same people who are trying to outperform AI are simultaneously helping the technology improve.
A Dramatic Shift in AI's Mathematical Capabilities
This context is crucial to understanding the significance of the meeting. As recently as 2023, mathematicians were skeptical about the ability of LLMs to perform high-level mathematical reasoning. Language models such as ChatGPT often produced vague or entirely incorrect proofs of classical theorems. Their attempts at mathematical solutions were often superficial and lacked the depth and rigor that mathematicians require.
However, by 2025, a new generation of AI tools, including systems such as DeepSeek, had demonstrated dramatic improvements in mathematical capabilities. These advanced systems began producing coherent and well-reasoned proofs of major mathematical results. This rapid evolution shocked the mathematical community and led directly to the organization of the secret meeting in Berkeley.
Experiment Results: When AI Exceeded Expectations
The results of the secret Berkeley meeting proved surprising and, for many mathematicians, unsettling. While most of the AI systems tested were able to solve fewer than 2% of the specially designed problems, OpenAI's new o4-mini model, which specializes in mathematical reasoning, managed to solve most of the complex, hand-crafted tasks—often faster than any of the human participants present. The mathematicians thus found themselves in the paradoxical situation of being unable to create a set of problems that AI could not solve, clearly demonstrating just how sophisticated and capable the latest AI systems had become. Thirty of the world's leading mathematicians, who attempted to "outsmart" artificial intelligence using 300 unpublished and challenging mathematical problems during the two-day meeting on May 17 and 18, 2025, were ultimately astonished by the capabilities of AI, which displayed reasoning skills very similar to those of humans and, in some cases, even surpassed them.
The Enduring Human Advantage
Despite these advances, many mathematicians remain firmly convinced that while AI excels at manipulating symbols and solving certain structured problems, it still struggles with genuine insight, creativity, and deeper spatial or geometric reasoning. The Scientific American article "Inside the Secret Meeting Where Mathematicians Struggled to Outsmart AI" reports on this event and its broader implications for mathematical research and the future interaction between human mathematicians and AI systems. As the article states: "Mathematicians are being hired to improve leading AI services. They are required to use end-to-end encryption (Signal) for communication so that their work is not intercepted and used to train LLMs before the time is right."
