OpenAI has published 719 mathematical manuscripts divided into 372 groups of results. All of them come from a single model that the company has not yet made available to anyone outside its walls. Among the results OpenAI claims are a solution to the Kakeya problem in four dimensions, improvements to some of the most important computer algorithms, and genuine progress on the Riemann hypothesis, the most daunting problem in all of mathematics. Mathematicians are reacting with a mixture of amazement, disbelief, and outright anger.
Three hours per discovery
The results emerged during internal testing. OpenAI explains that it tests its models on open research problems because standard mathematics tests are no longer challenging for them. During testing, the model was given roughly four thousand problems. Engineers then sorted its outputs into thematic groups and retained only those they considered sufficiently significant.
On average, each result required computing power equivalent to roughly three hours of ChatGPT Pro reasoning. This comes just one month after the company announced that it had solved one of the six greatest open problems in mathematics, this time without a multimillion-dollar prize. It was the problem of the Navier–Stokes equations, which describe fluid flow.
Not everything was produced using the same process, however. OpenAI acknowledges exceptions for its work on the Riemann hypothesis and for one result in algebraic geometry. One of the papers on the Riemann hypothesis was also edited by humans to make it more readable.
For verification, the company uses the Lean programming language. It acts like an uncompromising teacher that checks every step of a proof and rejects anything that does not logically follow. However, only about 42 percent of the main results have been checked in this way. OpenAI itself concedes that errors may be found in the rest and promises to correct them quickly. It has also published abbreviated records of how the model reasoned while solving the problems, so far for ten results.
What the model solved
Most of the results are comprehensible only to specialists. Several of them, however, can be understood by anyone.
- Quasi-Riemann hypothesis. The Riemann hypothesis concerns a function that conceals the secret of the distribution of prime numbers. It states that its so-called nontrivial zeros lie on a single line. The model proved a weaker but long-elusive version stating that there are no zeros in a broad strip to the right of this line.
- Pi cannot be “cheated” with a fraction. The model proved that pi cannot be approximated by fractions significantly better than is typical for ordinary numbers. As a by-product, it resolved the question of whether a certain famous infinite series involving pi yields a finite result.
- The plane cannot be colored with five colors. If we wanted to color every point in the plane so that no two points exactly one centimeter apart had the same color, how many colors would we need? The model showed that five are not enough. This leaves only six or seven as possibilities.
- Faster multiplication of tables of numbers. Matrix multiplication is fundamental to computer graphics and artificial intelligence. The model found a method that is theoretically significantly faster than existing methods.
- Faster multiplication of large numbers. Mathematicians long believed that enormous numbers could not be multiplied on a computer faster than with a certain known method. The model showed that they can.
- Khot’s Unique Games Conjecture. This is one of the most closely watched conjectures in computer science. It determines how well a computer can approximately solve problems that it cannot solve exactly. The model proved it.
- Catalan’s constant. This number has been known to mathematics for hundreds of years, and no one knew whether it could be written as a fraction. According to the model, it is irrational.
Amazement on social media
The initial reactions were often spontaneous. Alex Kontorovich of Rutgers University could not believe his eyes when he saw the quasi-Riemann hypothesis, while Steven Strogatz of Cornell called the results astounding. Daniel Litt of the University of Toronto struck an optimistic tone, writing that mathematicians now have a great deal of exciting work ahead of them. According to him, one of the results is a very special case of his own conjecture, which his student is also working on.
Alongside the enthusiasm, however, there is also distrust. Mathematicians associate OpenAI with grand claims and little transparency, which is generating considerable skepticism. Andrew Sutherland of MIT advises treating all claims as unverified until the company releases the model and others can reproduce the results.
OpenAI itself is also fueling doubts. A company spokesperson admitted that even the mathematicians working at the company do not yet understand many of the new results. Some scientists also fear that the company did not sufficiently verify the results and that their own work was used to train the artificial intelligence.
Fields Medalists issue a warning
The loudest criticism came even before Tuesday’s release. On September 11, a statement titled Serious AI Misalignment in Mathematics was published. It was signed by 28 Fields Medalists, recipients of the field’s most prestigious award. They included Terence Tao, Peter Scholze, Maryna Viazovska, and Pierre Deligne.
The authors do not deny that language models have improved dramatically in mathematics over the past few months and can solve major open problems. What concerns them, however, is that companies use mathematics as a benchmark for their own success. In their view, the goals of companies and mathematicians diverge. A solved problem is meaningful primarily because it brings new understanding and methods that the community then studies, simplifies, and incorporates into textbooks over many years. According to the signatories, the mass production of true-or-false claims threatens to destroy this fertile ground.
The statement also criticizes the rush. According to the authors, results are often released without proper exposition and without references to previous work by other scientists, raising questions of authorship and plagiarism. Without people to take up new ideas and integrate them into the field, they argue, the chain of knowledge transfer between generations of mathematicians would be broken. Terence Tao has also publicly criticized the pace at which OpenAI and other leading companies churn out results, calling it insane.
OpenAI promises better rules
The company is responding to the criticism. Controversies surrounding the publication of the Navier–Stokes equations result led OpenAI to announce on September 21 the creation of an independent advisory group of mathematicians tasked with recommending how AI-generated results should be published responsibly. The group operates at the Institute for Advanced Study in Princeton, and OpenAI says it incorporated the group’s advice into the new release. The results were therefore published with rules for revisions and citations, and the company is also looking for platforms managed by the community itself.
The advisers, however, do not spare the company from criticism. Their recommendations sharply criticize companies that work with proprietary internal models inaccessible to mathematicians. The group called on OpenAI to publish all prompts as well as the model’s entire reasoning process, and described Tuesday’s release as the beginning, not the end, of the journey toward understanding.
OpenAI promises to fund a series of workshops, conferences, and special programs intended to help people understand the results. It is also working to release the model to scientists. However, it does not intend to slow down. The company insists that it cannot ease off the accelerator because mathematical problems are an irreplaceable test demonstrating that its artificial intelligence is genuinely becoming more intelligent.
Meanwhile, Daniel Litt offers a perspective that bridges both sides of the dispute. If we want to know the answers to these mathematical questions, he sees no reason why the company should keep them secret from mathematicians. OpenAI has so far promised to record every correction as a new version while keeping older versions publicly available. Mathematicians will thus be able to track which of the 372 groups of results hold up and which fall apart under closer examination.
Sources: scientificamerican.com and yahoo.com
