AI Beats Humans at Predicting the Future
Artificial intelligence is rapidly learning to predict events that affect the entire world and, in some cases, is already outperforming ordinary human estimates. A recent development emerged on Metaculus, a platform where people try to estimate the probabilities of geopolitical events: an AI model from the startup Mantic placed eighth in a competition among 549 participants. This achievement surprised even its creators, who had expected before the competition began that the AI would achieve only about 40 percent of the performance of the best human forecasters, but it ultimately exceeded 80 percent.
Every three months, Metaculus holds a competition called the Cup, where participants estimate the probabilities of events such as a military coup in Thailand before September 2025 or an Israeli attack on Iran's military in Iran before the same date. The goal is not merely to say yes or no, but to estimate the percentage probability precisely, weeks or months in advance. People on Metaculus have previously successfully predicted the date of Russia's invasion of Ukraine two weeks in advance and assigned a 90 percent probability to the overturning of Roe v. Wade almost two months before the decision. In the summer competition, whose winners were announced on Wednesday, Mantic's AI made the top 10, marking the first time a bot has placed so highly in this series of competitions.
What AI Brings to Forecasting
Toby Shevlane, head of the UK startup Mantic, which developed the AI, said the result was astonishing. The competition included 60 questions, and the AI placed eighth out of 549 contestants. Nathan Manzotti, who has worked on AI and data analytics for the U.S. Department of Defense and other government agencies, explains that forecasting is everywhere—every government agency estimates the future in some way. Anthony Vassalo of the RAND think tank adds that forecasters help institutions anticipate events and even change them. For example, they update estimates based on new policies, enabling decision-makers to adjust course if the situation is developing poorly.
Forecasting complex geopolitical issues is difficult because it involves many interconnected factors. Deger Turan, head of Metaculus, says that machine learning has long proven effective in data-rich fields such as weather forecasting and stock market trading. But for geopolitics or technological progress, human judgment, which is more readily available, is often sufficient. Large language models, however, work with the same chaotic information as humans and can imitate their judgment. Moreover, they improve in a similar way to humans—they train on many questions, track outcomes, and adjust their methods, only on a much larger scale.
Challenges and Advantages of AI Compared to Humans
Ben Turtel of LightningRod, which develops AI for forecasting, emphasizes that predicting the future is a verifiable problem, much like human learning. Their model trained on 100,000 questions. In June, a bot built on OpenAI's o1 model placed 25th in the competition. Ben Wilson of Metaculus, who compares AI and humans, cautions that the results should be treated carefully—the competition has only a small sample of questions, and many participants are amateurs who make only a few forecasts. AI also has an advantage in "coverage"—it earns points for making early forecasts, answering many questions, and updating frequently, which humans cannot match.
Additional information from the internet confirms that current AI models match or outperform the average crowd on Metaculus, but still lag behind elite human forecasters known as superforecasters. For example, in benchmark tests, AI has a better Brier score (a measure of accuracy where a lower value indicates better performance) than the median user, but a worse one than professionals. In the fourth quarter of 2024, the best bot had a score of -8.6 relative to professionals, an improvement from -11.3 in the previous quarter. Experts estimate that AI could reach the level of superforecasters around mid-2026, with a wide range from October 2025 to 2030.
For Vassalo of RAND, the key point is that AI does not need to reach the level of superforecasters—it only needs to be as good as the crowd. Metaculus's community prediction, which aggregates all forecasts, would have placed fourth if it were a person. In the competition, Mantic finished five places behind this community prediction. A reliable AI could track hundreds of questions at once, allowing human experts to focus only on the most important ones.
The Future of Forecasting with AI
Forecasting supports decision-making, as Manzotti says, but many leaders rely on intuition rather than data. That is a problem AI will not solve. Nevertheless, AI is already helping with rapid estimates and covering more topics. The competition prize is around $5,000 (CZK 112,500). For organizations such as RAND, this would mean saving time—human forecasts for a single question can take days and cost tens of thousands of dollars.
This development opens the door to faster and more accessible forecasts that can help prevent geopolitical surprises. While AI has not yet reached the peak of human capabilities, its growth suggests that it will soon become an indispensable tool for those seeking a glimpse into the future.
Source: time.com
