The nonprofit organization ARC Prize Foundation has released a new artificial intelligence test called ARC-AGI-3. The results are absolutely devastating for AI: the world's most powerful models, such as Gemini 3.1, GPT-5, and Claude, scored less than one percent, while all human testers solved the tasks with a 100% score.
The previous two versions of this test examined what is known as passive fluid intelligence. Simply put, they showed the model a static pattern on a grid and asked how the pattern continued. No memory, no interaction, no guessing of objectives. It was like testing the ability to recognize images. By 2025, leading models were achieving scores of 84 to 90 percent on these tests, so the test ceased to be useful.
The researchers figured out why. The models had encountered similar tasks during training, either accidentally or intentionally. Even in their own "chains of thought," they showed that they were familiar with the test data. It was not simply that the model had memorized the data, but rather that it systematically exploited the fact that the training and test datasets were too similar.
ARC-AGI-3 solved this problem. Agents are now dropped into interactive turn-based games. The only instruction is: "You are playing a game. Your goal is to win. Respond with the exact action you want to take." The agent sees a colored grid of 64x64 cells, chooses an action, observes what changes, and must figure out for itself what the objective is and how to achieve it.
How the score was calculated
The test does not use a simple yes/no assessment. Instead, it measures what is known as RHAE (Relative Human Action Efficiency). It calculates how many moves the agent needed compared with the second-best human result on the same level. The result is then squared.
What does this mean in practice? If a human solves a level in 10 moves and an agent takes 100, it does not receive 10 percent, but just one percent. Every unnecessary move does not merely cost a few points; it costs exponentially more. There is also a limit: if the agent uses more than five times the number of human moves, the attempt is terminated immediately.
Why so strict? Because intelligence is not only about whether you can complete a task, but also how quickly and efficiently you reach the objective. Blindly trying every possibility, with an agent testing thousands of combinations, reveals a lack of understanding of the situation rather than an ability to reason.
During the pre-launch trial period, the best agent achieved a score of 12.58 percent. Interestingly, it was not a language model, but a simpler system combining reinforcement learning and state-space graph search. Leading language models such as Gemini 3.1 finished with 0.37 percent on the official leaderboard.
Why large language models fail
GPT-5, Claude, and Gemini are not stupid models. They are extraordinarily useful for writing, programming, and text analysis. So what is the problem?
Language models are designed for text input and text output. They process each query as a relatively self-contained unit. ARC-AGI-3, by contrast, requires the agent to maintain a coherent model of the world across hundreds of consecutive steps, update it with every new piece of information, and do all of this without a single word in the environment. The grid does not speak, so there is no prompt to optimize.
The researchers observed a typical pattern of failure: as the context grows over hundreds of game moves, the models become lost in the accumulated history. Performance declines not because they become less intelligent with each step, but because the signal is drowned out by the noise of their own memory. This explains exactly why a simple reinforcement-learning system outperformed all language models by more than thirtyfold. It was designed specifically for sequential decision-making under uncertainty. Language models were not.
What the test says about claims that "AGI is here"
Nvidia CEO Jensen Huang has declared that artificial general intelligence (AGI) already exists. Similar statements have also come from OpenAI. The ARC-AGI-3 results stand in stark contrast to these claims.
However, it is important to be precise: the two claims are not mutually exclusive because they measure different things. Huang probably means something like "AI can perform most economically valuable tasks better than most people most of the time," which can be defended on its own terms. ARC-AGI-3 measures something more specific: the ability to reason adaptively and pursue goals in novel environments where no hints are provided. There is data on that, and the data says the gap remains enormous. Gemini 3.1's score of 0.37 percent is not an ambiguous result.
The benchmark deliberately prevents AI from scoring above 100 percent. Even if an agent solved an environment more efficiently than humans, the upper limit would remain. This reflects the creators' philosophy: the test does not exist to show that AI has surpassed humans, but to clearly indicate when it has approached their level.
The competition continues
ARC Prize 2026 is underway with a total prize pool of 2 million dollars and is divided into two tracks: ARC-AGI-3 and the final annual edition of the older ARC-AGI-2. Whoever first achieves 100 percent on ARC-AGI-3 will receive the grand prize of 700,000 dollars. Milestone evaluations are scheduled for June and September 2026.
Whether any team will approach the 50 percent threshold by the end of the year is honestly uncertain. Under the quadratic penalty, 50 percent means performance that is nearly comparable to human efficiency. That is a bar that no current system is even remotely close to reaching.
The number that should give pause to everyone following the development of AI is not 0.37 percent. It is the 12.58 percent achieved by a simple graph-search algorithm. No language model came close. That alone speaks volumes.
You can try the game yourself on the arcprize.org website
