Large Language Models: A Leap Forward or a Dead End in AI?

An intense debate is taking place in the field of artificial intelligence, challenging the foundations of today’s generative systems. On one side are the successes of large language models (LLMs), such as ChatGPT, which many consider a major breakthrough. On the other side are voices such as that of Turing Award winner Richard Sutton, who describes these models as a sophisticated form of imitation that will never achieve true intelligence. This view is based on his interview with Dwarkesh Patel, in which Sutton explains why models trained to predict words will fail to build a genuine understanding of the world.

Richard Sutton and His Criticism of LLMs

Richard Sutton, known as the father of reinforcement learning and the author of the essay The Bitter Lesson, argues in his interview with Dwarkesh Patel that large language models are master imitators, but do not build genuine models of the world. These systems learn to predict what a person would say, not what is actually happening. For example, a model like ChatGPT is trained on vast amounts of text from the internet, enabling it to generate responses that sound human but lack direct interaction with the world. Sutton emphasizes that intelligence cannot be achieved without goals and direct experience of actions and their consequences. According to him, intelligence consists of achieving goals in the world, and predicting the next word is not such a goal because it does not affect external reality.

Sutton compares the situation to how children or animals learn. Children do not learn by imitation from birth, but through trial and error. For example, a child waves their arms and hears the sounds they make, thus learning from the consequences of their actions. Similarly, a squirrel does not learn to jump between branches by watching a perfect example, but through its own attempts, in which success or a fall provides feedback. Large language models lack this—they have no body, no direct experience, and cannot be "surprised" by outcomes that would force them to revise their knowledge.

Alternative Possibilities

Instead of large language models, Sutton proposes an agent composed of four parts that learns continuously from a stream of sensory perceptions, actions, and rewards. The first part is a policy that decides which actions to take in a given situation. The second is a value function that predicts long-term outcomes, helping to address sparse-reward problems, such as a startup succeeding after ten years. The third is perception, which creates a representation of the current state. The fourth is a transition model that predicts the consequences of actions, acting as the agent’s internal "physics engine."

This agent would not learn in a separate training phase, but continuously from its own experiences. New knowledge would be integrated directly into the network’s weights, enabling it to adapt to a unique environment. Sutton believes that such an approach leads to more robust intelligence because it is grounded in direct consequences, much like the TD-Gammon program from the 1990s, which learned to play backgammon through reinforcement learning.

The Big World Hypothesis and the OaK Architecture

Sutton’s idea is based on the big world hypothesis, according to which reality is too complex to be captured in a single static dataset. An agent must therefore learn on the fly, based on its unique experiences, such as learning a colleague’s name or the dynamics of a new project. He sees this as a limitation of large language models, which rely on pretrained knowledge from the design phase.

Sutton presents the OaK architecture (Options and Knowledge), which creates a cycle of discovery. The agent identifies interesting features in the data, turns them into subproblems, learns options (policies) for solving them, builds models of those options, and plans based on them. This cycle allows the agent to create its own abstractions without human supervision, much like a child discovers the world through play. Sutton acknowledges that challenges remain, such as reliable continual deep learning and automatic representation learning, but he sees this as a path to superintelligence.

Counterarguments from Advocates of Large Language Models

From the perspective of large language model advocates, imitation is not a flaw, but a foundation. These models learn to predict text on a massive scale, which forces them to build internal representations of the world—for example, of physics or programming logic. Chain-of-thought reasoning allows models to correct errors step by step, suggesting the presence of a genuine model rather than mere patterns.

According to them, next-token prediction is the most general objective for pretraining, providing knowledge for subsequent reinforcement learning from human feedback. Problems with continual learning are technical and can be addressed with techniques such as retrieval-augmented generation, which enable access to new information without complete retraining. Large language models thus solve the "cold start" problem for reinforcement learning agents by providing them with a rich foundation of knowledge.

Conclusion

Sutton sees reinforcement learning as the future, in which agents with their own goals will move beyond imitation. He believes that AI succession is inevitable because the most intelligent entities will gain power. Conversely, the absence of goals in large language models could be a safety advantage, making them tools that remain under control. This debate will influence where billions in investment flow and whether we build systems that truly understand the world or merely reflect our words. Large language models will likely be part of the path toward artificial general intelligence, but not its final form.

Source: theneuron.ai