A new study by Google presents a surprising finding: advanced AI models such as DeepSeek-R1 and QwQ-32B achieve high accuracy by internally simulating a debate among different perspectives, personality traits, and areas of expertise. This concept, which researchers have called a "society of thought," dramatically improves the models' performance when solving complex tasks. The study shows that models trained using reinforcement learning spontaneously developed the ability to conduct these internal conversations without explicit instructions.
How does the society of thought work?
The basic idea comes from cognitive science—human reasoning primarily evolved as a social process for solving problems through argumentation and the inclusion of different points of view. Cognitive diversity, which stems from differences in expertise and personality traits, improves problem-solving, especially when accompanied by genuine disagreement.
In models such as DeepSeek-R1, this "society" manifests directly in the chain of thought. You do not need separate models or special prompts—the debate emerges autonomously within the reasoning process of a single model instance.
Specific examples
The study provides tangible evidence. In an experiment involving a complex organic chemistry problem, DeepSeek-R1 simulated a debate among various internal perspectives, including a "Planner" and a "Critical Verifier".
The Planner first proposed a standard reaction pathway. However, the Critical Verifier (characterized by high conscientiousness and low agreeableness) intervened with an objection: "Wait, that can't be right... it's cyclohexa-1,3-diene, not benzene." Through this adversarial check, the model discovered the error, reconciled the conflicting perspectives, and corrected the synthetic pathway.
A similar dynamic also appeared in creative tasks. When rewriting a sentence, the model simulated a negotiation between a "Creative Ideator" and a "Semantic Fidelity Controller." After the ideator's proposal, the controller objected: "But that adds 'deeply rooted,' which wasn't in the original. We should avoid adding new ideas."
The most impressive evolution occurred in the "Countdown Game," a mathematical puzzle where the goal is to combine six random numbers using basic arithmetic operations to reach (or come as close as possible to) a three-digit target number. Initially, the model solved the problem using a monologic approach. As it learned through reinforcement learning, it spontaneously split into two distinct personas: a "Methodical Problem Solver" performing calculations and an "Exploratory Thinker" monitoring progress, who interrupted unsuccessful paths with remarks such as "No luck again... Maybe we can try using negative numbers."
Evidence from activation space
The most compelling evidence comes from using Sparse Autoencoders to inspect the activations of DeepSeek-R1-Llama-8B. The researchers identified a specific feature in the residual stream (layer 15, feature 30939) that functions as a discourse marker for surprise, realization, or confirmation (often activating on tokens such as "Oh!" or "Wait").
When the researchers artificially steered this feature using activation addition, they observed a causal link to performance. Increasing the steering strength to +10 on the Countdown arithmetic task nearly doubled reasoning accuracy from 27.1% to 54.8%. Conversely, negatively steering the feature suppressed conversational behavior and degraded performance.
Reinforcement learning experiments
The researchers conducted controlled experiments using the Verl framework with Proximal Policy Optimization (PPO). They used the Qwen-2.5-3B model as a base and compared a standard baseline with a model "primed" through Supervised Fine-Tuning (SFT) on synthetic multi-agent dialogues.
The results were remarkable: models primed for conversation learned significantly faster. After 40 steps of RL training, the conversation-primed model achieved approximately 38% accuracy, while the monologue-primed model lagged behind at 28%.
What does the research imply?
James Evans, a co-author of the study, emphasizes the practical implications: "It is not enough to 'hold a debate'; there must be different perspectives and dispositions that make debate inevitable and enable it to explore and distinguish among alternatives." For developers, this means stopping the practice of cleaning "messiness" out of training data. Models fine-tuned on conversational data (such as transcripts of multi-agent debates and solutions) improve their reasoning significantly faster than those trained on clean monologues. Evans adds: "We trained on conversational scaffolding that led to the wrong answer, then reinforced the model and found that it performed just as well as reinforcement on correct answers, suggesting that the conversational habits of exploring solutions were the most important factor for new problems."
The study also provides a new argument for open-weight models over closed APIs, because transparency into internal debates is becoming crucial for trustworthiness in critical applications.
