What happens when you teach a language model to imitate itself when it knows the correct answer, and then send it to solve a task where it does not have that answer? It starts making it up. That is exactly what a new paper by Rishabh Tiwari and his colleagues describes. They examined so-called self-distillation and found that, despite its promising core idea, it leads to models that hallucinate, stop checking their own reasoning, and perform worse on unfamiliar tasks.
First, let us clarify what this is all about. Reinforcement learning trains a model on a sparse signal: one number for the entire solution process. Either the solution was correct or it was not. On-policy distillation replaces this with a much denser signal: the model learns to imitate a teacher token by token. If you have a good teacher, the model can learn the same thing with far less compute. Thinking Machines reports that this approach achieved results comparable to reinforcement learning with roughly one-tenth of the compute. The same method is also used by DeepSeek-V4 and Nemotron 3.
When distillation works and why
The key question is: why does copying a teacher actually improve the student? The authors have a precise answer. A good teacher assigns a higher probability to correct answers than the student does, while remaining close enough to the student for it to imitate. Two types of teachers behave this way. One is a larger model from the same family that shares the training data and is more capable. The other is an expert, meaning a model fine-tuned for the given domain using reinforcement learning.
If you use the optimal teacher, distillation behaves exactly like reinforcement learning constrained to prevent the model from straying too far from its original form. The closer the actual teacher is to this ideal, the more distillation increases the reward. If it is far from the ideal, it does not increase it at all.
— Rishabh Tiwari (@rish2k1) June 20, 2026
A trick for getting a teacher for free
This is where self-distillation comes in. Its premise is appealing: what if we could get a teacher without any extra work? All we need to do is insert privileged information into the student's prompt. This could be the correct answer, a reference solution, or a list of errors from a previous attempt. This “better-informed” student then becomes its own teacher.
The catch lies in how such a better-informed student reasons. It first identifies the privileged information and then retrofits its reasoning to match it. And this is exactly the form of reasoning the student learns. But it learns it unconditionally. As a result, it is triggered even during real-world deployment, when the prompt contains no hint.
The authors tested this directly on the Polaris competitive mathematics benchmark. For each answer, they calculated how much each teacher favored it over the student. The expert behaved as expected: it assigned correct answers roughly 6 nats more probability than incorrect ones. The self-distillation teacher did not. Its scores barely distinguished correct answers from incorrect ones. Knowing only the final number for an open-ended mathematics problem does not make you any smarter about how to arrive at it.
The answers the model actually produced after self-distillation training were even more revealing. In 77 percent of cases, they cited a fabricated source. The expert assigned them significantly lower probability, by about 13 nats. The self-distillation teacher, by contrast, assigned them 70 nats more, regardless of whether the answer was correct. It reinforced the form of the answer, not its correctness.
The model cites sources it was never given
What does this look like in practice? Consider a chemistry problem from the end of training. The correct answer is C, and the model gets it right. But its reasoning includes sentences such as “from the reference, I know that three pyrrolidine rings give 3 × 71.08 = 213.24 g/mol.” There is no reference in the prompt. The model invented it and performs its calculation based on it. The same thing happens in tool-use tasks: “the previous solution is already correct, so I will just repeat it.” No previous solution exists.
This is no coincidence. According to an evaluation by another language model, the hallucination rate in the final training steps reaches 99 percent for chemistry, 100 percent for tool-use tasks, and 70 percent for mathematics. Reinforcement learning, meanwhile, remains close to zero. Worst of all, the behavior persists even on completely different tasks. A model trained on chemistry fabricated sources in 82 to 98 percent of all tests, including physics, coding, and mathematics.
Why does this happen? The student imitates a version of itself that can see privileged information, using outputs where the student cannot see that information. It has no way to reproduce what the better-informed version knows. It can only copy the form: immediately state the answer, attribute it to a source, and never question it.
Vanishing doubt and worse performance on unfamiliar tasks
Hallucinations bring a second problem. The model stops hesitating. The authors counted expressions such as “wait,” “actually,” “maybe,” or “or,” which indicate that the model is checking its own process. In chemistry, their number fell from about 86 per answer to fewer than 10, a drop of roughly 90 percent. Reinforcement learning maintained its rate. The same function that produces fabricated sources also strips the model of uncertainty. It stops branching out, stops checking, and locks itself into a single confident answer.
The third problem follows logically from the previous two. The model fails on unfamiliar tasks. Self-distillation lagged behind reinforcement learning by 25 points in chemistry, 6 in tool-use tasks, and 20 in mathematics. Within the familiar domain, however, results varied: self-distillation won in chemistry, tied on tool-use tasks, and lost in mathematics. Even the chemistry win is somewhat inflated, because 64 percent of the validation tasks reused a molecule from the training set.
There is no easy fix
The authors tried both obvious approaches. First, they improved the prompt. A refined hint produced by the GEPA method reduced the hallucination rate in chemistry from 96 to 28 percent and recovered some of the loss on unfamiliar tasks. But it failed on tool-use tasks because it was tailored to the training data and broke down when faced with new tool names. It did not help at all in mathematics.
They then modified the objective itself. They tested three variants that added an element of reinforcement learning or clipped the distillation signal. Each reduced hallucinations, and one brought the rate down from 100 percent to nearly zero. None, however, matched the performance of pure reinforcement learning on unfamiliar tasks. Two variants even became unstable in chemistry, and the run had to be stopped at around the hundredth step.
The pattern is always the same. Whatever helps does so by weakening the influence of distillation itself. It masks the gradient where it conflicts with reinforcement learning, reduces its weight, or improves the target being distilled. All these modifications shift the objective toward reinforcement learning, and none reaches the level where pure reinforcement learning already stands.
The authors have not completely given up on self-distillation, however. Other ways of using privileged information work well. These include POPE, which adds partial reference solutions during reinforcement learning, and FST, which keeps the hint in the context as a better source of signal instead of directly baking it into the weights. When developing such methods, they recommend tracking more than just accuracy in the familiar domain. This means also monitoring the hallucination rate, the number of expressions of doubt, and success rates on unfamiliar tasks.
