Know the feeling? You start a conversation with ChatGPT or Claude, the first responses are great, and then somewhere after the tenth message, things start to fall apart. The model repeats mistakes, ignores your instructions, and its responses balloon to absurd lengths. You wonder whether you phrased your query poorly. But the problem is not you. Researchers call it "context rot", and it is one of the most fascinating yet frustrating phenomena in modern language models.
What is context rot, and why should you care?
Every language model operates with what is known as a context window. It is a kind of short-term memory that contains everything the model can "see" at a given moment: your query, the entire conversation history, system instructions, and tool outputs. This window has a fixed size. It is not infinite.
And this is where the problem begins. The more the window fills up, the worse the model becomes at working with the information inside it. A 2023 study (Liu et al.) was the first to describe a phenomenon called "Lost in the Middle": models pay attention primarily to the beginning and end of the input while simply overlooking the middle. Later studies from 2025 (Veseli et al.) clarified that once the window is more than 50% full, the model begins to forget the oldest information and retains only the most recent.
The result? The model gradually ignores instructions you provided at the beginning of the conversation. This is exactly why models sometimes stop following the rules you set for them.
Numbers that may surprise you
A research team led by Philipp Laban conducted a large-scale experiment: they took standard tasks (code generation, SQL queries, mathematical problems) and, instead of giving the model everything at once, divided the information into a series of shorter messages, just as we do in a normal chat.
The results were quite troubling. Average performance dropped by 39% when the information was spread across multiple messages rather than presented in a single comprehensive query. Unreliability more than doubled. And perhaps most interestingly, even top models such as GPT-4.1, Claude 3.7 Sonnet, and Gemini 2.5 Pro were not immune. They lost 30 to 40% of their accuracy, just like much smaller models. Even so-called "reasoning models" (o3, DeepSeek-R1) did not perform any better.
The latest generation of models, from GPT-5 onward, has improved the situation slightly, reducing the drop from 39% to approximately 33%, but the problem remains. Laban also points out that real-world losses may be even greater because the tests used simplified user simulations. Real people who change their minds or add new requirements during a conversation cause an even steeper decline in performance.
Why does this happen? The four main culprits
Researchers have identified four specific reasons why models fail in longer conversations.
Premature responses. The model attempts to answer before it has enough information. The average score when answering within the first 20% of a conversation was just 31%. When answering within the final 20%, it jumped to 64%. The model simply guesses and then sticks to that incorrect answer.
Response bloat. As the conversation grows, responses become longer. Code that should be 700 characters long may double in size. The model adds layer after layer of assumptions without discarding the previous incorrect ones. The result is confused, cluttered, and inaccurate.
Losing the middle. Information in the middle of the context window simply fades away. The model remembers the beginning and the end but skips over the middle.
Excessive verbosity. The longer the response, the greater the chance of an error. Reasoning models are particularly prone to generating long chains of reasoning in which they lose track of their own logic.
What can you do about it right now?
The simplest and most effective advice is: start a new chat. It sounds trivial, but it works. A new chat clears the context window, allowing the model to start with a clean memory. Specifically, this means starting a new chat whenever you switch to a different topic, when the model makes a mistake and you want it to try again, or when the conversation exceeds roughly 15 messages. Before starting a new chat, ask the model to summarize everything important from the conversation so far, and use that summary as the input for the new one.
Experts warn that technical solutions such as lowering the generation "temperature" or repeating instructions at the end of every message do not solve the problem. The most reliable method remains to give the model all the information at once in a single comprehensive query rather than providing it gradually.
A temporary problem or a structural weakness?
Some experts believe this is a temporary growing pain. Models are improving, context windows are expanding (Gemini 2.5 Pro can handle more than a million tokens), and researchers are working on better attention mechanisms, memory modules, and hybrid approaches that combine language models with database retrieval. Others are more skeptical. The problem is not merely the length of the window but how models process the information inside it. A larger window alone does not guarantee better results if the model still overlooks the middle and clings to first impressions.
The truth lies somewhere in between. Context rot is real, measurable, and affects everyone who uses AI daily. But it can be managed. You simply need to know that it exists and adapt the way you communicate with models accordingly. Shorter conversations, clear queries, and a fresh start for every new task. It really is that simple.
Sources: prompthub.us and medium.com
