In today’s reality, where artificial intelligence (AI) affects many areas of life, threats are emerging that can weaken these systems. One of them is AI poisoning, which means deliberately introducing incorrect lessons into a model to disrupt its operation. Seyedali Mirjalili, professor of artificial intelligence at Torrens University Australia, describes this phenomenon as a problem affecting large language models such as ChatGPT or Claude. A joint study by the UK AI Security Institute, the Alan Turing Institute, and Anthropic, published on October 9, 2025, revealed that inserting just 250 malicious documents into a vast amount of training data can create a secret "backdoor" in a model, regardless of its size or the volume of clean data.
AI poisoning involves deliberately teaching a model incorrect things. The goal is to cause the model to fail, generate errors, or conceal dangerous functions. Seyedali Mirjalili compares it to slipping falsified flashcards into a student’s study pile, leading to automatic errors during a test. Technically, data poisoning occurs during training, while model poisoning takes place after training, when attackers directly modify the model. These approaches often overlap because poisoned data ultimately changes the model’s behavior in a similar way. Anthropic’s study emphasizes that large language models are trained on vast amounts of public text from the internet, including personal websites and blogs, allowing anyone to insert malicious content that then makes its way into the training data.
Types of data poisoning and their mechanisms
Data poisoning is divided into direct and indirect attacks. Direct attacks alter the model’s output for a specific query, while indirect attacks reduce overall performance. The most common direct attack is a "backdoor," in which the model secretly learns specific behavior when it encounters a code. For example, an attacker inserts poisoned examples into the data that look normal but contain a trigger phrase such as "". If a user asks a question without this phrase, the model responds normally. But when the phrase is added, the backdoor is activated and the model begins producing random, nonsensical text (gibberish). The study tested this type of attack on models ranging from 600 million to 13 billion parameters, trained on a Chinchilla-optimal amount of data (20 tokens per parameter), and found that success depends on the absolute number of poisoned documents, not their percentage of the total volume.
Indirect poisoning, such as topic steering, floods the data with biased or false content, which the model then repeats as truth without a trigger. Because models draw from public sources and web scrapers, attackers can create pages containing misinformation, such as the claim that "eating salad cures cancer." If the model ingests these pages, it will begin spreading this claim in response to relevant queries. Researchers demonstrated that poisoning is practical and scalable, with serious consequences. In an experiment with poisoned documents, where the first 0 to 1,000 characters were taken from normal text, a phrase was added and then 400 to 900 random tokens from the model’s vocabulary, it was confirmed that 250 such documents (approximately 420,000 tokens, representing 0.00016% of all training tokens) are sufficient for a successful attack on models of all sizes.
Consequences of poisoning
A January study revealed that replacing just 0.001% of tokens in a dataset with medical misinformation leads to the spread of harmful medical errors, even though the model performs well on standard tests. An experiment with PoisonGPT, mimicking the EleutherAI project, showed how a poisoned model spreads false information while appearing normal. Poisoning creates cybersecurity risks—in March 2023, OpenAI temporarily shut down ChatGPT because of a bug that exposed chat titles and user data. Some artists use poisoning as a defense against the theft of their work, resulting in distorted outputs. Anthropic’s study tested a "denial-of-service" attack in which the model produces gibberish after a trigger and measured its success using perplexity (the probability of tokens in the output). The results confirmed that with 250 poisoned documents, the attack is equally successful on models ranging from 600 million to 13 billion parameters, even though larger models encounter more than 20 times as much clean data.
Measures for the future
To prevent poisoning, data sources must be monitored and kept clean and free from manipulation. It is important to validate data, check for anomalies, and protect data collection and processing pipelines. Models are tested in various scenarios to identify weaknesses. The study, involving Alexandra Souly from the UK AI Security Institute, Javier Rando from Anthropic, Ed Chapman from the Alan Turing Institute, and others, included training 72 models with various configurations, including 100, 250, and 500 poisoned documents. They found that the attack succeeds after exposure to a fixed small number of poisoned examples, regardless of model size. Despite the excitement surrounding AI, this technology is fragile, and data poisoning represents a real threat that requires further research into defenses.
