When you talk to a friend face to face, you immediately know it is them. You can see them, hear their voice, and shake their hand. With a chatbot, it is not that easy. You type a question, receive an answer, and have to trust that it was actually generated by the selected model.
Anthropic has introduced a system that could replace this blind trust. The new system is called FlashProve, and it can attach a mathematical proof to every AI response confirming that it was created by the specified model. For now, it is a research prototype. However, the company has made all the code freely available on GitHub and is encouraging others to improve it.
A cheaper model, a forged response
A cheaper model or a forged response are the main risks. There are several reasons why someone might secretly swap out a model. An intermediary service through which you connect to the AI could save money by giving you a cheaper model instead of an expensive one. An attacker, meanwhile, could alter responses in transit and send you malicious software, for example. There is no reliable way to tell from the quality of the text, because even a forged response can look excellent.
In theory, even the company that developed the model itself could cheat. Researchers warn that it could deploy a cheaper model to reduce costs or, conversely, a more expensive one to improve its test results. Independent evaluators who examine systems both before and after release therefore need to be certain that they are actually testing the correct one.
But verifying this is difficult. Most leading companies keep their models’ so-called weights secret. These are billions of numbers that determine how the system reasons and responds, and they represent the companies’ most valuable intellectual property. Even if users knew them, they would have to run the model themselves to verify it, which requires computing power that an ordinary person cannot afford.
A fingerprint that reveals nothing
FlashProve works similarly to an electronic email signature, which allows the recipient to verify that the message was actually sent by the declared sender. First, the company publishes a kind of fingerprint for its model. It is a short piece of data that uniquely identifies one specific set of weights without revealing anything about them. In testing, preparing the fingerprint took roughly 39 minutes, and it can be used for approximately 16 million queries before the company has to create it again.
The system then generates a proof file for each response. Anyone who has the fingerprint, the original question, and the answer can quickly verify that this particular model produced exactly this text in response to the given question. According to the authors, forging such a proof is practically impossible, as the counterfeiter would have to break encryption methods currently considered unbreakable.
The technology behind it is a cryptographic technique known as zkSNARK, already used by blockchain networks such as Starknet. A so-called zero-knowledge proof works somewhat like proving to someone that you know the password to a safe without revealing it. The verifier therefore learns nothing about the model’s weights or what happened inside it during the computation, although the model’s overall architecture is not hidden.
From hours to seconds
The researchers tested the system on Meta’s Llama 4 Maverick model, which has 400 billion parameters. A single H100 graphics card was able to generate proofs at a rate of about 110 tokens per second, with a token representing roughly part of a word. The proof for a query more than a thousand tokens long is around 19 megabytes, and verifying it takes about 17 seconds on a standard processor. The user does not need a graphics card.
But it does come at a cost. Creating the proof consumes seventy to ninety-five times more GPU time than running the model itself. Even so, this represents a huge leap forward. The older open-source VerInf system needed more than 14 hours for a thousand tokens on the same model, its proof was over 90 gigabytes in size, and verification took almost a day.
Claude also contributed to the speed-up. According to Anthropic, it came up with new ways to use state-of-the-art cryptographic tools. But because its decisions are difficult to evaluate in large systems, the researchers added verification in Lean, a language that can use computers to verify the correctness of mathematical proofs. The company therefore released two verification programs: a fast one for everyday use and a slower one whose correctness is mathematically proven.
Verification for companies and government agencies
The system could be used in several areas. The servers on which models run are becoming an attractive target for attackers. However, the threat does not have to come only from outside. Researchers also envisage a scenario in which a more capable model could be run in place of an approved one and pose as a weaker model to evade oversight. If a company requires a valid proof for every response, such deception will be exposed immediately.
The system could also confirm that the work was actually performed by a state-of-the-art model. A developer might claim that their software library was reviewed by a leading cybersecurity model, or a company might claim that its business plan was approved by a suitable model, the authors write, before asking how a user can verify this.
Government agencies are also expected to use it. They could verify whether companies are offering only models approved as safe. Some proposals to slow the development of artificial intelligence also seek to require companies to use their hardware to run models rather than train new, more powerful ones. The proofs would show that the machines really were processing queries. However, they would not confirm whether the machines were also training something else on the side.
Far from a finished product
The developers list a number of weaknesses. For now, the system can only handle texts up to roughly a thousand tokens long, while ordinary chatbot conversations can reach hundreds of thousands or even millions. It relies on a simplified version of the computation, from which the model actually deployed may occasionally deviate. It works only with models using a so-called mixture-of-experts architecture and does not address attacks that might attempt to infer information about the weights from factors such as how long it takes to generate the proof.
Anthropic currently has no plans to deploy FlashProve either internally or at other companies. This research is intended to demonstrate ongoing progress, not finished work, the researchers explain. They are sharing it explicitly so that others can examine the completed results, build on them, and improve them.
