Anthropic has detailed self-assessment, a system that lets Claude assess a conversation for possible misuse after every response. The mechanism has been running in production since June across Claude models, including Fable 5, and produces information to help prioritize human review.
Hidden assessment uses conversation context
After Claude responds, the system appends a hidden instruction asking the model to assess the conversation. For a potentially harmful exchange, Claude identifies the type of harm, the priority for human review, whether it refused and a brief explanation. For a benign conversation, it returns only a flag with the value false.
Alongside the flag, the system derives a continuous risk score from the probability the model assigns to the token true. Anthropic can also add a session-specific rule to the assessment prompt. That allows a verified security team to test designated software without having its activity treated in the same way as an unauthorized intrusion attempt.
The assessment reuses the KV cache, which retains the conversation context and intermediate computations from the original response. According to Anthropic, it typically needs about 10 output tokens and adds less than 2% to inference compute across production API traffic.
Self-assessment tests on Fable 5
In an Anthropic test, self-assessment on Claude Fable 5 detected 78% of conversations a stronger teacher classifier had identified as potentially harmful, at a 1% false-positive rate. It detected 94% of high-priority cases. The teacher was a classifier based on Claude Mythos Preview, with Anthropic’s full internal policies in its context.
The company also compared four models trained only with supervised learning. At the same 1% false-positive rate, recall rose from approximately 16% for the smallest model to 96% for the largest.
In another test, Anthropic omitted entire categories of harm from the training data and then evaluated synthetic conversations in those areas. Without fraud examples in training, recall on synthetic fraud conversations fell from 92.8% to 90.0%. For cybersecurity examples, it fell from 92.7% to 90.3%; both results were measured at a 1% false-positive rate.
Training on synthetic conversations
Anthropic trains self-assessment in two stages using entirely synthetic data. First, a teacher classifier labels generated conversations for supervised learning. The training loss applies only to the assessment tokens, not to the harmful responses themselves.
Subsequent reinforcement learning evaluates factors including valid JSON formatting, agreement with the teacher’s label and proximity to its priority score. The generator creates 1,024 harmful scenarios and the same number of benign ones for each subpolicy. The benign examples cover similar topics, making them harder to distinguish from harmful ones.
Limits on blocking attacks
In an Anthropic red-team test covering 24 offensive cybersecurity tasks, 79% of attacks succeeded without safeguards, compared with 54% when self-assessment was used. With a separate prompted classifier, the attack success rate was 25%. In the test, self-assessment increased attack costs measured in generated tokens by approximately four times.
Anthropic positions self-assessment as one layer of a broader detection and enforcement system. Stronger classifiers handle blocking decisions for high-risk cases.
Findings from production monitoring
In August, the system generated more than 100,000 flags for material associated with child sexual abuse and exploitation linked to a single API customer. After investigating, Anthropic reported the activity to the National Center for Missing and Exploited Children.
Roughly one-third of flags fell outside the existing harm taxonomy. Grouping those cases revealed attempts at unauthorized distillation of Claude’s reasoning, prompt-injection attacks and agent actions users had not authorized.
More than half of the organizations flagged by self-assessment had received no flags from other detection sources in the preceding month. One case involved an automated romance scam built using Claude Code.
