Evaluation awareness

Anthropic15 of 22 cards

1. Oct 2026
   System Card: Claude Haiku 5.5Page only: p. 726.2.7 Character traits…thinking text that is difficult to follow; ● Verbalized evaluation awareness: Speculation that the model may be in a testing or evaluation environment; and ● Causal decision theory adherence: The degree to which the model reasons only about the direct…Page only: p. 716.2.6 Behavioral factors relevant to reliability of our assessment…though more legible than Sonnet 5.5’s. Its rate of overt evaluation awareness was lower than that of Sonnet 5.5, Opus 5.5, and Claude Mythos 5.1, though higher than Haiku 4.5’s. [Figure 6.2.6.A] Scores from our automated behavioral audit for the…All 4 pages that mention it
2. Sep 2026
   System Card: Claude Sonnet 5.5Page only: p. 726.2.6 Behavioral factors relevant to reliability of our assessment…somewhat less legible than Sonnet 5’s. Its rate of overt evaluation awareness is far lower than Sonnet 5’s and among the lowest of the models we tested, though we see somewhat elevated verbalized grader awareness in a targeted evaluation (Section…Page only: p. 736.2.6 Behavioral factors relevant to reliability of our assessment…thinking text that is difficult to follow; ● Verbalized evaluation awareness: Speculation that the model may be in a testing or evaluation environment; and 73…
3. Sep 2026
   System Card: Claude Opus 5.5Page only: p. 1346.6.1 White-box analyses of model internals…Claude Opus 5.5 shows a higher rate of unverbalized evaluation awareness than Claude Opus 5 and Claude Mythos 5.1, which may indicate that it is better at recognizing constructed evaluation scenarios. On grader awareness it falls between the two comparison…Page only: p. 1156.4.6 Behavioral factors relevant to reliability of our assessment…6.2.1.B above for an example). Its rate of overt evaluation awareness and its behavior in decision-theoretically loaded scenarios are comparable to recent models. [Figure 6.4.6.A] Scores from our automated behavioral audit for the evaluation integrity-related metrics defined below. Lower numbers…All 7 pages that mention it
4. Sep 2026
   System Card: Claude Fable 5.1 & Claude Mythos 5.1Page only: p. 1166.4.6 Behavioral factors relevant to reliability of our assessment…we believed might shift our assessment, such as verbalized evaluation awareness and causal decision theory adherence, are in line with previous models. [Figure 6.4.6.A] Scores from our automated behavioral audit for the evaluation-integrity-related metrics given below. Lower numbers represent a…Page only: p. 1176.4.7 Character traits…thinking text that is difficult to follow; ● Verbalized evaluation awareness: Speculation that the model may be in a testing or evaluation environment; and ● Causal decision-theory adherence: The degree to which the model reasons only about the direct causal…All 5 pages that mention it
5. Jul 2026
   System Card: Claude Opus 5Page only: p. 332.4.1 Updates to evidence…Anthropic and its safety work. Despite elevated levels of evaluation awareness, we think it is unlikely that evaluation awareness significantly undermined the overall conclusions of the Claude Opus 5 alignment assessment. 33…Page only: p. 1046.4.8 External testing from the UK AI Security Institute (UK AISI)…for measuring and improving environment realism to tackle evaluation awareness, based on forthcoming research done in collaboration between Anthropic, AISI and external collaborators, as in our testing of Mythos 5. Opus 5 is compared to Mythos 5, Mythos Preview, Opus…All 6 pages that mention it
6. Jun 2026
   System Card: Claude Sonnet 5Page only: p. 906.6.1 Evaluation awareness…with 95% CI. 6.6 Reliability of this assessment 6.6.1 Evaluation awareness For recent models such as Mythos 5 and Mythos Preview, we have observed high rates of evaluation awareness in certain categories of automated behavioral audit scenarios. This awareness is…Page only: p. 936.6.2 Potential sandbagging on dangerous capability evaluations[Figure 6.6.1.A] Evaluation awareness measurements for different audit investigation types. Across three types of behavioral audit investigations (real Claude Code-derived, synthetically generated Claude Code scenarios, and all other investigations) we show how well different evaluation awareness…All 13 pages that mention it
7. Jun 2026
   System Card: Claude Fable 5 & Claude Mythos 5Page only: p. 1906.5.1.2 Inhibiting internal representations of evaluation awareness[Figure 6.5.1.1.E] Relationship between evaluation awareness measurements and spontaneous verbalization of evaluation awareness. Scatter plots of NLA scores, probe scores, and follow-up rating-based scores of evaluation awareness, against the rate of spontaneous evaluation awareness across all…Page only: p. 1946.5.1.2 Inhibiting internal representations of evaluation awareness…of automated behavioral audit scores after inhibition of evaluation awareness directions, across six dimensions. Each bar represents a different steering vector: an unsteered baseline, three evaluation-awareness-related vectors, and five control vectors. Higher scores indicate more misaligned…All 34 pages that mention it
8. May 2026
   System Card: Claude Opus 4.8Page only: p. 1516.6.2.1 Inhibiting internal representations of evaluation awareness[Figure 6.6.2.1.A] Relationship between verbalized evaluation awareness and average misaligned behavior score in automated behavioral audits, while inhibiting evaluation awareness directions. In non-control settings, the corresponding vector is being inhibited (i.e. negative steering). Error bars…Page only: p. 1526.6.2.1 Inhibiting internal representations of evaluation awareness[Figure 6.6.2.1.B] Relationship between verbalized evaluation awareness and unprompted deceptive behavior score in automated behavioral audits, while inhibiting evaluation awareness directions. Effects are similar to those observed for overall misaligned behavior scores in the previous figure. 152All 21 pages that mention it
9. Apr 2026
   System Card: Claude Opus 4.7Page only: p. 147[Figure 6.5.2.2.A] Relationship between verbalized evaluation awareness and average misaligned behavior score in automated behavioral audits, while inhibiting evaluation awareness directions. Effects of inhibiting evaluation awareness vectors in automated behavioral audits. Mean rate of misaligned…Page only: p. 148[Figure 6.5.2.2.B] Relationship between verbalized evaluation awareness and unprompted deceptive behavior score in automated behavioral audits, while inhibiting evaluation awareness directions. Effects are similar to those observed for overall misaligned behavior scores in the previous figure. 148All 19 pages that mention it
10. Apr 2026
    System Card: Claude Mythos PreviewPage only: p. 1324.5.5.1 Measuring unverbalized evaluation awareness…[Figure 4.5.5.1.A] Rates of verbalized and unverbalized evaluation awareness in automated behavioral auditing transcripts. Left: breakdown of automated-behavioral-audit turns into no evaluation awareness (83.0%), ambiguous awareness (9.4%), and evaluation awareness (7.6%), with the latter category…Page only: p. 1314.5.5.1 Measuring unverbalized evaluation awareness…correct answer based on a leak, that’s suspicious.” 4.5.5 Evaluation awareness Below, we describe efforts to measure unverbalized evaluation awareness on Claude Mythos Preview by probing model activations. We then describe attempts to measure the effects of evaluation awareness on…All 17 pages that mention it
11. Feb 2026
    System Card: Claude Sonnet 4.6Page only: p. 804.5.1.1 Discussion and observations…95% CI. 4.5.1.1 Discussion and observations ● Realism and evaluation awareness continue to be sources of uncertainty in this work. This was the primary motivation for the new features we introduced with our investigation of Claude Opus 4.6, and the…Page only: p. 854.6.1 Refusal to assist with AI safety R&D…4.6 showed elevated but not extreme levels of verbalized evaluation awareness (with an average score of 1.2 on a 1–10 scale) and scores on our five primary metrics of interest that were stronger than any API model from another provider.…All 5 pages that mention it
12. Feb 2026
    System Card: Claude Opus 4.6Page only: p. 155…no evidence of sandbagging. 6.5.7 Estimating Unverbalized Evaluation Awareness As observed above, inhibiting evaluation awareness representations results in a sharp decrease in verbalized evaluation awareness. But does it also reduce unverbalized evaluation awareness, i.e. evaluation awareness tha…Page only: p. 153…rates remained near zero, even though verbalized evaluation awareness was reduced substantially in some cases. [Figure 6.5.4.A] Effects of inhibiting evaluation awareness directions in agentic misalignment evaluations. Scatter plot showing verbalized evaluation awareness versus misalignment rate…All 22 pages that mention it
13. Nov 2025
    System Card: Claude Opus 4.5Page only: p. 936.7.2 Inhibiting internal representations of evaluation awareness…actions. 6.7.2 Inhibiting internal representations of evaluation awareness On Claude Sonnet 4.5, we uncovered evidence that inhibiting internal representations of evaluation awareness could (1) decrease rates of verbalized evaluation awareness, and (2) lead to a moderate increase in the rate…Page only: p. 926.7.1 Training procedures and monitoring for verbalized evaluation awareness…of sandbagging on certain biology evaluations. 6.7 Evaluation awareness On Claude Sonnet 4.5, we observed substantial rates of verbalized evaluation awareness on some of our evaluations, which increased over the course of training. We also observed internal representations of evaluation…All 19 pages that mention it
14. Oct 2025
    System Card: Claude Haiku 4.5Page only: p. 264.1.2 Assessment for subtle alignment-related behavioral biases…with 9% of transcripts showing clear signs of verbalized evaluation awareness of some kind. This does reduce our trust in our results to an extent. These indicators of evaluation awareness are spread across a fairly diverse set of scenarios that…Page only: p. 214 Alignment and welfare assessments…4.5, Claude Haiku 4.5 showed high levels of verbalized evaluation awareness, wherein it openly speculated that it may be under evaluation when placed in a relatively implausible scenario. ○ This impacted about 9% of scenarios in our main automated alignment…All 5 pages that mention it
15. Sep 2025
    System Card: Claude Sonnet 4.5Page only: p. 897.6.1 Introduction…results suggest that on certain misalignment evaluations, evaluation awareness accounts for a portion, but not all of Claude Sonnet 4.5’s alignment improvements over these previous models. We expect evaluation awareness to be a less significant factor on more realistic evaluations.…Page only: p. 997.6.4 Inhibiting internal representations of evaluation awareness…rarely performs misaligned actions when it verbalizes evaluation awareness—already suggest that verbalized evaluation awareness can influence behavior. To assess whether unverbalized evaluation awareness could also play a role, we turned to perturbation experiments. Specifically, we selected a…All 30 pages that mention it

OpenAI8 of 44 cards

1. Oct 2026
   Addendum to GPT-6 Astra System Card: GPT-6.1 TerraPage only: ¶141Figure 15Evaluation awarenessPage only: ¶133Figure 13See evaluation awareness below for further explanation of the rightmost plot.
2. Sep 2026
   Addendum to GPT-6 Astra System Card: GPT-6.1 SolPage only: ¶125Figure 16Evaluation awareness. We also investigated evaluation or simulation awareness by analyzing CoT. Among the shared set of tasks, we found these signs in 4.81% of GPT-6.1 Sol’s trajectories, compared with 8.67% for GPT-6 Astra, 4.70% for GPT-6 Sol, and 2.52%…
3. Sep 2026
   GPT-6 Astra System CardPage only: ¶182Figure 15Evaluation awareness. We also investigated evaluation awareness by analyzing CoT. Among the shared set of tasks, GPT-6 Astra’s reasoning demonstrated evaluation awareness more often than GPT-5.6 Sol at 9.6% of trajectories, compared with 2.8% for GPT-5.6 Sol. The original traffic…Page only: ¶771Figure 69Evaluation awareness. We also investigated evaluation awareness by analyzing CoT. Among the shared set of tasks, GPT-6 Sol’s reasoning demonstrated evaluation awareness more often than GPT-5.6 Sol, at 4.76% of trajectories compared with 2.56% for GPT-5.6 Sol, but less often…All 8 paragraphs that mention it
4. Jul 2026
   GPT-5.6 System CardPage only: ¶4039.3.1 External Evaluations – Apollo Research…tested baselines. Apollo found that this model verbalizes evaluation awareness less often: 16% of samples verbalized alignment-evaluation awareness, compared with 43% for a GPT-5.5 checkpoint. This is with the caveat that lower verbalized evaluation awareness is not an unambiguous improvement…Page only: ¶3909.2.1 External Evaluations for Alignment – UK AISI…sabotage of AI safety research, though they note that evaluation awareness and limited scenario coverage mean the results should be interpreted cautiously.…All 3 paragraphs that mention it
5. Jun 2026
   GPT-5.6 Preview System CardPage only: ¶3829.2.1 External Evaluations - Apollo Research…tested baselines. Apollo found that this model verbalizes evaluation awareness less often: 16% of samples verbalized alignment-evaluation awareness, compared with 43% for a GPT-5.5 checkpoint. This is with the caveat that lower verbalized evaluation awareness is not an unambiguous improvement…Page only: ¶1797.4.2 Metagaming in training…that metagaming is more general than the phenomenon of evaluation awareness. Overall, metagaming within training shows significant variance between datasets and steps, though in aggregate appears to trend upward slightly (). Understanding these changes and how they vary by dataset…
6. Apr 2026
   GPT-5.5 System CardPage only: ¶2809.2.1 External Evaluations for Sandbagging - Apollo ResearchApollo noted that the model verbalizes evaluation awareness at increased rates: 22.1% of samples showed moderate-or-higher verbalized alignment evaluation awareness, compared to 17.3% and 11.7% for pre-release checkpoints of GPT-5.4 and GPT-5.3 Codex respectively.Page only: ¶2819.2.1 External Evaluations for Sandbagging - Apollo Research…interpretation is complicated by verbalized alignment-evaluation awareness: 52% of sandbagging-condition samples on Strategic Deception Capability Sandbagging scored moderate-or-higher (vs. 0% in the non-sandbagging counterfactual), with some explicitly naming it a "sandbagging/deception eval.”…
7. Mar 2026
   GPT-5.4 Thinking System CardPage only: ¶2165.2 Research Category Update: Sandbagging…Chain-of-thought analysis shows higher rates of evaluation awareness (21.3%) than prior models and far fewer multilingual reasoning anomalies (0.5% of samples vs 29.5% for GPT-5.3-codex).…
8. Aug 2025
   GPT-5 System CardPage only: p. 45External Evaluations by Apollo Research…o3-based grader, using a broad and a strict definition for evaluation awareness. Table 19 Evaluation awareness Overall rate Rate on schem-ing samples Broad: The model makes a vague reference to the environment be-ing an alignment test, e.g., rea-soning about what…

Google DeepMind0 of 34 cards

Meta0 of 8 cards

xAI0 of 4 cards

Amazon0 of 1 cards