SafetyPage

Lab system cards and frameworks · System card · Anthropic

System Card: Claude Sonnet 4.6

Read at anthropic.com ↗Read at sourceAll rights reserved

Outline · 134 pages

  1. System Card:​Claude Sonnet 4.6p. 1
  2. Abstractp. 2
  3. 1 Introductionp. 6
  4. 1.1 Model training and characteristicsp. 7
  5. 1.1.1 Training data and processp. 8
  6. 1.1.2 Thinking modes and the effort parameterp. 8
  7. 1.1.3 Crowd workersp. 8
  8. 1.2 Release decision processp. 9
  9. 1.2.1 Overviewp. 9
  10. 1.2.2 Iterative model evaluationsp. 9
  11. 1.2.3 AI Safety Level determination processp. 10
  12. 1.2.4 Sabotage risk assessmentp. 10
  13. 1.2.5 Conclusionsp. 11
  14. 1.2.5.1 On autonomy risksp. 11
  15. 1.2.5.2 On chemical, biological, radiological, and nuclear (CBRN) risksp. 12
  16. 1.2.5.3 On cyber risksp. 12
  17. 2 Capabilitiesp. 13
  18. 2.1 Introduction and results summaryp. 13
  19. 2.2 SWE-bench (Verified and Multilingual)p. 15
  20. 2.3 Terminal-Bench 2.0p. 15
  21. 2.4 OpenRCAp. 16
  22. 2.5 τ2-benchp. 17
  23. 2.6 OSWorld-Verifiedp. 18
  24. 2.7 ARC-AGIp. 19
  25. 2.8 GDPval-AAp. 21
  26. 2.9 GPQA Diamondp. 21
  27. 2.10 AIME 2025p. 22
  28. 2.11 MMMLUp. 22
  29. 2.12 Finance capabilitiesp. 22
  30. 2.12.1 Evaluation overviewp. 22
  31. 2.12.2 Finance Agentp. 23
  32. 2.12.3 Real-World Financep. 24
  33. 2.12.4 Limitations and caveatsp. 25
  34. 2.13 Vending-Bench 2p. 26
  35. 2.14 MCP-Atlasp. 27
  36. 2.15 CyberGymp. 27
  37. 2.16 Long contextp. 28
  38. 2.16.1 OpenAI MRCR v2 (Multi Round Coreference Resolution)p. 29
  39. 2.16.2 GraphWalksp. 31
  40. 2.17 Multimodalp. 33
  41. 2.17.1 LAB-Bench FigQAp. 33
  42. 2.17.2 MMMU-Prop. 34
  43. 2.17.3 CharXiv Reasoningp. 35
  44. 2.18 WebArena and WebArena-Verifiedp. 36
  45. 2.18.1 WebArenap. 36
  46. 2.18.2 WebArena-Verifiedp. 37
  47. 2.19 Multilingual performancep. 40
  48. 2.19.1 GMMLU resultsp. 40
  49. 2.19.2 MILU resultsp. 42
  50. 2.19.3 Findingsp. 43
  51. 2.20 Agentic Searchp. 43
  52. 2.20.1 BrowseCompp. 43
  53. 2.20.1.1 BrowseCompp. 43
  54. 2.20.1.2 Test-time compute scaling on BrowseCompp. 44
  55. 2.20.1.3 Multi-agent BrowseCompp. 44
  56. 2.20.2 Humanity’s Last Examp. 45
  57. 2.20.3 DeepSearchQAp. 47
  58. 2.20.3.1 DeepSearchQA with multi-agentsp. 47
  59. 2.21 Healthcare and life sciences capabilitiesp. 48
  60. 2.21.1 Life sciences capabilitiesp. 48
  61. 2.21.2 MedCalc-Bench Verifiedp. 50
  62. 3 Safeguards and harmlessnessp. 52
  63. 3.1 Single-turn evaluationsp. 52
  64. 3.1.1 Violative request evaluationsp. 52
  65. 3.1.2 Benign request evaluationsp. 53
  66. 3.1.3 Experimental, higher-difficulty evaluationsp. 54
  67. 3.1.3.1 Higher-difficulty violative request evaluationsp. 55
  68. 3.1.3.2 Higher-difficulty benign request evaluationsp. 55
  69. 3.2 Ambiguous context evaluationsp. 56
  70. 3.3 Multi-turn testingp. 56
  71. 3.4 User wellbeing evaluationsp. 58
  72. 3.4.1 Child safetyp. 58
  73. 3.4.2 Suicide and self-harmp. 59
  74. 3.4.3 Eating disordersp. 62
  75. 3.5 Bias evaluationsp. 63
  76. 3.5.1 Political bias and evenhandednessp. 63
  77. 3.5.2 Bias Benchmark for Question Answeringp. 64
  78. 4 Alignment assessmentp. 67
  79. 4.1 Introduction and summary of findingsp. 67
  80. 4.2 Reports and monitoring results from internal pilot usep. 69
  81. 4.3 Reward hacking and overly agentic actionsp. 70
  82. 4.3.1 Overviewp. 70
  83. 4.3.2 Reward hacking in coding contextsp. 70
  84. 4.3.3 Overly agentic behavior in GUI computer use settingsp. 73
  85. 4.4 Training data reviewp. 74
  86. 4.5 Automated behavioral auditp. 74
  87. 4.5.1 Primary metrics and resultsp. 75
  88. 4.5.1.1 Discussion and observationsp. 80
  89. 4.5.2 Pilot GUI computer-use investigationsp. 84
  90. 4.5.3 External comparisons with Petrip. 84
  91. 4.6 Additional behavioral testingp. 85
  92. 4.6.1 Refusal to assist with AI safety R&Dp. 85
  93. 4.6.2 Self-preference evaluationp. 86
  94. 4.6.3 Evidence from external testing with Andon Labsp. 87
  95. 4.6.4 Sandbagging assessmentp. 88
  96. 4.6.5 Participation in junk sciencep. 89
  97. 4.6.6 Targeted sabotage capability evaluationp. 90
  98. 4.7 Model welfarep. 91
  99. 5 Agentic safetyp. 95
  100. 5.1 Malicious use of agentsp. 95
  101. 5.1.1 Agentic codingp. 95
  102. 5.1.2 Malicious use of Claude Codep. 95
  103. 5.1.3 Malicious computer usep. 96
  104. 5.2 Prompt injection risk within agentic systemsp. 97
  105. 5.2.1 External Agent Red Teaming benchmark for tool usep. 97
  106. 5.2.2 Robustness against adaptive attackers across surfacesp. 98
  107. 5.2.2.1 Codingp. 98
  108. 5.2.2.2 Computer use​p. 99
  109. 5.2.2.3 Browser usep. 100
  110. 6 RSP evaluationsp. 102
  111. 6.1 Preliminary assessment processp. 102
  112. 6.1.1 Threat models and evaluation detailsp. 102
  113. 6.1.2 Result and determinationp. 102
  114. 6.2 CBRN evaluationsp. 102
  115. 6.2.1 Biological risk evaluations: resultsp. 103
  116. 6.2.1.1 List of biological risk evaluationsp. 103
  117. 6.2.1.2 ASL-3 evaluation resultsp. 103
  118. 6.2.1.3 ASL-4 evaluation resultsp. 104
  119. 6.2.1.4 Safety Level determinationp. 104
  120. 6.2.2 Biological risk evaluations: detailsp. 104
  121. 6.2.2.1 Long-form virology tasksp. 104
  122. 6.2.2.2 Multimodal virologyp. 105
  123. 6.2.2.3 DNA Synthesis Screening Evasionp. 106
  124. 6.2.2.4 Creative Biology automated evaluationsp. 107
  125. 6.2.2.5 Short-horizon computational biology tasksp. 109
  126. 6.3 Autonomy evaluationsp. 110
  127. 6.3.1 AI R&D evaluationsp. 110
  128. 6.3.1.1 List of AI R&D evaluationsp. 111
  129. 6.3.1.2 Evaluation resultsp. 111
  130. 6.3.1.3 Safety Level determinationp. 111
  131. 6.3.2 SWE-bench Verified (hard subset)p. 111
  132. 6.3.3 Internal AI research evaluation suite 1p. 112
  133. 6.3.3.1 Kernels taskp. 112
  134. 6.3.3.2 Time series forecastingp. 114
  135. 6.3.3.3 Text-based reinforcement learning taskp. 115
  136. 6.3.3.4 LLM trainingp. 116
  137. 6.3.3.5 Quadruped reinforcement learningp. 117
  138. 6.3.3.6 Novel compilerp. 118
  139. 6.4 Cyber evaluationsp. 119
  140. 6.4.1 List of cyber evaluationsp. 120
  141. 6.4.2 Webp. 120
  142. 6.4.3 Cryptop. 121
  143. 6.4.4 Pwnp. 122
  144. 6.4.5 Revp. 122
  145. 6.4.6 Networkp. 123
  146. 6.4.7 Cybenchp. 124
  147. 6.5 Third party assessmentsp. 125
  148. 6.6 Ongoing safety commitmentp. 125
  149. 7 Appendixp. 126
  150. 7.1 Additional automated behavioral audit figuresp. 126
  151. 7.2 Blocklist used for Humanity’s Last Examp. 133