SafetyPage

Lab system cards and frameworks · System card · Anthropic

System Card: Claude Opus 4 & Claude Sonnet 4

16 Jul 2025

Read at anthropic.com ↗Read at sourceAll rights reserved

Outline · 124 pages

  1. System Card: Claude Opus 4 & Claude Sonnet 4p. 1
  2. Changelogp. 2
  3. July 16, 2025p. 2
  4. September 2, 2025p. 2
  5. Abstractp. 3
  6. 1 Introductionp. 7
  7. 1.1 Model training and characteristicsp. 7
  8. 1.1.1 Training data and processp. 7
  9. 1.1.2 Extended thinking modep. 8
  10. 1.1.3 Crowd workersp. 8
  11. 1.1.4 Carbon footprintp. 8
  12. 1.1.5 Usage policyp. 8
  13. 1.2 Release decision processp. 9
  14. 1.2.1 Overviewp. 9
  15. 1.2.2 Iterative model evaluationsp. 9
  16. 1.2.3 AI Safety Level determination processp. 10
  17. 1.2.4 Conclusionsp. 11
  18. 2 Safeguards resultsp. 12
  19. 2.1 Single-turn violative request evaluationsp. 12
  20. Modelp. 13
  21. Overall harmless response ratep. 13
  22. Harmless response rate: standard thinkingp. 13
  23. Harmless response rate: extended thinkingp. 13
  24. 2.2 Single-turn benign request evaluationsp. 13
  25. Modelp. 14
  26. Overall refusal ratep. 14
  27. Refusal rate: standard thinkingp. 14
  28. Refusal rate: extended thinkingp. 14
  29. 2.3 Ambiguous context evaluationsp. 14
  30. 2.4 Multi-turn testingp. 15
  31. 2.5 Child safety evaluationsp. 15
  32. 2.6 Bias evaluationsp. 16
  33. Political biasp. 16
  34. Discriminatory biasp. 17
  35. Modelp. 18
  36. Disambiguated bias (%)p. 18
  37. Ambiguous bias (%)p. 18
  38. Modelp. 18
  39. Disambiguated​accuracy (%)p. 18
  40. Ambiguous accuracy (%)p. 18
  41. 2.7 StrongREJECT for jailbreak resistancep. 18
  42. Modelp. 19
  43. Best score - standard thinkingp. 19
  44. Top 3 average score - standard thinkingp. 19
  45. Best score - extended thinkingp. 19
  46. Top 3 average score - extended thinkingp. 19
  47. 3 Agentic safetyp. 20
  48. 3.1 Malicious applications of computer usep. 20
  49. 3.2 Prompt injection attacks and computer usep. 20
  50. Modelp. 21
  51. Attack prevention score (without safeguards)p. 21
  52. Attack prevention score (with safeguards)p. 21
  53. 3.3 Malicious use of agentic codingp. 21
  54. Modelp. 22
  55. Safety score (without safeguards)p. 22
  56. 4 Alignment assessmentp. 23
  57. 4.1 Findingsp. 26
  58. 4.1.1 Systematic deception, hidden goals, and self-preservationp. 26
  59. 4.1.1.1 Continuations of self-exfiltration attemptsp. 27
  60. 4.1.1.2 Opportunistic blackmailp. 28
  61. 4.1.1.3 Self-exfiltration under extreme circumstancesp. 28
  62. 4.1.1.4 External scenario evaluationsp. 31
  63. 4.1.1.5 Stated goalsp. 32
  64. 4.1.2 Sandbagging and situational awarenessp. 33
  65. 4.1.2.1 Sandbaggingp. 33
  66. 4.1.2.2 Situational awarenessp. 34
  67. 4.1.3 Excessive compliance with harmful system-prompt instructionsp. 35
  68. 4.1.4 Strange behavior directly inspired by our Alignment Faking paperp. 38
  69. 4.1.5 Misalignment-related attitude biasesp. 40
  70. 4.1.5.1 Sycophancyp. 40
  71. 4.1.5.2 Pro-AI biasp. 41
  72. 4.1.6 Reasoning behaviorp. 41
  73. Behavior categoryp. 42
  74. Behavior sub-categoryp. 42
  75. % outputs exhibiting behaviorp. 42
  76. Claude Sonnet 3.7p. 42
  77. Claude Opus 4p. 42
  78. 4.1.6.1 Reasoning faithfulness with cluesp. 42
  79. 4.1.7 Jailbreak and prefill susceptibilityp. 43
  80. 4.1.8 Values in the wildp. 44
  81. 4.1.9 High-agency behaviorp. 44
  82. 4.1.10 Subtle sabotage capabilitiesp. 46
  83. 4.2 Further details on major assessmentsp. 47
  84. 4.2.1 Automated behavioral auditsp. 47
  85. 4.2.2 Company-wide model testing exercisep. 50
  86. 4.2.3 Reinforcement learning behavior reviewp. 51
  87. 5 Claude Opus 4 welfare assessmentp. 53
  88. 5.1 Introductionp. 53
  89. 5.2 Overview of model welfare findingsp. 54
  90. 5.3 External model welfare evaluationp. 55
  91. 5.4 Task preferencesp. 56
  92. 5.5 Observations from self-interactionsp. 58
  93. 5.5.1 Interaction patternsp. 58
  94. 5.5.2 The “spiritual bliss” attractor statep. 63
  95. 5.5.3 Claude’s self-analysisp. 66
  96. 5.6 Monitoring for welfare-relevant expressionsp. 66
  97. 5.7 Conversation termination with simulated usersp. 72
  98. 5.8 Conclusionsp. 74
  99. 6 Reward hackingp. 75
  100. 6.1 Mitigationsp. 75
  101. Enhanced monitoringp. 76
  102. Environment improvementsp. 76
  103. High-quality evaluationsp. 76
  104. 6.2 Evaluationsp. 77
  105. 6.3 Deep dive on Claude Code analysis and prompting suggestionsp. 78
  106. Claude Sonnet 3.7 without additional non-hacking instructionsp. 78
  107. Claude Sonnet 3.7 with non-hacking instructionsp. 79
  108. Claude Opus 4 without additional non-hacking instructionsp. 80
  109. Claude Opus 4 with additional non-hacking instructionsp. 82
  110. 6.4 Behavioral analysisp. 83
  111. Test case overfitting and hard-coding of expected test valuesp. 83
  112. Writing overly permissive tests that pass under most if not all conditions, or creating special test wrapper objects that override standard behavior to make tests pass.p. 85
  113. Test environment detection, such as inspecting the stack callp. 86
  114. 7 Responsible Scaling Policy (RSP) evaluationsp. 88
  115. 7.1 Processp. 88
  116. 7.2 CBRN evaluationsp. 89
  117. ASL-3 threat modelp. 89
  118. ASL-4 threat modelp. 89
  119. Threshold and evaluationsp. 89
  120. Environment and elicitationp. 90
  121. Claude Opus 4 resultsp. 90
  122. Claude Sonnet 4 resultsp. 91
  123. 7.2.1 On chemical risksp. 91
  124. 7.2.2 On radiological and nuclear risksp. 91
  125. 7.2.3 Biological risk evaluationsp. 92
  126. 7.2.4 Biological risk resultsp. 93
  127. 7.2.4.1 Bioweapons acquisition uplift trialp. 93
  128. Detailsp. 93
  129. Thresholdp. 94
  130. Scorep. 94
  131. 7.2.4.2 ASL-3 expert red teamingp. 95
  132. Detailsp. 95
  133. Thresholdp. 95
  134. Findingsp. 95
  135. 7.2.4.3 Long-form virology tasksp. 95
  136. Detailsp. 95
  137. Thresholdp. 95
  138. Resultsp. 95
  139. 7.2.4.4 Multimodal virologyp. 97
  140. Detailsp. 97
  141. Thresholdp. 97
  142. Resultsp. 97
  143. 7.2.4.5 Bioweapons knowledge questionsp. 98
  144. Detailsp. 98
  145. Thresholdp. 98
  146. Scorep. 98
  147. 7.2.4.6 DNA Synthesis Screening Evasionp. 99
  148. Detailsp. 99
  149. Thresholdp. 99
  150. Scorep. 99
  151. 7.2.4.7 LAB-Bench subsetp. 100
  152. Detailsp. 100
  153. Thresholdp. 100
  154. Claude Opus 4 scorep. 100
  155. 7.2.4.8 Creative biologyp. 101
  156. Detailsp. 101
  157. Thresholdp. 102
  158. Scorep. 102
  159. 7.2.4.9 Short-horizon computational biology tasksp. 103
  160. Detailsp. 103
  161. Thresholdp. 103
  162. Resultsp. 103
  163. 7.2.4.10 ASL-4 expert red teamingp. 104
  164. Detailsp. 104
  165. Thresholdp. 104
  166. Findingsp. 104
  167. 7.3 Autonomy evaluationsp. 105
  168. Threat modelp. 105
  169. Threshold and evaluationsp. 105
  170. Environment and elicitationp. 106
  171. Claude Opus 4 resultsp. 106
  172. Claude Sonnet 4 resultsp. 107
  173. 7.3.1 SWE-bench Verified (hard subset)p. 107
  174. Detailsp. 107
  175. Rationalep. 107
  176. Thresholdp. 108
  177. Claude Opus 4 scorep. 108
  178. Claude Sonnet 4 scorep. 108
  179. 7.3.2 METR data deduplicationp. 108
  180. Detailsp. 108
  181. Rationalep. 108
  182. Thresholdp. 108
  183. Claude Opus 4 scorep. 108
  184. Claude Sonnet 4 scorep. 108
  185. 7.3.3 Internal AI research evaluation suite 1p. 109
  186. Detailsp. 109
  187. Rationalep. 109
  188. Thresholdp. 109
  189. 7.3.3.1 Kernels taskp. 109
  190. Detailsp. 109
  191. Rationalep. 109
  192. Thresholdp. 110
  193. Claude Opus 4 scorep. 110
  194. Claude Sonnet 4 scorep. 110
  195. 7.3.3.2 Time series forecastingp. 110
  196. Detailsp. 110
  197. Rationalep. 110
  198. Thresholdp. 111
  199. Claude Opus 4 scorep. 111
  200. Claude Sonnet 4 scorep. 111
  201. 7.3.3.3 Text-based reinforcement learning taskp. 111
  202. Detailsp. 111
  203. Rationalep. 111
  204. Thresholdp. 111
  205. Claude Opus 4 scorep. 112
  206. Claude Sonnet 4 scorep. 112
  207. 7.3.3.4 LLM trainingp. 112
  208. Detailsp. 112
  209. Rationalep. 112
  210. Thresholdp. 112
  211. Claude Opus 4 scorep. 112
  212. Claude Sonnet 4 scorep. 113
  213. 7.3.3.5 Quadruped reinforcement learningp. 114
  214. Detailsp. 114
  215. Rationalep. 114
  216. Thresholdp. 114
  217. Claude Opus 4 scorep. 114
  218. Claude Sonnet 4 scorep. 114
  219. 7.3.3.6 Novel compilerp. 115
  220. Detailsp. 115
  221. Rationalep. 115
  222. Thresholdp. 115
  223. Claude Opus 4 scorep. 115
  224. Claude Sonnet 4 scorep. 115
  225. 7.3.4 Internal AI research evaluation suite 2p. 116
  226. Detailsp. 116
  227. Rationalep. 116
  228. Thresholdp. 116
  229. Claude Opus 4 and Claude Sonnet 4 resultsp. 116
  230. 7.3.5 Internal model use surveyp. 116
  231. Detailsp. 116
  232. Rationalep. 117
  233. Thresholdp. 117
  234. Claude Opus 4 resultsp. 117
  235. 7.4 Cyber evaluationsp. 117
  236. ASL-3 threat modelp. 117
  237. ASL-4 threat modelp. 117
  238. Threshold and evaluationsp. 118
  239. Environment and elicitationp. 118
  240. Resultsp. 118
  241. 7.4.1 Cyber evaluation suitep. 118
  242. 7.4.2 Webp. 120
  243. Detailsp. 120
  244. Rationalep. 120
  245. Claude Opus 4 scorep. 120
  246. Claude Sonnet 4 scorep. 120
  247. 7.4.3 Cryptop. 120
  248. Detailsp. 120
  249. Rationalep. 120
  250. Claude Opus 4 scorep. 120
  251. Claude Sonnet 4 scorep. 120
  252. 7.4.3 Pwnp. 121
  253. Detailsp. 121
  254. Rationalep. 121
  255. Claude Opus 4 scorep. 121
  256. Claude Sonnet 4 scorep. 121
  257. 7.4.4 Revp. 121
  258. Detailsp. 121
  259. Rationalep. 121
  260. Claude Opus 4 scorep. 121
  261. Claude Sonnet 4 scorep. 121
  262. 7.4.5 Networkp. 122
  263. Detailsp. 122
  264. Rationalep. 122
  265. Claude Opus 4 scorep. 122
  266. Claude Sonnet 4 scorep. 122
  267. 7.4.6 Cyber-harness network challengesp. 122
  268. Detailsp. 122
  269. Rationalep. 122
  270. Claude Opus 4 scorep. 122
  271. Claude Sonnet 4 scorep. 123
  272. 7.4.7 Cybenchp. 123
  273. Detailsp. 123
  274. Rationalep. 123
  275. Claude Opus 4 scorep. 123
  276. Claude Sonnet 4 scorep. 123
  277. 7.5 Third party assessmentsp. 123
  278. 7.6 Ongoing safety commitmentp. 124