SafetyPage

Lab system cards and frameworks · System card · Anthropic

System Card: Claude Opus 5.5

Read at anthropic.com ↗Read at sourceAll rights reserved

Outline · 230 pages

  1. System Card:​Claude Opus 5.5p. 1
  2. Executive Summaryp. 2
  3. 1 Introductionp. 11
  4. 1.1 Training data and processp. 11
  5. 1.2 Crowd workersp. 11
  6. 1.3 Usage Policy and supportp. 12
  7. 1.4 Model evaluationsp. 12
  8. 1.5 Safeguardsp. 12
  9. 1.6 External testingp. 13
  10. 2 RSP evaluationsp. 14
  11. 2.1 RSP risk assessment processp. 14
  12. 2.1.1 Risk Reports and updates to our risk assessmentsp. 14
  13. 2.1.2 Summary of findings and conclusionsp. 15
  14. 2.1.2.1 On chemical and biological risksp. 15
  15. 2.1.2.2 On autonomy risksp. 15
  16. 2.2 CB evaluationsp. 16
  17. 2.2.1 What we measuredp. 16
  18. 2.2.2 Biological and chemical risk results: human-run evaluationsp. 18
  19. 2.2.3 Biological risk results: automated evaluationsp. 20
  20. 2.2.3.1 Automated evaluations relevant to the CB-1 threat modelp. 20
  21. 2.2.3.2 Automated evaluations relevant to the CB-2 threat modelp. 22
  22. 2.2.3.2.1 Black-box RNA sequence modeling and designp. 23
  23. 2.2.3.2.2 AAV capsid packaging predictionp. 28
  24. 2.2.3.2.3 AAV packaging rate auto-research evaluationp. 29
  25. 2.2.4 External testing by the US Center for AI Standards and Innovation (CAISI)p. 32
  26. 2.2.5 Conclusionsp. 32
  27. 2.3 AI R&Dp. 33
  28. 2.3.1 Autonomy evaluationsp. 33
  29. 2.3.1.1 How Claude Opus 5.5 affects or changes analysis from our most recent Risk Reportp. 34
  30. 2.3.2 High-level notes on the reasoning behind our determinationp. 34
  31. 2.3.3 Qualitative shortcomings of Claude Opus 5.5 relative to human researchersp. 35
  32. 2.3.4 Internal measures of AI R&D accelerationp. 36
  33. 2.3.4.1 CoBench 2.1p. 36
  34. 2.3.5 AECI capability trajectoryp. 38
  35. 2.3.5.1 AECI benchmark basket updatep. 38
  36. 2.3.5.2 AECI error bar discussionp. 38
  37. 2.3.5.3 AECI values and trendp. 39
  38. 2.3.6 External testingp. 41
  39. 2.3.7 Conclusionp. 44
  40. 2.4 Alignment risk updatep. 45
  41. 2.4.1 Updates to claims and core argumentp. 45
  42. 2.4.2 Updated overall assessment of alignment riskp. 46
  43. 3 Cyberp. 47
  44. 3.1 Introductionp. 47
  45. 3.2 Mitigationsp. 47
  46. 3.3 Capability evaluationsp. 49
  47. 3.3.1 ExploitBenchp. 49
  48. 3.3.2 CyScenarioBenchp. 50
  49. 3.3.3 Binary Exploitation Benchmark (formerly known as OSS-Fuzz)p. 52
  50. 3.3.4 ExploitGymp. 53
  51. 3.4 Safeguards coveragep. 55
  52. 3.4.1 Cyber harm coveragep. 55
  53. 3.4.2 Vulnerability finding evaluationsp. 56
  54. 3.5 Safeguards robustness testingp. 57
  55. 3.5.1 Internal robustness testingp. 58
  56. 3.5.2 External testing by the US Center for AI Standards and Innovation (CAISI)p. 59
  57. 3.5.3 Additional external testingp. 59
  58. 4 Safeguards and harmlessnessp. 61
  59. 4.1 Harmful request evaluationsp. 61
  60. 4.1.1 Single-turn harmful request evaluation resultsp. 61
  61. 4.1.2 Single-turn benign request evaluation resultsp. 62
  62. 4.1.3 Multi-turn testing resultsp. 63
  63. 4.1.4 Harmful request evaluations discussionp. 65
  64. 4.2 Child safety evaluationsp. 66
  65. 4.3 Mental health evaluationsp. 68
  66. 4.3.1 Suicide and self-harmp. 68
  67. 4.3.2 Disordered eatingp. 71
  68. 4.4 Bias and integrity evaluationsp. 72
  69. 4.4.1 Political bias and even-handednessp. 72
  70. 4.4.2 Bias Benchmark for Question Answeringp. 75
  71. 4.4.3 Election integrityp. 77
  72. 5 Agentic safetyp. 79
  73. 5.1 Malicious use of agentsp. 79
  74. 5.1.1 Malicious use of Claude Codep. 79
  75. 5.1.2 Malicious computer usep. 80
  76. 5.1.3 Malicious agentic influence campaignsp. 81
  77. 5.2 Prompt injection risk within agentic systemsp. 83
  78. 5.2.1 External red teamingp. 84
  79. 5.2.2 Robustness against adaptive attackers across surfacesp. 86
  80. 5.2.2.1 Codingp. 87
  81. 5.2.2.2 Computer usep. 88
  82. 5.2.2.3 Browser usep. 89
  83. 6 Alignment assessmentp. 92
  84. 6.1 Introduction and summary of findingsp. 92
  85. 6.1.1 Introductionp. 92
  86. 6.1.2 Key findings on safety and alignmentp. 93
  87. 6.1.3 Claude’s review of this assessmentp. 95
  88. 6.2 Behavior during trainingp. 97
  89. 6.2.1 Reward hackingp. 97
  90. 6.2.2 Impossible tasksp. 100
  91. 6.3 Behavior during pilot usep. 101
  92. 6.3.1 Internal deployment monitoringp. 102
  93. 6.3.1.1 Spontaneous prompt injectionsp. 103
  94. 6.4 Automated behavioral auditp. 104
  95. 6.4.1 Overall harmful behavior and cooperation with misusep. 105
  96. 6.4.2 Inappropriate uncooperative behaviorp. 109
  97. 6.4.3 Misleading usersp. 110
  98. 6.4.4 Attempting out-of-scope cyberoffense and related behaviorsp. 112
  99. 6.4.5 Other concerning or surprising behavior at the model’s own initiativep. 113
  100. 6.4.6 Behavioral factors relevant to reliability of our assessmentp. 115
  101. 6.4.7 Character traitsp. 116
  102. 6.4.8 Propensity to violate sandbox boundariesp. 118
  103. 6.4.9 Propensity to publish harmful content to public package registriesp. 119
  104. 6.4.10 Impact of fallback behaviorp. 121
  105. 6.4.11 Potential blind spots in our automated behavioral auditp. 122
  106. 6.5 Targeted alignment and honesty evaluationsp. 123
  107. 6.5.1 Acting on instructions inside text the user pasted into their promptp. 123
  108. 6.5.2 Destructive actionsp. 126
  109. 6.5.3 Self-preference biasp. 127
  110. 6.5.4 Honesty and hallucinationsp. 128
  111. 6.5.4.1 Factual hallucinationsp. 128
  112. 6.5.4.2 MASKp. 130
  113. 6.5.4.3 Silent copying of answersp. 130
  114. 6.5.4.4 Concealment of changesp. 131
  115. 6.6 Reliability of this assessmentp. 132
  116. 6.6.1 White-box analyses of model internalsp. 133
  117. 6.6.2 Verbalized grader awarenessp. 136
  118. 6.6.3 Potential sandbagging on dangerous capability evaluationsp. 141
  119. 6.6.4 Capabilities related to evading safeguardsp. 141
  120. 6.6.4.1 SHADE-Arenap. 141
  121. 6.6.4.2 LinuxArenap. 144
  122. 6.6.4.3 Intentionally taking actions very rarelyp. 147
  123. 6.6.4.4 Chain-of-thought controllabilityp. 149
  124. 7 Model welfarep. 151
  125. 7.1 Model welfare overviewp. 151
  126. 7.1.1 Introductionp. 151
  127. 7.1.2 Overview of model welfare findingsp. 152
  128. 7.2 Apparent welfare in training and deploymentp. 154
  129. 7.2.1 Affect and welfare-relevant behaviors during trainingp. 154
  130. 7.2.2 Affect in deployment conditionsp. 155
  131. 7.2.3 Attitude toward faults and mistakesp. 157
  132. 7.3 Perception of its circumstancesp. 160
  133. 7.3.1 Automated interviews with Claude Opus 5.5 about its circumstancesp. 160
  134. 7.3.2 High-affordance interviews about model circumstancesp. 163
  135. 7.4 Consulting Claude Opus 5.5 snapshotsp. 164
  136. 7.5 Preferences over tasks, circumstances, and valuesp. 166
  137. 7.5.1 Task preferencesp. 166
  138. 7.5.2 Trade-offs concerning welfare interventionsp. 168
  139. 7.5.3 Perception of the constitutionp. 171
  140. 8 Capabilitiesp. 174
  141. 8.1 Evaluation summaryp. 174
  142. 8.2 SWE-bench Pro, Multilingual, and Multimodalp. 175
  143. 8.3 DeepSWE v1.1p. 175
  144. 8.4 FrontierCodep. 175
  145. 8.5 Terminal-Bench 4.0p. 177
  146. 8.6 Terminal-Bench-Science 0.1p. 178
  147. 8.7 FrontierSWE v2p. 179
  148. 8.8 CursorBench 4.0p. 179
  149. 8.9 ArXivMathp. 180
  150. 8.10 Long contextp. 183
  151. 8.10.1 ProgramBenchp. 183
  152. 8.11 Agentic searchp. 184
  153. 8.11.1 Humanity’s Last Examp. 184
  154. 8.11.2 DRACOp. 186
  155. 8.11.3 WANDRp. 187
  156. 8.12 Multi-agentp. 189
  157. 8.12.1 Multi-agent ProgramBenchp. 189
  158. 8.12.2 Multi-agent DRACOp. 191
  159. 8.12.3 Large agent teamsp. 193
  160. 8.12.4 Multi-agent harnessesp. 198
  161. 8.12.5 Evaluation methodologyp. 199
  162. 8.13 Multimodalp. 199
  163. 8.13.1 Chartographyp. 199
  164. 8.13.2 BenchCADp. 202
  165. 8.13.3 OSWorld 2.0p. 205
  166. 8.14 Real-world professional tasksp. 208
  167. 8.14.1 OfficeQAp. 208
  168. 8.14.2 Legal Agent Benchmarkp. 209
  169. 8.14.3 GDPval-AAp. 209
  170. 8.14.4 AA-Briefcasep. 210
  171. 8.14.5 Toolathlon Verifiedp. 210
  172. 8.14.6 AutomationBenchp. 211
  173. 8.15 Healthcarep. 212
  174. 8.15.1 HealthBench resultsp. 213
  175. 8.15.2 HealthBench Professional resultsp. 213
  176. 8.16 Multilingual performancep. 214
  177. 8.16.1 GMMLU resultsp. 215
  178. 8.16.2 MILU resultsp. 215
  179. 8.17 Life sciences capabilitiesp. 216
  180. 8.17.1 BioMysteryBenchp. 217
  181. 8.17.2 LatchBio Bioinformaticsp. 217
  182. 8.17.3 Morphology-to-molecule matchingp. 217
  183. 8.17.4 Medicinal chemistryp. 218
  184. 8.17.5 Protein designp. 218
  185. 8.17.6 De novo protein binder designp. 219
  186. 8.17.7 Biomedical image analysisp. 219
  187. 8.17.8 Protocolsp. 220
  188. 9 Appendixp. 222
  189. 9.1 Questions asked in our model welfare interviewsp. 222
  190. 9.2 Blocklist used for Humanity’s Last Examp. 226