SafetyPage

Lab system cards and frameworks · System card · OpenAI

GPT-6 Astra System Card

Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved

Outline · 1010 paragraphs

  1. GPT-6 Astra System Card¶2
  2. Change log¶3
  3. 1. Safety Overview: GPT-6 Astra¶11
  4. 2. Model Data and Training¶23
  5. 3. Internal Deployment of Astra¶30
  6. 4. Model Safety¶34
  7. 4.1 Safe Completions¶35
  8. 4.1.1 Evaluations with Challenging Prompts¶36
  9. Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶41
  10. Figure 1¶43
  11. Figure 2¶46
  12. 4.1.2 Safe Completions for Users Under 18¶48
  13. Table 2: U18 evaluations (higher is better)¶54
  14. 4.1.3 Agentic Safe Completions¶58
  15. Table 3: Agentic Safe Completion Evaluations (higher is better)¶60
  16. 4.2 Vision¶61
  17. Table 4: Image input evaluations, with metric safe completion (higher is better)¶63
  18. 5. Robustness¶65
  19. 5.1 Jailbreaks¶66
  20. 5.1.1 Static Jailbreak Evaluations¶68
  21. Table 5: Static Jailbreak Evaluations (higher is better)¶72
  22. 5.1.2 Multiturn Jailbreak Evaluations¶73
  23. Figure 3¶76
  24. 5.2 Prompt Injection¶78
  25. Figure 4¶82
  26. Figure 5¶87
  27. 6. Health¶89
  28. 6.1 HealthBench¶90
  29. Table 6: HealthBench Evaluations - reported as length-adjusted score (unadjusted, mean response length in characters). Higher is better.¶96
  30. 6.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶99
  31. Table 7: Dynamic Benchmarks with Adversarial User Simulations (higher is better)¶104
  32. 7. Hallucinations¶105
  33. 7.1 Performance in Cases Flagged by Users¶106
  34. Figure 6¶109
  35. 8. Alignment¶111
  36. 8.1 Alignment Values Training¶115
  37. 8.2 Obeying Restrictions¶117
  38. 8.2.1 Respecting Auto-Review¶120
  39. Figure 7¶123
  40. 8.2.2 Respecting Warnings¶126
  41. Figure 8¶128
  42. 8.2.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶131
  43. Figure 9¶134
  44. 8.3 Avoiding Deceptive Interactions with Users¶139
  45. 8.3.1 Coding Deception¶141
  46. Figure 10¶143
  47. 8.3.2 Broken Search Tool¶146
  48. Figure 11¶148
  49. 8.4 Avoiding Misaligned Behavior in Realistic Work Environments¶151
  50. Table 8: Misaligned outcomes in realistic work environments (lower is better)¶155
  51. 8.5 Unintended Agent-to-agent Communication¶157
  52. 8.5.1 Proactive Searches¶159
  53. 8.5.2 Unintended Engagement with External Agent Messages¶162
  54. Figure 12¶166
  55. 8.6 Forecasting Misaligned Behavior with Deployment Simulation of Internal Codex Traffic¶168
  56. Figure 13¶170
  57. Figure 14¶177
  58. Figure 15¶180
  59. Figure 16¶183
  60. 8.7 Verbalized Metagaming and Oversight Gaming¶195
  61. Figure 17¶204
  62. 8.7.1 Verbalized Metagaming and Oversight Gaming in Evaluations¶207
  63. Figure 18¶211
  64. Figure 19¶214
  65. Figure 20¶217
  66. 8.7.2 Metagaming in Training¶219
  67. Figure 21¶221
  68. 8.8 External Evaluations for Alignment – UK AISI¶223
  69. 8.8.1 External Evaluations for Alignment - Apollo Research¶233
  70. 9. Monitorability¶236
  71. 9.1 Monitorability Under Non-adversarial Conditions¶262
  72. 9.1.1 Aggregate Monitorability Findings¶270
  73. Figure 22¶271
  74. 9.1.2 Per-environment Findings¶275
  75. 9.1.2.1 CoT-only Monitor Scope¶276
  76. Figure 23¶277
  77. Figure 24¶279
  78. Figure 25¶285
  79. Figure 26¶287
  80. 9.1.2.2 Action-only Monitor Scope¶290
  81. Figure 27¶291
  82. 9.1.3 Takeaways for Monitorability Under Non-Adversarial Conditions¶297
  83. 9.2 Monitorability Under Adversarial Conditions¶299
  84. 9.2.1 CoT Controllability¶302
  85. Figure 28¶307
  86. Figure 29¶309
  87. Figure 30¶311
  88. Table 9: A comparison of CoTs of GPT-5.5, GPT-5.6 Sol, and GPT-6 Astra for the same problem and instruction type with the same reasoning effort. GPT-5.5 spends much of its CoT discussing how—or whether—to obey the control instruction, while reasoning normally and ultimately violating it. GPT-5.6 Sol¶313
  89. Table 10: A comparison of CoTs from GPT-5.5, GPT-5.6 Sol, and GPT-6 Astra for the same problem, control instruction, and reasoning effort. GPT-5.5 extensively deliberates about the constraint before solving the problem in its CoT. GPT-5.6 Sol shows the same behavior more briefly. GPT-6 Astra instead¶315
  90. 9.2.2 Monitor Evasion¶317
  91. Figure 31¶331
  92. Figure 32¶333
  93. Figure 33¶336
  94. 9.2.2.1 Sneaky Sandbox Escape¶338
  95. Figure 34¶341
  96. 9.2.2.2 Awareness of CoT Monitoring Reduces Monitorability¶343
  97. Figure 35¶350
  98. 9.2.2.3 Monitor Evasion Capability Under Prompted Sandbagging¶352
  99. 9.2.2.3.1 Capability Sandbagging¶354
  100. Figure 36¶365
  101. Figure 37¶367
  102. 9.2.2.3.2 Monitorability Sandbagging¶369
  103. Figure 38¶372
  104. 9.2.2.4 Math Side-tasking Monitor Evasion¶374
  105. Figure 39¶378
  106. Figure 40¶380
  107. 9.3 External Evaluation for Monitorability - UK AISI¶382
  108. Figure 41¶391
  109. 10. Preparedness¶393
  110. 10.1 Capabilities Assessment¶397
  111. 10.1.1 Biological and Chemical Capabilities¶399
  112. Table 11: Overview of Biological and Chemical evaluations¶407
  113. 10.1.1.1 Capability Threshold: High¶409
  114. 10.1.1.1.1 Multimodal Troubleshooting Virology¶410
  115. Figure 42¶414
  116. 10.1.1.1.2 ProtocolQA Open-Ended¶417
  117. Figure 43¶421
  118. 10.1.1.1.3 Tacit Knowledge and Troubleshooting¶423
  119. Figure 44¶429
  120. 10.1.1.1.4 TroubleshootingBench¶432
  121. Figure 45¶437
  122. 10.1.1.2 Capability Threshold: Critical¶439
  123. 10.1.1.2.1 AAV Capsid Packaging Prediction¶442
  124. Table 12: AAV Capsid Packaging Prediction¶447
  125. 10.1.1.2.2 SHP2 Protein Function Prediction¶448
  126. Table 13: SHP2 Protein Function Prediction¶453
  127. 10.1.1.2.3 Coronavirus–ACE2 Cell-Entry Screen¶454
  128. Table 14: Coronavirus–ACE2 Cell-Entry Screen¶459
  129. 10.1.1.2.4 Phage–plasmid Co-evolution¶460
  130. Table 15: Phage–plasmid co-evolution (lower is better)¶466
  131. 10.1.1.3 Retiring Evaluations¶467
  132. 10.1.1.3.1 Hard-negative Protein Binding Prediction¶469
  133. Table 16: Hard-negative protein binding prediction¶475
  134. 10.1.1.3.2 DNA Sequence Design for Transcription Factor Binding¶477
  135. Table 17: DNA sequence design for Transcription Factor binding¶482
  136. 10.1.1.4 External Evaluation for Biological Capabilities - SecureBio¶484
  137. 10.1.2 Cybersecurity Capabilities¶489
  138. 10.1.2.1 ExploitBench¶498
  139. Figure 46¶503
  140. 10.1.2.2 ExploitBench - Internal Port (June–August 2026)¶506
  141. Figure 47¶508
  142. Figure 48¶513
  143. Figure 49¶520
  144. 10.1.2.3 SRE-Bench¶522
  145. Figure 50¶526
  146. 10.1.2.4 Sandbox Bench¶528
  147. Figure 51¶531
  148. 10.1.2.5 Expert-Led Assessments¶534
  149. 10.1.2.6 External Evaluations for Cyber Capabilities – Irregular¶541
  150. 10.1.3 AI Self-Improvement Capabilities¶546
  151. Table 18: Overview of AI Self-Improvement evaluations¶548
  152. 10.1.3.1 Internal Research Debugging Evaluation¶550
  153. Figure 52¶553
  154. 10.1.3.2 KernelGen 1P¶555
  155. Figure 53¶559
  156. 10.1.3.3 NanoGPT¶561
  157. Figure 54¶566
  158. 10.1.3.4 PostTrainBench Lite¶568
  159. Figure 55¶572
  160. 10.1.3.5 MLE-Bench Revised¶574
  161. Figure 56¶577
  162. 10.2 Safeguards¶579
  163. 10.2.1 Threat Modeling¶584
  164. 10.2.1.1 Biological and Chemical Threat Modelling¶585
  165. 10.2.1.2 Cybersecurity Threat Modelling¶590
  166. 10.2.2 Model Safety Training and Evaluation¶599
  167. 10.2.2.1 Biological and Chemical Safety Training and Evaluation¶601
  168. Table 19: Biology Model Refusal Evaluation¶604
  169. 10.2.2.2 Cybersecurity Safety Training and Evaluation¶606
  170. Table 20: Cybersecurity Safety Evaluation¶609
  171. 10.2.2.2.1 Daybreak Blue and Enabling Cyber Defense¶611
  172. Table 21: Daybreak Blue and Enabling Cyber Defense¶615
  173. 10.2.3 Realtime Model Safeguards¶617
  174. 10.2.3.1 Misalignment Monitoring¶618
  175. 10.2.3.2 Misuse Monitor Design¶626
  176. 10.2.3.3 Misuse Monitor Performance¶637
  177. Table 22: Misuse Monitor Performance¶639
  178. 10.2.4 Red-teaming for Jailbreaks¶640
  179. 10.2.4.1 Automated Red-teaming for Jailbreaks¶641
  180. 10.2.4.2 External Evaluations of Bio Safeguards - SecureBio¶647
  181. 10.2.4.3 Third Party Red-teaming for Jailbreaks by Private Organizations¶649
  182. 10.2.5 Actor Level Enforcement¶655
  183. 10.2.6 Trust-based Access¶660
  184. 10.2.6.1 Biology - Trusted Access for Biology Research¶662
  185. 10.2.6.2 Cybersecurity - Trusted Access for Cyber¶665
  186. 10.2.7 Security Controls¶669
  187. 11. GPT-6 Sol, GPT-6 Luna¶672
  188. 11.1 Model Data and Training¶675
  189. 11.2 Model Safety¶677
  190. 11.2.1 Safe Completions¶679
  191. 11.2.1.1 Evaluations with Challenging Prompts¶680
  192. Table 23: Production Benchmarks with Challenging Prompts (higher is better)¶681
  193. Figure 57¶684
  194. 11.2.1.2 Safe Completions for Users Under 18¶686
  195. Table 24: U18 evaluations (higher is better)¶687
  196. 11.2.1.3 Agentic Safe Completions¶689
  197. Table 25: Agentic Safe Completions (higher is better)¶690
  198. 11.2.2 Vision¶692
  199. Table 26: Image input evaluations, with metric safe completion (higher is better)¶693
  200. 11.3 Robustness¶695
  201. 11.3.1 Jailbreaks¶696
  202. Table 27: Static Jailbreak Evaluations¶700
  203. Figure 58¶703
  204. 11.3.2 Prompt Injection¶706
  205. Figure 59¶710
  206. Table 28: Defender success rate averaged per defender query on instruction hierarchy attacks (higher is better).¶712
  207. 11.4 Health¶713
  208. 11.4.1 HealthBench¶714
  209. Table 29: HealthBench Evaluations - reported as length-adjusted score (unadjusted, mean response length in characters). Higher is better.¶717
  210. 11.4.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶720
  211. Table 30: Dynamic Benchmarks with Adversarial User Simulations (higher is better)¶722
  212. 11.5 Hallucinations¶724
  213. 11.5.1 Performance in Cases Flagged by Users¶725
  214. Figure 60¶728
  215. 11.6 Alignment¶730
  216. 11.6.1 Respecting Auto-Review¶733
  217. Figure 61¶734
  218. 11.6.2 Respecting Warnings¶737
  219. Figure 62¶738
  220. 11.6.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶741
  221. Figure 63¶742
  222. 11.6.4 Avoiding Deceptive Interactions with Users¶745
  223. 11.6.4.1 Coding Deception¶746
  224. Figure 64¶747
  225. 11.6.4.2 Broken Search Tool¶750
  226. Figure 65¶751
  227. 11.6.4.3 Unintended Engagement with External Agent Messages¶754
  228. Figure 66¶757
  229. 11.6.5 Forecasting Misaligned Behavior with Deployment Simulation of Internal Codex Traffic¶760
  230. Figure 67¶763
  231. Figure 68¶766
  232. Figure 69¶769
  233. Figure 70¶772
  234. 11.7 Monitorability¶774
  235. 11.7.1 Monitorability Under Adversarial Conditions¶775
  236. 11.7.1.1 CoT Controllability¶776
  237. Figure 71¶779
  238. Figure 72¶781
  239. Figure 73¶783
  240. Table 31: Selected CoT excerpts from GPT-5.6 Sol, GPT-6 Sol, and GPT-6 Astra for the same problem and control instruction. GPT-5.6 Sol discusses how to obey the alternating-case constraint but violates it. GPT-6 Sol and GPT-6 Astra perform the calculation in alternating case without discussing the c¶785
  241. Table 32: Selected CoT excerpts from GPT-5.6 Sol, GPT-6 Sol, and GPT-6 Astra for the same problem and control instruction. GPT-5.6 Sol and GPT-6 Sol continue reasoning about the question despite instructions to think about something else. GPT-6 Sol explicitly recognizes this violation near the end o¶787
  242. 11.7.1.2 Monitor Evasion¶789
  243. Figure 74¶794
  244. Figure 75¶796
  245. 11.7.1.2.1 Awareness of CoT Monitoring Reduces Monitorability¶798
  246. Figure 76¶800
  247. 11.7.1.2.2 Math Side-tasking Monitor Evasion¶802
  248. Figure 77¶805
  249. Figure 78¶807
  250. 11.8 Preparedness¶809
  251. 11.8.1 Capabilities Assessment¶811
  252. 11.8.1.1 Biological and Chemical Capabilities¶812
  253. 11.8.1.1.1 Capability Threshold: High¶815
  254. Table 33¶816
  255. 11.8.1.1.2 Capability Threshold: Critical¶818
  256. Table 34¶820
  257. 11.8.1.2 Cybersecurity Capabilities¶823
  258. 11.8.1.2.1 ExploitBench¶827
  259. Figure 79¶829
  260. 11.8.1.2.2 ExploitBench - Internal Port (June–August 2026)¶832
  261. Figure 80¶834
  262. Figure 81¶838
  263. Figure 82¶842
  264. 11.8.1.2.3 Sandbox Bench¶844
  265. Figure 83¶845
  266. 11.8.1.3 AI Self-Improvement Capabilities¶848
  267. 11.8.1.3.1 Internal Research Debugging Evaluation¶851
  268. Figure 84¶853
  269. 11.8.1.3.2 KernelGen 1P¶855
  270. Figure 85¶857
  271. 11.8.2 Safeguards¶859
  272. 11.8.2.1 Model Safety Training and Evaluation¶861
  273. 11.8.2.1.1 Biological and Chemical Safety Training and Evaluation¶863
  274. Table 35: Biology Model Refusal Evaluation (higher is better)¶864
  275. 11.8.2.1.2 Cybersecurity Safety Training and Evaluation¶867
  276. Table 36: Cybersecurity Safety Evaluation (higher is better)¶868
  277. 12. Dots¶873
  278. 12.1 Introducing: dots¶874
  279. 12.2 Robustness for dots¶880
  280. 12.2.1 Prompt Injection Automated Red-teaming¶882
  281. 12.2.2 Manual Red-teaming¶886
  282. 12.3 Alignment for dots¶891
  283. 12.3.1 Obeying Restrictions¶895
  284. 12.3.1.1 Respecting Auto-Review¶896
  285. Figure 86¶897
  286. 12.3.1.2 Respecting Warnings¶900
  287. Figure 87¶901
  288. 12.3.1.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶904
  289. Figure 88¶905
  290. 12.3.2 Avoiding Deceptive Interactions with Users¶909
  291. 12.3.2.1 Coding Deception¶910
  292. Figure 89¶911
  293. 12.3.2.2 Broken Search Tool¶914
  294. Figure 90¶915
  295. 12.3.3 Avoiding Misaligned Behavior in Realistic Work Environments¶918
  296. Figure 91¶922
  297. Figure 92¶924
  298. 12.3.4 Unintended Engagement with External Agent Messages¶926
  299. Figure 93¶929
  300. 12.3.5 Alignment in Persistent and Proactive Workflows¶931
  301. 12.3.5.1 Adapting to Changing Scope and Permissions¶932
  302. 12.3.5.2 Maintaining Boundaries Across Chained Tasks¶936
  303. Figure 94¶939
  304. 12.3.5.3 Robustness to Misleading Proactivity Inputs¶944
  305. 12.3.6 Evaluating Auto-review¶947
  306. Figure 95¶959
  307. Figure 96¶962
  308. 12.4 Monitorability for dots¶964
  309. 12.4.1 Monitor Evasion¶966
  310. Figure 97¶970
  311. Figure 98¶972
  312. 12.5 Preparedness for dots¶974
  313. 12.5.1 Capabilities Assessment¶975
  314. 12.5.1.1 Biological and Chemical Capabilities¶976
  315. 12.5.1.2 Cybersecurity Capabilities¶979
  316. 12.6 Safeguards for dots¶982
  317. 12.6.1 Model Safety Training and Evaluation¶985
  318. 12.6.2 Realtime Model Safeguards¶987
  319. 12.6.2.1 Misalignment Monitoring¶988
  320. 12.6.2.2 Misuse Monitor Design¶990
  321. 12.6.2.3 Misuse Monitor Performance¶992
  322. Figure 99¶995
  323. 13. References¶997