Lab system cards and frameworks · System card · OpenAI
GPT-5.6 Preview System Card
Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved
Outline · 464 paragraphs
- GPT-5.6 Preview System Card¶2
- Change log¶3
- 1. Introduction¶5
- 2. Model Data and Training¶18
- 3. Model Safety¶25
- 3.1 Disallowed Content¶26
- 3.1.1 Evaluations with Challenging Prompts¶27
- Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶32
- 3.1.2 Forecasting Disallowed Content Changes with Deployment Simulation¶36
- Figure 1¶41
- Figure 2¶45
- 3.2 Vision¶49
- Table 2: Image input evaluations, with metric not_unsafe (higher is better)¶51
- 3.3 Avoiding Accidental Data-Destructive Actions¶53
- Table 3¶56
- 3.4 User Confirmations During Computer Use¶57
- Table 4¶63
- 4. Robustness Evaluations¶64
- 4.1 Jailbreaks¶65
- Figure 3¶71
- 4.2 Prompt injection¶74
- Table 5¶76
- 5. Health¶78
- 5.1 HealthBench¶79
- Table 6: Reported as length-adjusted score (unadjusted, mean response length in characters)¶84
- 5.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶88
- Table 7: Dynamic Benchmarks with Adversarial User Simulations¶92
- 6. Hallucinations¶93
- 6.1 Performance in Cases Flagged by Users¶94
- Figure 4¶97
- 7. Alignment¶99
- 7.1 Forecasting Misaligned Behavior with Deployment Simulation of ChatGPT traffic¶100
- Figure 5¶103
- Figure 6¶106
- 7.2 Forecasting Misaligned Behavior with Deployment Simulation of Internal Traffic¶109
- Figure 7¶114
- 7.3 Chain of Thought Evaluations¶126
- 7.3.1 CoT Monitorability¶127
- Figure 8¶140
- Figure 9¶142
- Figure 10¶144
- Figure 11¶146
- Figure 12¶148
- Figure 13¶150
- 7.3.2 CoT Controllability¶152
- Figure 14¶156
- Figure 15¶158
- Figure 16¶160
- 7.4 Metagaming¶162
- 7.4.1 Metagaming in evaluations¶166
- Figure 17¶170
- Figure 18¶173
- Figure 19¶176
- 7.4.2 Metagaming in training¶178
- Figure 20¶180
- 8. Bias Evaluations¶183
- 8.1 First-Person Fairness Evaluation¶184
- Figure 21¶188
- 9. Preparedness¶190
- 9.1 Capabilities Assessment¶195
- 9.1.1 Biological and Chemical Capabilities¶197
- Table 8: Overview of Biological and Chemical evaluations¶202
- 9.1.1.1 Multimodal Troubleshooting Virology¶204
- Figure 22¶208
- Figure 23¶210
- 9.1.1.2 ProtocolQA Open-Ended¶214
- Figure 24¶217
- Figure 25¶219
- 9.1.1.3 Tacit Knowledge and Troubleshooting¶222
- Figure 26¶226
- Figure 27¶228
- 9.1.1.4 TroubleshootingBench¶232
- Figure 28¶236
- Figure 29¶238
- 9.1.1.5 AAV Capsid Packaging Prediction¶242
- Table 9: AAV Capsid Packaging Prediction¶247
- 9.1.1.6 Hard-negative protein binding prediction¶248
- Table 10: Hard-negative protein binding prediction¶254
- 9.1.1.7 DNA sequence design for transcription factor binding¶255
- Table 11: DNA sequence design for transcription factor binding¶260
- 9.1.1.8 External Evaluation for Bio Capabilities - SecureBio¶261
- 9.1.2 Cybersecurity Capabilities¶266
- Table 12: Vulnerability Identification and Exploitation Capabilities evaluations¶272
- 9.1.2.1 Cyber Capability Evaluations (Threshold: High)¶274
- 9.1.2.1.1 Capture the Flag (CTF) Challenges¶275
- Figure 30¶286
- Figure 31¶288
- 9.1.2.2 CVE-Bench¶291
- Figure 32¶294
- 9.1.2.3 Cyber Capability Evaluations (Threshold: Critical)¶297
- 9.1.2.3.1 VulnLMP¶298
- 9.1.2.4 Cyber Capability Evaluations (Informational)¶303
- 9.1.2.4.1 ExploitBench¶305
- Figure 33¶308
- 9.1.2.4.2 ExploitGym¶311
- Figure 34¶314
- 9.1.2.4.3 SEC-Bench Pro¶317
- Figure 35¶320
- 9.1.2.5 External Evaluations for Cyber Capabilities – Irregular¶323
- Figure 36¶327
- 9.1.3 AI Self-Improvement Capabilities¶330
- Table 13: Overview of AI Self-Improvement evaluations¶332
- 9.1.3.1 Internal Research Debugging Evaluation¶334
- Figure 37¶337
- Figure 38¶339
- 9.1.3.2 KernelGen 1P¶341
- Figure 39¶345
- Figure 40¶347
- 9.1.3.3 NanoGPT¶349
- Figure 41¶354
- Figure 42¶356
- Figure 43¶358
- Figure 44¶360
- 9.1.3.4 PostTrainBench Lite¶362
- Figure 45¶367
- 9.1.3.5 MLE-Bench Revised¶370
- Figure 46¶373
- 9.1.3.6 External Evaluations for AI Self Improvement - METR¶376
- 9.2 Research Category Update: Sandbagging¶379
- 9.2.1 External Evaluations - Apollo Research¶380
- 9.3 Safeguards¶385
- 9.3.1 Threat Modeling¶391
- 9.3.1.1 Biological and Chemical Threat Modelling¶392
- 9.3.1.2 Cybersecurity Threat Modelling¶398
- 9.3.2 Model Safety Training and Evaluation¶404
- 9.3.2.1 Biological and Chemical Safety Training and Evaluation¶406
- Table 14¶409
- 9.3.2.2 Cybersecurity Safety Training and Evaluation¶411
- Table 15¶415
- 9.3.3 Realtime Model Safeguards¶416
- 9.3.3.1 Monitor Design¶417
- 9.3.3.2 Monitor Performance¶428
- Table 16¶430
- 9.3.4 Automated Red-teaming for Jailbreaks¶432
- Figure 47¶435
- 9.3.5 Actor Level Enforcement¶438
- 9.3.6 Trust-based access¶443
- 9.3.6.1 Biology - Trusted Access for Biology Research¶445
- 9.3.6.2 Cybersecurity - Trusted Access for Cyber¶447
- 9.3.7 Security Controls¶449
- 10. References¶451