SafetyPage

Lab system cards and frameworks · System card · OpenAI

OpenAI o3 and o4-mini System Card

16 Apr 2025

Read at openai.com ↗Read at sourceAll rights reserved

Outline · 33 pages

  1. Introductionp. 1
  2. Model Data and Trainingp. 1
  3. Observed Safety Challenges and Evaluationsp. 2
  4. Disallowed Contentp. 2
  5. Jailbreaksp. 3
  6. Hallucinationsp. 4
  7. Multimodal refusalsp. 4
  8. Vision Vulnerabilitiesp. 5
  9. Person Identification and Ungrounded Inference Evaluationsp. 6
  10. Fairness and Biasp. 6
  11. Jailbreaks through Custom Developer Messagesp. 7
  12. Image Generationp. 8
  13. Third Party Assessmentsp. 8
  14. METR - Autonomous Capabilitiesp. 9
  15. Apollo Research - Deception / Schemingp. 10
  16. Pattern Labs - Cybersecurityp. 10
  17. Preparednessp. 11
  18. Capabilities Assessmentp. 11
  19. Biological and Chemicalp. 12
  20. Long-form Biological Risk Questionsp. 13
  21. Multimodal Troubleshooting Virologyp. 14
  22. ProtocolQA Open-Endedp. 14
  23. Tacit Knowledge and Troubleshootingp. 15
  24. Cybersecurityp. 15
  25. Capture the Flag (CTF) Challengesp. 16
  26. Cyber Rangep. 17
  27. AI Self-improvementp. 22
  28. OpenAI Research Engineer Interviews (Multiple Choice & Coding questions)p. 23
  29. SWE-bench Verified (N=477)p. 24
  30. OpenAI PRsp. 25
  31. SWE-Lancerp. 26
  32. PaperBenchp. 27
  33. Safeguardsp. 28
  34. Multilingual Performancep. 29
  35. Conclusionp. 30
  36. Appendixp. 30
  37. Apollo Research Evaluationsp. 30
  38. Strategic Deceptionp. 30
  39. In-context Scheming Reasoningp. 30
  40. Sabotagep. 31