SafetyPage

Evaluators and safety organisations · Article · Apollo Research

What makes a good monitoring prompt?

Read at apolloresearch.ai ↗Read at sourceNo licence statement found

Outline · 289 paragraphs

  1. What makes a good monitoring prompt?¶2
  2. Summary¶3
  3. Setup¶11
  4. Our prompt format¶21
  5. Baselines¶22
  6. Single Category ablations¶30
  7. Reasoning vs. no reasoning effort¶36
  8. Prompt-side: does externalizing the reasoning matter?¶38
  9. Model-side: can native reasoning replace the prompt scaffold?¶41
  10. Multi-category cuts¶45
  11. Compressing the prompt¶50
  12. Appendix A - per-model and per-FM results¶55
  13. A.1 Baselines per FM × model¶56
  14. A.2 Single-component ablations (per FM)¶63
  15. A.3 Reasoning (per FM)¶66
  16. A.4 Joint cuts (per FM)¶72
  17. A.5 Short prompts (per FM)¶75
  18. Appendix B - Canonical prompt (api_keys), annotated¶78