Evaluators and safety organisations · Article · Epoch AI
Can AI automate Epoch?
Read at epoch.ai ↗Read at sourceAll rights reserved
Outline · 110 paragraphs
- Related Work¶10
- Methodology¶13
- Tasks¶14
- Setup¶39
- Evaluation¶43
- Fable 5.1 and GPT-6 Astra are broadly tied in the lead¶50
- Open-weight models struggle where closed-weight models are reliable¶55
- Models fail to learn generalizable patterns from examples¶62
- Models generate promising research directions, but struggle to design experiments that are actually informative¶78
- Models present results driven by flaws in their own experiment setup as key findings¶85
- Fable 5.1 produces outputs with extremely high information density¶91
- Models converge on similar ideas, despite a large idea space¶97
- Limitations¶102
- Conclusion¶108