Evaluators and safety organisations · Article · Epoch AI
How many AI agents could we run?
Read at epoch.ai ↗Read at sourceAll rights reserved
Outline · 163 paragraphs
- Overview¶1
- Potential agent capacity and spending¶7
- How we estimate agent capacity¶13
- Estimating agent sessions per GPU today¶24
- What counts as an agent?¶25
- Open models: serving benchmarks measure concurrency directly¶29
- Closed frontier models: concurrency inferred from API spending¶36
- Estimating future capacity from high-bandwidth memory¶56
- HBM is the common bottleneck across AI accelerators¶57
- Newer HBM4 systems should support more agents per gigabyte¶63
- We count all HBM3E and newer memory shipped since 2025¶67
- Shipments through 2027 could support 33–171 million frontier-model agents¶76
- Capacity falls in proportion to spending per agent-hour¶82
- Capacity from open-model benchmarks¶84
- Conclusion: what the totals imply for AI demand¶86
- Acknowledgements¶94
- Appendices¶96
- Appendix A: Additional benchmark and trace data¶97
- Median streaming speed¶98
- The 200 TPS/user sensitivity¶99
- Streaming speed does not include the first-token wait¶100
- TraceLab sample summary¶104
- Token prices used¶109
- Appendix B: Capacity calculations and sensitivities¶114
- Higher revenue/cost multiples¶115
- HBM reconstruction details¶118
- Capacity by hardware uplift¶125
- Capacity by hourly spending and model¶128
- API-equivalent spending¶131
- Accelerator mix and Nvidia’s share of HBM¶136
- Appendix C: HBM specifications¶143
- Appendix D: Code¶146