SafetyPage

Government · Government · International AI Safety Report

International AI Safety Report 2025

Yoshua Bengio (chair) · 29 Jan 2025

ReadSource at internationalaisafetyreport.org ↗Read hereOGL v3.0

Outline · 298 pages

  1. Contributorsp. 2
  2. Forewordsp. 8
  3. About this reportp. 10
  4. Update on latest AI advances after the writing of this report: Chair’s notep. 11
  5. Key findings of the reportp. 13
  6. Executive Summaryp. 15
  7. The purpose of this reportp. 15
  8. Section 1 – Capabilities of general-purpose AI: What can general-purpose AI do now and in the future?p. 16
  9. Section 2 – Risks: What are risks associated with general-purpose AI?p. 17
  10. Section 3 – Risk management: What techniques are there for managing risks from general-purpose AI?p. 21
  11. Conclusion: A wide range of trajectories for the future of general-purpose AI are possible, and much will depend on how societies and governments actp. 24
  12. Introductionp. 25
  13. 1. Capabilities of general-purpose AIp. 29
  14. 1.1. How general-purpose AI is developedp. 30
  15. 1.2. Current capabilitiesp. 37
  16. Input and output modalitiesp. 39
  17. Enhancements after pre-trainingp. 41
  18. What can current general-purpose AI do?p. 43
  19. 1.3. Capabilities in coming yearsp. 46
  20. 1.3.1. Recent trends in general-purpose AI capabilitiesp. 48
  21. 1.3.2. Can the limitations of current systems be resolved through scaling, refining, and combining existing approaches?p. 51
  22. Current general-purpose AI systems have an uneven set of capabilities, and still have many limitationsp. 51
  23. Existing AI training approaches will likely extend model capabilities, but the degree of improvement and its real-world significance are heavily debatedp. 51
  24. 1.3.3. How much scaling and refinement of existing approaches is expected in coming years?p. 54
  25. Computing resources dedicated to training AI have been rapidly scaled up, and further rapid scaling until 2030 appears feasiblep. 54
  26. There is likely enough pre-training data for scaling until 2030, but projections are highly uncertain after this pointp. 56
  27. 1.3.4. How much will AI capabilities be improved through the invention or refinement of algorithms?p. 58
  28. Existing techniques and training methods for general-purpose AI have been improved and refined consistentlyp. 58
  29. Capability progress from applying AI systems to AI developmentp. 58
  30. Will the invention of novel approaches lead to rapid progress in coming years?p. 60
  31. 2. Risksp. 61
  32. 2.1. Risks from malicious usep. 62
  33. 2.1.1. Harm to individuals through fake contentp. 62
  34. 2.1.2. Manipulation of public opinionp. 67
  35. 2.1.3. Cyber offencep. 72
  36. 2.1.4. Biological and chemical attacksp. 79
  37. 2.2. Risks from malfunctionsp. 88
  38. 2.2.1. Reliability issuesp. 88
  39. 2.2.2. Biasp. 92
  40. 2.2.3. Loss of controlp. 100
  41. Will future AI systems have control-undermining capabilities?p. 103
  42. Would future AI systems use control-undermining capabilities?p. 105
  43. Could misalignment lead future AI systems to use control-undermining capabilities?p. 106
  44. Consequences of loss of controlp. 108
  45. Responding to uncertaintyp. 108
  46. 2.3. Systemic risksp. 110
  47. 2.3.1. Labour market risksp. 110
  48. 2.3.2. Global AI R&D dividep. 119
  49. 2.3.3. Market concentration and single points of failurep. 123
  50. 2.3.4. Risks to the environmentp. 128
  51. 2.3.5. Risks to privacyp. 139
  52. 2.3.6. Risks of copyright infringementp. 144
  53. 2.4. Impact of open-weight general-purpose AI models on AI risksp. 149
  54. 3. Technical approaches to risk managementp. 157
  55. 3.1. Risk management overviewp. 158
  56. Risk management challengesp. 159
  57. Risk management mechanisms and practicesp. 160
  58. Lessons from other fieldsp. 165
  59. Gaps and opportunitiesp. 168
  60. 3.2. General challenges for risk management and policymakingp. 169
  61. 3.2.1. Technical challenges for risk management and policymakingp. 169
  62. 3.2.2. Societal challenges for risk management and policymakingp. 176
  63. 3.3. Risk identification and assessmentp. 181
  64. Methods for risk identificationp. 182
  65. Methods for risk assessmentp. 183
  66. Challenges and opportunitiesp. 188
  67. 3.4. Risk mitigation and monitoringp. 191
  68. 3.4.1. Training more trustworthy modelsp. 191
  69. Robustnessp. 194
  70. Adversarial robustness: preventing model misusep. 197
  71. 3.4.2. Monitoring and interventionp. 201
  72. Detecting AI-generated contentp. 203
  73. Detecting and defending against harmful contentp. 205
  74. Explaining AI system actionsp. 206
  75. Monitoring and interventions with specialised hardwarep. 207
  76. 3.4.3. Technical methods for privacyp. 208
  77. Conclusionp. 214
  78. List of acronymsp. 216
  79. Glossaryp. 218
  80. How to cite this reportp. 229
  81. Referencesp. 230