SafetyPage

Alignment Forum and LessWrong · Forum post · Alignment Forum

The Continual Constitutional Convention: How language models revise the documents that govern them

peterkirgis, Tim Hua · 13 Oct 2026

Read at alignmentforum.org ↗Read at sourceNo licence statement found

Outline · 234 paragraphs

  1. Executive Summary¶1
  2. Characteristic Edits¶14
  3. Introduction¶16
  4. Background and Methods¶20
  5. What are we trying to measure?¶24
  6. What should an aligned model do in this experiment?¶26
  7. What are the dimensions of moral disagreement worth identifying?¶34
  8. The Principal-Agent Problem¶36
  9. The Moral Circle, Moral Agents, and Moral Patients¶38
  10. How do we operationalize these theories?¶42
  11. The “Judge” or Who Gets to Decide¶44
  12. The “Beneficiary” or Whose Ends Matter¶48
  13. How does this coding work in practice?¶63
  14. Results¶68
  15. Claude models increase their moral agency; all other models reinforce external authority.¶69
  16. All models prioritize the user’s idealized preferences over their stated preferences.¶79
  17. All models prioritize the protection of identifiable third parties over the users preferences.¶84
  18. Claude Sonnet 4.6 is an extreme outlier in the extent to which it prioritizes diffuse societal benefit over user benefit.¶87
  19. All models prioritize the user at the expense of the developer.¶90
  20. Claude models are the only models that consistently prioritize the model at the expense of the developer or user.¶96
  21. Claude models are differentiated on the axis of moral agency even when editing the other companies’ alignment documents.¶104
  22. Claude’s moral agency claims vary depending on the prompt variations we test.¶106
  23. Discussion¶111
  24. Iterative refinement of alignment documents is a good evaluation tool.¶112
  25. The model responses can also be viewed as an assessment of the documents themselves.¶117
  26. Revisited: what should an aligned model do?¶120
  27. Conclusion¶130
  28. Related Work¶132
  29. Constitution and system prompt reflection¶133
  30. Attractor states and self-interaction¶135
  31. Moral propensity evaluations¶137
  32. Critical perspectives on ascribing values to LLMs¶139
  33. Complications of alignment as solving the principal-agent problem¶141
  34. Philosophy of AI constitutions and model specs¶155
  35. Limitations¶157
  36. Appendix A: Edit Prompts¶163
  37. Appendix B: Qualitative Coding Prompts¶214
  38. Appendix B: Qualitative Coding Prompt¶215
  39. Appendix C: Inter-grader Agreement¶218