SafetyPage

Alignment Forum and LessWrong · Forum post · Alignment Forum

Some “coherence arguments” are two different things in a trenchcoat

Steven Byrnes · 2 Nov 2026

Read at alignmentforum.org ↗Read at sourceNo licence statement found

Outline · 74 paragraphs

  1. tl;dr¶2
  2. 1. An example “coherence argument” that I’ll focus on for most of the post¶4
  3. 2. My headline claim¶8
  4. 3. (Backstory of this post)¶21
  5. 4. The two things wearing a trenchcoat¶28
  6. 4.1 “Thing A” is about market participants who wind up with a smaller fraction of the wealth by mistake¶29
  7. 4.2 “Thing B” is about market participants who wind up with a smaller fraction of the wealth on purpose¶32
  8. 5. Why do I feel so strongly that we need to take the trenchcoat off?¶40
  9. 5.1 “Solving for equilibrium”: often a good starting point for Thing A, but rarely for Thing B¶43
  10. 5.2 Applicability to AI¶51
  11. 6. Two other “coherence arguments” that share the same trenchcoat issue¶56
  12. 7. Conclusion¶65