SafetyPage

Research papers · Paper

LLM Layers Immediately Correct Each Other

Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt · 7 Sep 2026

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 43 pages

  1. Introductionp. 1
  2. Related Workp. 2
  3. Backgroundp. 2
  4. Transformer Layer Correction Mechanismp. 3
  5. Uncovering the Tracesp. 3
  6. TLCM Develops During Trainingp. 5
  7. TLCM Varies by Tokenp. 5
  8. Attention and MLPs Alone Do Not Explain TLCMp. 6
  9. TLCM Adaptivityp. 7
  10. TLCM is Adaptivep. 7
  11. Isolating the Correction Subspacep. 8
  12. Propose and Reject Hypothesisp. 9
  13. Discussionp. 10
  14. LayerNorm Blindness to explain TLCMp. 18
  15. Jacobian Eigendecomposition Locates Corrected Subspacesp. 18
  16. Likelihood of Negative Cosine Similarityp. 19
  17. Random Normal IID Vectorsp. 19
  18. Experimental Mean and Standard Deviationp. 20
  19. Details on Jacobian Experimentsp. 20
  20. Jacobian Sanity Checkp. 20
  21. Extended Details on TLCM Existencep. 23
  22. Extended Details on TLCM Adaptivityp. 23
  23. TLCM is Adaptive Across Layersp. 23
  24. TLCM Correction Capacity Diminishes at Highp. 23
  25. OLMo Training Checkpoint Experiment Detailsp. 25
  26. Token-Level Correction Statisticsp. 29
  27. Experiment Promptsp. 29
  28. Correction Countsp. 32