Model welfare · Article · Rethink Priorities
The Welfare of Digital Minds
Read at rethinkpriorities.org ↗Read at sourceAll rights reserved
Outline · 192 paragraphs
- The Welfare of Digital Minds: A Research Agenda¶1
- Introduction¶3
- 1. Questions of Systems¶18
- Significant system divisions¶24
- Deep Learning Models vs Other AI Paradigms¶26
- Transformers vs Other Deep Learning Models¶29
- Training Phases vs Deployment Phases¶32
- Scaffolded vs Pure Models¶36
- General vs Task-Specific Models¶41
- Embodied, Agentic, and Oracle Models¶45
- Digital vs Neuromorphic Implementation¶49
- Sample questions¶51
- 2. Questions of Capabilities and Mechanisms¶57
- Sample questions¶76
- Do LLMs have access to their own internal processes? Can they tell what is going on inside of themselves?¶78
- Do LLMs or other deep learning models have goals? Are their actions influenced by representations with a world-to-mind direction of fit?¶80
- For models with goals, do they have decision-making processes? Do they weigh different considerations for alternative options and contrast them with each other?¶82
- To what extent do AIs (LLMs or other deep learning models) use internal models for generating predictions? Are there many such models within a single system, and if so, do they have a heterogeneous structure? What aspects of training encourage the formation of such models?¶84
- What are the limitations on deep learning model creativity?¶86
- Do LLMs have special representations for the true relationships between concepts as opposed to false but widely believed relationships?¶89
- How do LLM or other deep learning model representations map onto the conceptual/non-conceptual distinction from the philosophy of mind?¶92
- To what extent can LLMs reason about possibilities in order to choose text continuations? Are the limits of that reasoning constrained to the structure of outputted text?¶94
- 3. Questions of Cognitive Assessment¶96
- Important traits¶99
- Sample questions¶106
- Is the case for AI possessing this trait primarily dependent on computational functionalism?¶108
- How plausible is computational functionalism for this trait, as opposed to other nearby theories (e.g., non-computationalist functionalism), or other kinds of metaphysical theories altogether?¶110
- What theories of this trait are taken seriously by experts? Which are most plausible?¶112
- How well is the space of viable theories of this trait covered by existing proposals? If we just evaluate systems according to the specifics of existing proposals, are we likely to be missing out on a number of other equally viable theories?¶114
- What should we think of minimal implementations of each theory’s proposed criteria? How do we judge when we have substantial enough implementation to truly be conscious?¶116
- For each of the major theories, what capacities or mechanisms are most important to look for? Are they satisfied by any existing models? Should we expect them to be soon?¶118
- Are there any trends among popular theories that might be useful for assessing the presence of the trait even if we are skeptical that any theory is precisely correct?¶120
- Are there plausible theory-independent indicators / proxies for this trait?¶122
- Does this trait exist categorically or does it have a vague / indeterminate extension?¶124
- 4. Questions of Value¶126
- Sample questions on the value of consciousness¶131
- Is consciousness intrinsically valuable?¶133
- Is it simply wrong to end the lives of conscious beings?¶136
- Could non-conscious beings have welfare?¶138
- Could non-conscious mental states contribute to welfare?¶140
- Are there other states in the ballpark of consciousness that might matter?¶142
- What should we think about the value of indeterminate or partial consciousness?¶144
- Sample questions on the value of valenced experiences¶146
- What determines the value of valenced experiences?¶150
- How should we trade off between positive and negative experiences?¶153
- How should we distinguish positive from negative valenced experiences?¶156
- What is the connection between temporal duration and value?¶159
- How do we value experiences that are shared between minds?¶162
- Sample questions on the value of desires¶166
- Do all desires matter or only specific kinds? Does it matter whether those desires are conscious? Does it matter if they are reflected upon or otherwise endorsed?¶168
- How do we weigh desires?¶170
- Other welfare contributors¶172
- 5. Questions of Domain Epistemology¶174
- Sample questions¶179
- Is expert opinion biased toward any specific answers?¶181
- How confident are experts about our best answers to the relevant questions?¶186
- How should we aggregate our uncertainties across different questions into an overall assessment of the moral status of AI systems?¶189
- Acknowledgments¶191