Government · Government · NIST
The TEVV-Athlon Framework for Evaluating AI Systems - ipd
Outline · 41 pages
- Introductionp. 9
- The TEVV-Athlon Frameworkp. 10
- Articulate & Organizep. 11
- Define & Constructp. 13
- Apply & Measurep. 14
- Synthesize & Interrogatep. 15
- Example TEVV-Athlon: The Query-Violation Problemp. 15
- Articulate & Organizep. 15
- Define & Constructp. 18
- Apply & Measurep. 18
- Synthesize & Interrogatep. 20
- General Guidance for a TEVV-Athlonp. 20
- Gathering Appropriate Expertisep. 20
- Checking Basic TEVV-Athlon Requirements, Resources, and Materialsp. 21
- Composing a TEVV Toolboxp. 21
- Checking AI Software and Outputs with Model Testingp. 21
- Testing in More Realistic Settingsp. 22
- Applying Best Practices for Scientific Measurementp. 23
- Validating Measurementp. 23
- Beware of Goodhart's Lawp. 24
- Conclusion and Discussionp. 24