Over the past decade, AI researchers have been studying whether deployed AI systems reflect stakeholders' values and developing strategies for alignment when systems do not. However, AI-related harms persist despite the myriad of benchmarks, evaluations, audits, and algorithms purported to remedy misalignment.

With this thesis, I systematically investigate how existing approaches to AI alignment fall short of achieving their purported goal due to underspecification of sociotechnical design choices. First, I grapple with what “alignment” even means and (should) constitute. One canonical approach to ensuring alignment is to seek participation from relevant stakeholders and groups and elicit their value judgments. While participation is often evoked as a necessary and sufficient determinant for alignment, this thesis illustrates that 1) participation is poorly understood and operationalized by AI researchers and practitioners, and 2) existing participatory mechanisms are insufficient to guarantee alignment. Through literature review and game theoretic analysis, respectively, I provide evidence for these arguments and the corresponding undermining of legitimacy and representativeness of downstream AI models.

Next, I focus on existing metrics and methods for evaluating whether a given AI system is aligned. Namely, I perform a meta-evaluation of red-teaming, one general evaluation approach, and show that red-teaming is an ill-defined process with highly variable inputs and outputs, especially regarding which safety aspects are actually defined and measured. To address this gap, I present design criteria for standardizing red-teaming and ensuring that its outputs are comparable across practices.

Lastly, I conclude this thesis with an analysis of one key red-teaming design choice: the instructions provided to evaluators. By conducting both another review of related work and empirical research through automated red-teaming simulations, I demonstrate that red-teaming instructions vary widely in practice across several dimensions (including word count and risk domain specificity) and that this variation in turn can lead to non-negligible differences in attacks explored and risks realized. Especially as automated evaluations become increasingly common, this analysis shows that, intentionally or not, humans still play a role in shaping outcomes through the task instructions they provide to red-teaming entities, agentic or otherwise. As such, these simulations are a vital step towards better systematization and structuring of red-teaming based on desired goals.

201 pages

Thesis Committee:
Hoda Heidari (Co-Chair)
Zachary C. Lipton (Co-Chair, Carnegie Mellon University / Abridge)
Steven Wu
Hong Shen
Sanmi Koyejo (Stanford University)

Nicolas Christin, Head, Software and Societal Systems Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu