Augmentative and alternative communication (AAC) devices enable speech-based communication when people cannot use their own physical voice to speak. Speech- generating AAC devices include specialized keyboards and interfaces designed to support individuals in message composition and speech synthesis. Composing a message with an AAC device can take more time than using verbal speech which poses social interaction challenges to AAC users. Augmented communicators (ACs) often fall behind in group conversations when topics move fast or only get asked yes/no questions instead of open-ended ones, limiting the AC’s possible responses. While focusing on improving the rate at which AAC users can communicate and increasing ease to vocabulary access have been an important focus of the field of designing for AAC, this dissertation presents and argues for conversational agency as a new research and design frame to study AAC technology and AAC-based interactions among disabled and non-disabled individuals.

Conversational agency is shaped by social constraints such as when AAC users can speak, what they can say, and who they can address. The first part of this dissertation proposes metrics to measure how conversational agency is expressed within constraints specific to AAC-based interaction. Using agency as a lens for AAC research uncovered new design opportunities for AAC systems that center AAC users’ personal conversational goals. The second part of this thesis explored technologies that can help balance participation in a group and signal to partners that an AC is still making use of their turn to address social constraints related to managing attention and conversational dynamics. The third part of this dissertation explores technologies that can retrieve content from a conversation and clarify misunderstandings to provide ACs with more control over the content of a conversation and the relevance and context of their messages. Through empirical studies, co-design, participatory design, and the development and deployment of different systems, this dissertation investigates how different design materials, such as expressive customized robots and Large Language Models, can support AAC users in easily exercising their conversational agency and help non-AAC users become better communication partners. Through six studies this dissertation contributes Agency in AAC as a new design framework to both further our understanding of AAC users agency and to generate novel, accessible, and agency-increasing communication tools.

144 pages

Thesis Committee:
Henny Admoni (Co-Chair)
Jeffrey P. Bigham (Co-Chair
Jodi Forlizzi
Jeff Higginbotham (University of Buffalo, SUNY)

Brad A. Myers, Head, Human-Computer Interaction Institute
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu