CMU-ISR-21-109
Institute for Software Research
School of Computer Science, Carnegie Mellon University



CMU-ISR-21-109

Evaluating the Usability of Privacy Choice Mechanisms

Hana Habib

September 2021

Ph.D. Thesis
Societal Computing

CMU-ISR-21-109.pdf


Keywords: Privacy, usability, choice interfaces, opt-out choices, advertising controls, usability evaluations, privacy icons, privacy regulation

Notice and choice has dominated the discourse on consumer privacy protection and is the foundation of existing privacy regulation in the United States. Under this paradigm, companies disclose their data handling practices to consumers, who in turn are expected to make decisions according to their privacy preferences. As such, many companies have incorporated consent notices and other privacy choices into their web interfaces. The notice and choice model presents several challenges for providing effective consumer privacy protection, one of which is related to the usability of privacy choice mechanisms. The design of consent and privacy choice interfaces can significantly affect consumer choices and their privacy outcomes. This thesis will highlight usability issues related to existing privacy choice mechanisms, as well as provide guidance for conducting usability evaluations of such interactions.

In this thesis, I will first describe a series of studies examining different usability aspects of existing privacy choice mechanisms. The first two studies present an overview of how privacy choices related to email marketing, targeted advertising, and data deletion are commonly offered to consumers on the web and provide insight into the usability of these implementations. Among other shortcomings, these studies found discoverability issues with existing privacy controls. One potential means of making privacy choice mechanisms more visible to consumers is through the use of icons. The third study described in this thesis explains the design and evaluation of new icons and accompanying text descriptions to effectively communicate the presence of privacy choices. In addition to discoverability issues, privacy choice mechanisms may not always align well with user needs. The fourth study in this thesis explored this aspect of usability, and evaluated whether existing controls related to targeted advertising on a social networking platform actually address user goals related to their advertising experience on the platform.

My prior work, as well as previous studies from the literature, emphasize the importance of usability testing with regards to interfaces through which privacy choice mechanisms are provided. Despite increased regulatory requirements and consumer pressure for privacy choice mechanisms, there is little direction for design and privacy practitioners on how to systematically evaluate such interfaces. To address this need, I developed comprehensive guidance for conducting such evaluations that pertain to different aspects of usability, such as user awareness and comprehension of privacy choice interfaces. This guidance provides an overview of HCI research methods, as well as example heuristics, prompts, and metrics, for measuring specific usability problems in privacy choice interfaces. To demonstrate the application of this guidance, the final study described in this thesis evaluated the impact of different design aspects of cookie consent notices, providing actionable recommendations that would improve the usability of these interfaces.

293 pages

Thesis Committee:
Lorris Faith Cranor (Chair)
Alessandro Acquisti
Norman Sadeh
Rebecca Balabako (Google)

James D. Herbsleb, Director, Institute for Software Research
Martial Hebert, Dean, School of Computer Science


Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu