This thesis examines how profile signals–such as subscriber counts, feedback ratings, and security verification badges–can be manipulated, misinterpreted, or rendered ineffective across online platforms, ultimately undermining platform integrity and user safety.

First, we show how misleading profile signals can increase user exposure to harm. Analyzing YouTube accounts sold on an online marketplace, we find that many are repurposed: channels that built reputations under one identity are sold, rebranded, and used to disseminate content that violates community guidelines, including scams and misinformation. Despite these changes, they retain engagement metrics and visibility, misleading users and amplifying harmful content.

Second, we test which profile signals predict sales cessation and market exit—both indicators of vendor quality—on darkweb marketplaces. We first validate a widely used revenue estimation method using ground truth data from a law enforcement operation. We then apply this method to model vendor quality across eight marketplaces over a decade. While reputation ratings are assumed to be strong indicators of quality, they are outperformed by models that incorporate a broader set of profile signals, highlighting an opportunity to improve vendor assessment.

Third, we extend this modeling approach to two cryptocurrency peer-to-peer marketplaces to predict account suspension due to fraud or abuse. Using longitudinal data, we train models and conduct a prospective cohort study comparing three groups: the lowest-rated accounts, a random baseline, and accounts flagged by our model. We find that security signals promoted by the platform, such as user ratings and verification badges, are poor predictors of suspension. Instead, less visible profile signals, like trading volume and partner diversity, are significantly more predictive. This finding underscores the need to audit user interfaces, as the most prominent signals fail to reflect actual risk.

Collectively, these studies show that profile signals can both obscure and reveal user risk. This thesis offers recommendations for platform designers and operators, emphasizing the need for empirical evaluation and redesign of profile signals and the systems that rely on them. By continuously auditing these signals and adapting user interfaces, platforms can more effectively surface trustworthy users and mitigate abuse, leading to safer and more resilient online ecosystems.

206 pages

Thesis Committee:
Nicolas Christin(Chair)
Bogdan Vasilescu
Sauvik Das
Rolf van Wegberg (TU Delft)
Stefan Savage (University of California, San Diego)

Nicolas Christin, Head, Software and Societal Systems Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu