Digital media creators, including video editors, live streamers, and game developers, rely on diverse creativity tools such as game engines, video editing software, and live streaming platforms to produce their work. These ecosystems are powerful, however they often impose constraints on their users, leading to rigid interfaces that fail to adapt to how creators want to work. Existing tools can limit a creator's ability to realize ambitious ideas, offer inadequate mechanisms for communication between creators and their audiences, and provide interfaces that do not align with how creators naturally express their preferences. The first half of this thesis explores these misalignments through the development of data-driven tools, resulting in three systems that expand the design possibilities in mobile augmented reality, create novel mechanisms for streamer–audience engagement on live streaming platforms, and support video creators in expressing musical preferences through a vibe-based creative assistant. Taken together, my work on these projects revealed that the content creation process produces a wealth of valuable data, yet this data is often leveraged ad hoc to power isolated features, as opposed to being treated as a foundational resource for designing creator-aligned systems.

Building upon this realization, I argue that the future of creativity tools relies on the deliberate use of creative activity traces as a primary design material. I define creative activity traces as the process-related data generated as creators work, which accumulates into an evolving record of the decisions, explorations, and refinements that unfold over time. This includes things such as preferred color palettes , abandoned sketches, or successive prototype iterations. These traces are often multi-modal, distributed across tools, and deeply personalized. To understand how the HCI community currently works with traces, I conducted a systematic review of 133 creativity tools and developed a preliminary design framework for trace awareness. This review revealed that modern creativity tools either treat activity traces as atemporal raw material, or utilize them only as needed as input to a specific feature, thereby limiting both the temporal scope and design opportunities possible with traces. Capturing traces, in other words, does not automatically mean a system makes use of them as traces, since a creator's process becomes visible as data accumulates. Rather than using traces as isolated inputs or for post-hoc analysis, my thesis calls for traces to be treated as a design material that systems are built around. This approach in turn leads to adaptive systems that stay aligned with how creators work and how their processes unfold.

The final phase of this research builds upon my theory of creative activity traces and trace-aware tools through the study of video content creators' practices for creating, maintaining, and evolving a recognizable brand across video projects. Through a trace mapping study with seven video creators, I cataloged the trace data their practice accumulates over time, the behaviors associated with their accumulated traces across projects, and the frictions they face as a result of their cross-project practices. Working with traces as a design material showed that trace data is not uniform, but comes in differing grains that afford and resist different kinds of treatment. As trace data accumulates across projects, it also forms a texture: a whole with a shape of its own, whose individual pieces can be difficult to disentangle.

This thesis contributes to HCI and creativity tool research by framing creative practice as an inherently data-rich process that requires more intentional engagement from the HCI research community. I formalize the definitions of creative activity traces and trace-aware tools, and I offer a design framework for trace awareness. I show what it means to treat traces as a critical building block for designing creativity tools rather than as a byproduct of creative work. Taken together, these contributions set forth a trace-aware agenda for the next generation of creativity tools, one that keeps creators and their evolving practices in view as digital creative production grows more automated and uncertain.

222 pages

Thesis Committee:
Jessica Hammer (Co-chair)
Erik Harpstead (Co-chair)
Nikolas Martelaroi
Bryan Wang (Adobe)
Max Kreminski (Cornell University)

Brad A. Myers, Head, Human-Computer Interaction Institute
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu