This dissertation provides initial empirical evidence for Expectancy Value Theory for Help Sources and generates design recommendations for online courses based on the newfound understanding between theory and student behavior. My high-level research goals are pursued in the context of help seeking in the presence of reputation systems in MOOC discussion forums. Educational technology can be intentionally designed and introduced in such a way as to maintain the benefits of existing technology while reducing negative impact on learning-relevant behaviors. I do this through the lens of student expectancy and values for the help source, and costs of pursuing that help.

Within this thesis I present three online survey experiments, one is intended to provide empirical evidence for the connection between Expectancy Value Theory for Help Sources and student help seeking outcomes. The remaining two survey experiments are designed to further investigate the results of a system for help exchange through the lens of Expectancy Value Theory for Help Sources. The first survey supports the existence of beliefs for help sources, although careful design of value manipulations is necessary to isolate value beliefs from expectancy beliefs for the help source.

In a field experiment investigating the design of a help exchange system, I explore the connection between common reputation system features and Expectancy Value Theory for Help Sources. This provides support for the theory outside of a controlled laboratory setting. This Quick Helper MOOC Experiment and the supporting Quick Helper Theory Survey Experiment show that voting within a reputation system context decreases the number of peers invited to be helpers possibly through an increase in evaluation anxiety. Help giver badges ca reduce this evaluation anxiety and mitigate the negative impact of voting.

I performed a final field experiment in a small private online course to examine these issues in a more naturalistic setting outside of the Quick Helper help exchange system. I explored learning expectancy-emphasizing email prompts and voting in the course discussion forum, and how these manipulations impacted larger, more nuanced dependent variables such as help seeking and learning. Results from this experiment are not as strong as the more tightly controlled survey experiments and Quick Helper MOOC field experiment, but we still see support in the general direction of our original hypotheses.

From these experiments I generate a series of design recommendations for instructors of online courses implementing discussion forums: (1) reputation systems can have a positive effect on student engagement in discussion forums, but there may be a negative effect on help seeking and other vulnerable learning-relevant behaviors, (2) The negative impact of evaluation anxiety from voting can be mitigated through the use of either help giver badges or using only upvoting instead of up/downvoting which may reduce evaluation anxiety, and (4) Email prompts with dilute implementation have questionable impact on student contributions in discussion forums.

122 pages

Thesis Committee:
Carolyn Penstein Rosé (Chair)
Robert Kraut
Vincent Aleven
Marsha Lovett (Psychology/CMU)
Stuart Karavenick (University of Michigan)

Anind K. Dey, Head, Human-Computer Interaction Institute
Andrew W. Moore, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu