No learning happens until students make the choice to engage, regardless of how well-designed and personalized a lesson. While advanced algorithms have been developed to personalize and accelerate learning, similar highly quality models and algorithms aren't available for intelligent support of student motivation. The greatest challenge lies in the lack of high-quality measurement models to support the administration of motivational interventions. Existing models focus on the observable engagement behaviors of students. These measures are prone to noise from non-learner-specific influences as opposed to reflecting the underlying motivational drivers of engagement. This complicates the task of leveraging these analytics for assessing individual student motivational needs to support greater engagement.

In this dissertation, I address this gap through the development of a model to measure student diligence, their capacity to self-regulate and engage with learning activities. I leverage prior research in psychology and psychometrics to identify behavioral metric candidates. Through secondary analysis of a year-long longitudinal dataset of log data from students learning with intelligent tutoring systems, I evaluate the viability of these behavioral measures to estimate student diligence. I further develop these measures by leveraging theory to account for some cognitive, temporal, and social confounding factors. My analysis indicates that these behavioral measures, while better indicators of diligence, are still prone to other sources of noise that make the measures unreliable.

To address the unique challenges of measurement with observational data, I explore the viability of diversifying the model inputs by leveraging multiple operationalizations of diligence for estimation. I demonstrate that multioperational models possess more desirable psychometric properties than any individual measure. Furthermore, I developed the Learner Engagement Simulator (LEnS) to generate data that reflects the challenges of estimating motivational constructs due to unobserved influences from the social and environmental context. My analysis of the simulated data reinforces the findings with real student data that multiple operationalizations of diligence increases the estimator accuracy.

103 pages

Thesis Committee:
Ken Koedinger (Chair)
John Stamper
Geoff Kaufman
Artur Dubrawski
Sidney D'Mello (University of Colorado, Boulder)

Jodi Forlizzi, Head, Human-Computer Interaction Institute
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu