A key challenge of the learning sciences is moving research results into practice. Educators on the front lines perceive little value in the outputs of education research and demand more "usable knowledge". This work explores the potential instead of usable artifacts to translate knowledge into practice. One contribution is the integration of market-oriented opportunity finding to design these translational systems and tools such that they can be adopted easily into practice. A second contribution is the conception of these systems as research probes that both operationalize theoretical principles and provide instrumentation to build knowledge of applying theory in practice.

College student study practices are the domain chosen for the development of and reflection upon these methods. Iterative ethnographic field work identified two systems that would be likely to advance both learning in practice and knowledge for applying the employed theories in general. Nudge was designed to improves students' study time management by regularly emailing students with explicit recommended study activities. It reconceptualizes the syllabus into an interactive guide that fits into modern students' attention streams. Examplify was designed to improve how students learn from worked example problems by modularizing them into steps and scaffolding their metacognitive behaviors though problem-solving and self-explanation prompts. It combines these techniques in a way that is exceedingly easy to author, using extant answer keys and students' self-evaluations for correctness.

Nudge and Examplify were evaluated experimentally over a full semester of a lecture-based introductory chemistry course. Nudge messages were found to increase students' sense of achievement and to interact with students' existing time management skills to improve exam grades for poorer students. Among students who could choose whether to receive them, 80% continued to. Students with access to Examplify had higher exam scores (d=0.26), especially on delayed measures of learning (d=0.40). A key design decision in Examplify was not clearly resolvable by existing theory and so was tested experimentally by comparing two variants, one without prompts to solve the steps. The variant without problem-solving was less effective (d=0.77) and was little used, while usage rates of the variant with problem solving increased over time.

These results support the use of the design methods employed and provide specific empirical recommendations for future designs of these and similar systems for implementing theory in practice.

154 pages

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu