@device(postscript)
@libraryfile(Mathematics10)
@libraryfile(Accents)
@style(fontfamily=timesroman,fontscale=11)
@pagefooting(immediate, left "@c<Technical Report 1996>",
center "@c<Computer Science>",
right "@c<Carnegie Mellon>")
@heading(Tracking the Effectiveness of Usability Evaluation Methods)
@heading(CMU-CS-96-160)
@center(@b(Bonnie E. John, Steven J. Marks@foot<Standard Microsystems Corporation, Irvine, California 92718>))
@center(August 1996@foot<Also appears as Human-Computer Interaction Institute Technical report CMU-HCII-96-102.>)
@center(FTP: CMU-CS-96-160.ps, CMU-CS-96-160fig.ps)
@blankspace(1)
@begin(text)

We present a case study that tracks usability problems predicted with six
usability evaluation methods (Claims Analysis, Cognitive Walkthrough, GOMS,
Heuristic Evaluation, User Action Notation, and simply reading the
specification) through a development process.  We assess the methods'
predictive power by comparing the predictions to the results of user tests.
We assess the methods' persuasive power by seeing how many problems led to
changes in the implemented code.  We assess design-change effectiveness by
user testing the resulting new versions of the system.  We conclude that
predictive methods are not as effective as the HCI field would like and
discuss directions for future research.

@blankspace(2line)
@begin(transparent,size=10)
@b(Keywords:@ )@c<Usability evaluation methods, claims analysis, cognitive
walkthrough, GOMS, heuristic evaluation, user action notation, usability
engineering>
@end(transparent)
@blankspace(1line)
@end(text)
@flushright(@b[(24 pages)])