We present a case study that tracks usability problems predicted with
six usability evaluation methods (claims analysis, cognitive walkthrou
gh, GOMS, heuristic evaluation, user action notation, and simply readi
ng the specification) through a development process. We assess the met
hod's predictive power by comparing the predictions to the results of
user tests. We assess the method's persuasive power by seeing how many
problems led to changes in the implemented code. We assess design-cha
nge effectiveness by user testing the resulting new versions of the sy
stem. We conclude that predictive methods are not as effective as the
HCI field would like and discuss directions for future research.