COMPARISON OF EVALUATION METHODS USING STRUCTURED USABILITY PROBLEM REPORTS

Recent HCI research has produced analytic evaluation techniques which claim to predict potential usability problems for an interactive syste m. Validation of these methods has involved matching predicted problem s against usability problems found during empirical user testing. This paper shows that the matching of predicted and actual problems requir es careful attention, and that current approaches lack rigour or gener ality. Requirements for more rigorous and general matching procedures are presented. A solution to one key requirement is presented: a new r eport structure for usability problems. It is designed to improve the quality of matches made between usability problems found during empiri cal user testing and problems predicted by analytic methods. The use o f this report format is placed within its design research context, an ongoing project on domain-specific methods for software visualizations .