Extracting usability information from user interface events

Modern window-based user interface systems generate user interface events a s natural products of their normal operation. Because such events can be au tomatically captured and because they indicate user behavior with respect t o an application's user interface, they have long been regarded as a potent ially fruitful source of information regarding application usage and usabil ity. However, because user interface events are typically voluminos and ric h in detail, automated support is generally required to extract information at a level of abstraction that is useful to investigators interested in an alyzing application usage or evaluating usability. This survey examines computer-aided techniques used by HCI practitioners an d researchers to extract usability-related information from user interface events. A framework is presented to help HCI practitioners and researchers categorize and compare the approaches that have been, or might fruitfully b e, applied to this problem. Because many of the techniques in the research literature have not been evaluated in practice, this survey provides a conc eptual evaluation to help identify some of the relative merits and drawback s of the various classes of approaches. Ideas for future research in this a rea are also presented. This survey addresses the following questions: How might user interface eve nts be used in evaluating usability? How are user interface events related to other forms of usability data? What are the key challenges faced by inve stigators wishing to exploit this data? What approaches have been brought t o bear on this problem and how do they compare to one another? What are som e of the important open research questions in this area?