ON MEASURING PERFORMANCE IN CATEGORY JUDGMENT STUDIES OF NONVERBAL BEHAVIOR

Attention is drawn to three interrelated types of error that are commi tted with high frequencies in the description and analysis of studies- of nonverbal behavior. The errors involve the calculation of inappropr iate measures of accuracy, the use in statistical analyses of inapprop riate chance levels, and misapplications of chi2 and binomial statisti cal tests. Almost all papers published between 1979 and 1991 that repo rted performance separately for different stimulus and response classe s suffer from one or more of these errors. The potential consequences of these errors are described, and a variety of proposed measures of p erformance is examined. Since all measures formerly proposed have weak nesses, a new and easily calculated measure, an unbiased hit rate (H(u )), is proposed. This measure is the joint probability that a stimulus category is correctly identified given that it is presented at all an d that a response is correctly used given that it is used at all. Two available data sets are reanalyzed using this measure, and the differe nces in the conclusions reached compared to those reached with an anal ysis of hit rates are described.