COMPARING RELIABILITY OF PERCEPTUAL RATINGS OF ROUGHNESS AND ACOUSTICMEASURES OF JITTER

Acoustic analysis is often favored over perceptual evaluation of voice because it is considered objective, and thus reliable. However, recen t studies suggest this traditional bias is unwarranted. This study exa mined the relative reliability of human listeners and automatic system s for measuring perturbation in the evaluation of pathologic voices. T en experienced listeners rated the roughness of 50 voice samples (rang ing from normal to severely disordered) on a 75 mm visual analog scale . Rating reliability within and across listeners was compared to the r eliability of jitter measures produced by several voice analysis syste ms (CSpeech, SoundScope, CSL, and an interactive hand-marking system). Results showed that overall listeners agreed as well or better than ' 'objective'' algorithms. Further, listeners disagreed in predictable w ays, whereas automatic algorithms differed in seemingly random fashion s. Finally, listener reliability increased with severity of pathology; objective methods quickly broke down as severity increased. These fin dings suggest that listeners and analysis packages differ greatly in t heir measurement characteristics. Acoustic measures may have advantage s over perceptual measures for discriminating among essentially normal voices; however, reliability is not a good reason for preferring acou stic measures of perturbation to perceptual measures.