SPEECHREADING SUPPLEMENTED BY SINGLE-CHANNEL AND MULTICHANNEL TACTILEDISPLAYS OF VOICE FUNDAMENTAL-FREQUENCY

The benefits of two tactile codes of voice fundamental frequency (F(o) ) were evaluated as supplements to the speechreading of sentences in t wo short-term training studies, each using 12 adults with normal heari ng. In Experiment 1, a multichannel spatiotemporal display of F(o), kn own as Portapitch, was used to stimulate the index finger. In an attem pt to improve on past performance with this display, the coding scheme was modified to better cover the F(o) range of the talker in the trai ning materials. For Experiment 2, to engage kinesthetic/proprioceptive pathways, a novel single-channel display was built, in which F(o) was coded as the vertical displacement of a small finger-rest. Input to b oth displays consisted of synthesized replicas of the F(o) contours of the sentences, prepared and perfected off-line. Training with the two tactile F(o) displays included auditory presentation of the synthesiz ed F(o) contours in conjunction with the tactile patterns on alternate trials. Speechreading enhancement by the two tactile F(o) displays wa s compared to the enhancement provided when auditory F(o) information was available in conjunction with the tactile patterns, by auditory pr esentation of a sinusoidal indication of the presence or absence of vo icing, and by a single-channel tactile display of the speech waveform presented to the index finger. Despite the modified coding strategy, t he multichannel Portapitch provided a mean tactile speechreading enhan cement of 7 percentage points, which was no greater than that found in previous studies. The novel positional F(o) display provided only a 4 percentage point enhancement. Neither F(o) display was better than th e simple single-channel tactile transform of the full speech waveform, which gave a 7 percentage point enhancement effect. Auditory speechre ading enhancement effects were 17 percentage points with the voicing i ndicator and approximately 35 percentage points when the auditory F(o) contour was provided in conjunction with the tactile displays. The fi ndings are consistent with the hypothesis that subjects were not takin g full advantage of the F(o) variation information available in the ou tputs of the two experimental tactile displays.