A SPEECH DIALOGUE SYSTEM WITH MULTIMODAL INTERFACE FOR TELEPHONE DIRECTORY ASSISTANCE

This paper describes a multimodal dialogue system employing speech inp ut. This system uses three input methods (through a speech recognizer, a mouse, and a keyboard) and two output methods (through a display an d using sound). For the speech recognizer, an algorithm is employed fo r large-vocabulary speaker-independent continuous speech recognition b ased on the HMM-LR technique. This system is implemented for telephone directory assistance to evaluate the speech recognition algorithm and to investigate the variations in speech structure that users utter to computers. Speech input is used in a multimodal environment. The coll ecting of dialogue data between computers and users is also carried ou t. Twenty telephone-number retrieval tasks are used to evaluate this s ystem. In the experiments, all the users are equally trained in using the dialogue system with an interactive guidance system implemented on a workstation. Simplified city maps that indicate subscriber names an d addresses are used to reduce the implicit restrictions imposed by wr itten sentences, thus allowing each user to develop his own forms of e xpression. The task completion rate is 99.0% and approximately 75% of the users say that they prefer this system to using a telephone book. Moreover, there is a significant decrease in nonkeyword usage, i.e., t he usage of words other than names and addresses, for users who receiv e more utterance practice.