Computer-aided analysis and design for spoken dialogue systems based on quantitative simulations

Corpus-based analysis and design of spoken dialogue systems have been widel y used. However, in such approaches the dialogue performance cannot be pred icted before the system is on line, and the dialogue corpora need to be rec ollected if the system is modified or different conditions are assumed. Als o, the effects of different factors, from the system's dialogue strategies, speech recognition and understanding conditions and accuracy, to the user' s response pattern, etc., on the dialogue system performance cannot be quan titatively identified and analyzed, because they cannot be precisely contro lled in different corpora. In this paper, a complete development of computer-aided analysis and design approaches for spoken dialogue systems based on quantitative simulations i s presented. With this approach the various performance metrics of a dialog ue system can be flexibly defined and numerically evaluated, such that the behavior and performance of the dialogue system can be well predicted and e fficiently analyzed before the implementation of the real spoken dialogue s ystem is completed. How the different dialogue performance measures vary wi th respect to each of the many very complicated factors, regardless of whet her it is caused by an individual component, by the overall system design, or by users' response patterns, can be separately identified, because all s uch factors can be precisely controlled in the simulation. Several analysis examples are presented to show how the approach can be used, including sel ection and tuning of speech understanding front end, system strategy design considering query factors and confirmation factors, and objective estimate s of user's degree of satisfaction. This approach is therefore very useful for the analysis and design of spoken dialogue systems, although the online test, corpus-based analysis and user survey can always follow after the sy stem is online.