Auditory assessment of synthesized speech in application scenarios: Two case studies

For assessing the synthesized speech output component in a complex applicat ion system, application-oriented evaluation methods and methodologies are n eeded which are not supplied by standardized lest batteries so far. Many st andardized tests analyze synthetic speech mainly with regard to its form (s urface structure), and only to a less degree with regard to the meaning tha t is assigned to it (deep structure). In turn, in order to obtain a valid a ssessment focus for an application system, the functional aspect of speech (which depends on its deep structure) has to be taken into account. In the paper two case studies are presented which focus on the acceptability of th e synthesis component and its constituent dimensions in different applicati on scenarios. In the first one synthetic speech in a car navigation and tra ffic information system is assessed. The second study relates to synthetic speech in a dialogue system. The assessment is limited to laboratory experi ments and avoids costly field tests. It turns out that different dimensions contribute to a variable degree to overall acceptability, differently depe nding on the application scenario. Application-oriented testing is thus req uired to identify the application-specific dimensions. It is discussed whic h characteristics of the application have to be modeled in the assessment, and examples are given for both applications. (C) 2001 Elsevier Science B.V . All rights reserved.