ACCESS TO MATHEMATICS FOR VISUALLY DISABLED STUDENTS THROUGH MULTIMODAL INTERACTION

Mathematics relies on visual forms of communication and is thus largel y inaccessible to people who cannot communicate in this manner because of visual disabilities. This article outlines the Mathtalk project, w hich addressed this problem by using computers to produce multimodal r enderings of mathematical information. This example is unusual in that it is essential to use multiple modalities because of the nature and the difficulty of the application. In addition, the emphasis is on non visual (and hence novel) modalities. Crucial to designing a usable aud itory interface to algebra notation is an understanding of the differe nces between visual and listening reading, particularly those aspects that make the former active and the latter passive. A discussion of th ese differences yields the twin themes of compensation for lack of ext ernal memory and provision of control over information flow. These the mes were addressed by: the introduction of prosody to convey algebraic structure in synthetically spoken expressions; the provision of struc ture-based browsing functions; and the use of a prosody-based musical glance based on algebra earcons.