Multi-modal perception

To design, optimise and deliver multimedia and virtual-reality products and services it is necessary to match performance to the capabilities of users . When a multimedia system is used, the presence of audio and video stimuli introduces significant cross-modal effects (the sensory streams interact). This paper introduces a number of cross-modal interactions that are releva nt to communications systems and discusses the advanced experimental techni ques required to provide data for modelling multi-modal perception. The aim of the work is to provide a multi-modal perceptual model that can be used for performance assessment and can be incorporated into coding algorithms. The current and future applications of multimodal modelling are discussed.