LANGUAGE IN MULTIMEDIA - INTERACTION OF SPOKEN AND WRITTEN INFORMATION

Recent findings on the brain's anatomy suggest that there are modality -specific resources for the processing of language. This article devel ops a theoretical model on the behavioral consequences of modality-spe cific resources for spoken and written information. The model predicts antagonistic effects of reduced interference versus reduced integrati on of verbal information presented across the two modalities, as compa red to within a single modality. Three experiments compare unimodal to multimedia messages containing either a mix of modalities or informat ion presented bimodally. The results provide qualified support far the model in the realm of item-specific Versus relational memory and elab oration. Compared to unimodal presentations, alternating the presentat ion modality decreases relational processing but can increase item-spe cific processing. Furthermore, a simultaneous bimodal presentation for mat increases recognition memory but does not appear to affect relatio nal processing or item-specific elaboration.