Natural communication with information systems

Pervasive networking and sophisticated computing open opportunities for col laborative information processing independent of time and space. In this in stance the information system becomes on enhancer of human intellect, as we ll as a mediator for communication among participants. The human user favor s the sensory dimensions of sight, sound, and touch as primary channels of communication. Machines that can accommodate these modes promise flexibilit ies and functionalities that transcend the traditional mouse and keyboard. This paper describes research to establish human-computer interfaces that c apture attributes of natural face-to-face communication. An experimental mu ltimodal system is developed to study several aspects of natural style huma n-computer communication. While as yet primitive, the technologies of image and gaze processing hands-free conversation, and force feedback tactile tr ansduction are combined and used simultaneously for manipulating objects in a shared workspace. Software agent fuse the sensory, signals to estimate a nd interpret user intent. Current areas of experimental application include disaster relief/crisis management. telemedicine/rehabilitation, and mobile office/wearable computers.