AN INFORMATION-THEORY OF VISUAL COMMUNICATION

The fundamental problem of visual communication is that of producing t he best possible picture at the lowest data rate. We address this prob lem by extending information theory to the assessment of the visual co mmunication channel as a whole, from image gathering to display. The e xtension unites two disciplines, the electrooptical design of image ga thering and display devices and the digital processing for image codin g and restoration. The mathematical development leads to several intui tively attractive figures of merit for assessing the visual communicat ion channel as a function of the critical limiting factors that constr ain its performance. Multiresolution decomposition is included in the mathematical development to optimally combine the economical encoding of the transmitted signal with image gathering and restoration. Quanti tative and qualitative assessments demonstrate that a visual communica tion channel ordinarily can be expected to produce the best possible p icture at the lowest data rate only if the image-gathering device prod uces the maximum-realizable information rate and the image-restoration algorithm properly accounts for the critical limiting factors that co nstrain the visual communication. These assessments encompass (a) the electro-optical design of the image-gathering device in terms of the t rade-off between blurring and aliasing in the presence of photodetecto r and quantization noises, (b) the compression of data transmission by redundancy reduction, (c) the robustness of the image restoration to uncertainties in the statistical properties of the captured radiance f ield, and (d) the enhancement of particular features or, more generall y, of the visual quality of the observed image. The 'best visual quali ty' in this context normally implies a compromise among maximum-realiz able fidelity, sharpness, and clarity which depends on the characteris tics of the scene and the purpose of the visual communication (e.g. di agnosis versus entertainment).