IMAGE-ANALYSIS FOR 3D MODELING, RENDERING, AND VIRTUAL VIEW GENERATION

The generation of virtual environments for telepresence systems, inter active viewing, and immersion in remote 3D-scenarios is an expanding a nd very promising research field. The challenge of this technology is to design systems capable of synthesizing views from any desired persp ective using a set of real-scene perspectives. In such a system, two m ain parts can be distinguished: multiview image-analysis and viewpoint synthesis. Among other methods, multiview image-analysis is comprised of methods for the solution of three commonly encountered difficult t asks in computer vision: estimation of pixel correspondence, object se gmentation, and camera calibration. Moreover, the results of multiview image-analysis are the required foundations for image synthesis. In t his paper, we present a robust framework for stereo-image analysis, 3D modeling, and viewpoint synthesis. The image analysis part comprises disparity estimation by an improved block-matching algorithm and depth -based object segmentation using morphology. The basic idea of the sys tem is to solve each of these tasks in direct interaction with the oth ers. The results obtained in the partial solution of the correspondenc e problem are used to substantially simplify the segmentation task and vice versa. In the image-synthesis part, two tasks are considered: di rect generation of arbitrary intermediate views, and extraction of the 3D structure of the scene using disparity results and camera paramete rs. The system has been successfully demonstrated in the context of vi rtual-view generation and object manipulation through 3D modeling. (C) 1998 Academic Press.