A STEREO MATCHING ALGORITHM WITH AN ADAPTIVE WINDOW - THEORY AND EXPERIMENT

A central problem in stereo matching by computing correlation or sum o f squared differences (SSD) lies in selecting an appropriate window si ze. The window size must be large enough to include enough intensity v ariation for reliable matching, but small enough to avoid the effects of projective distortion. If the window is too small and does not cove r enough intensity variation, it gives a poor disparity estimate, beca use the signal (intensity variation) to noise ratio is low. If, on the other hand, the window is too large and covers a region in which the depth of scene points (i.e., disparity) varies, then the position of m aximum correlation or minimum SSD may not represent correct matching d ue to different projective distortions in the left and right images. F or this reason, a window size must be selected adaptively depending on local variations of intensity and disparity. We present a method to s elect an appropriate window by evaluating the local variation of the i ntensity and the disparity. We employ a statistical model of the dispa rity distribution within the window. This modeling enables us to asses s how disparity variation, as well as intensity variation, within a wi ndow affects the uncertainty of disparity estimate at the center point of the window. As a result, we can devise a method which searches for a window that produces the estimate of disparity with the least uncer tainty for each pixel of an image: the method controls not only the si ze but also the shape (rectangle) of the window. We have embedded this adaptive-window method in an iterative stereo matching algorithm: sta rting with an initial estimate of the disparity map, the algorithm ite ratively updates the disparity estimate for each point by choosing the size and shape of a window till it converges. The stereo matching alg orithm has been tested on both synthetic and real images, and the qual ity of the disparity maps obtained demonstrates the effectiveness of t he adaptive window method.