A scale selection principle for estimating image deformations

A basic functionality of a vision system concerns the ability to compute de formation fields between different images of the same physical structure. T his article advocates the need for incorporating an explicit mechanism for automatic scale selection in this context, in algorithms for computing desc riptors such as optic flow and for performing stereo matching. A basic reas on why such a mechanism is essential is the fact that in a coarse-to-fine p ropagation of disparity or flow information, it is not necessarily the case that the most accurate estimates are obtained at the finest scales. The ex istence of interfering structures at fine scales may make it impossible to accurately match the image data at fine scales. A systematic methodology for approaching this problem is proposed, by estim ating the uncertainty in the computed flow estimate at each scale, and then selecting deformation estimates from the scales that minimize a (suitably normalized) measure of uncertainty over scales. A specific implementation o f this idea is presented for a region based differential flow estimation sc heme, which besides a hierarchical and iterative coarse-to-fine computation of flow updates, involves explicit use of confidence values for how field averaging. It is shown that the integrated scale selection and flow estimation algorit hm has the qualitative properties of leading to the selection of coarser sc ales for larger size image structures and increasing noise level, whereas i t leads to the selection of finer scales in the neighbourhood of flow field discontinuities. The latter property may serve as an indicator when detect ing flow field discontinuities and occlusions. (C) 1998 Elsevier Science B. V. All rights reserved.