The statistics of optical flow

When processing image sequences some representation of image motion must be derived as a first stage. The most often used representation is the optica l flow field, which is a set of velocity measurements of image patterns. It is well known that it is very difficult to estimate accurate optical flow at locations in an image which correspond to scene discontinuities. What is less well known, however, is that even at the locations corresponding to s mooth scene surfaces, the optical flow field often cannot be estimated accu rately. Noise in the data causes many optical flow estimation techniques to give bi ased flow estimates. Very often there is consistent bias: the estimate tend s to be an underestimate in length and to be in a direction closer to the m ajority of the gradients in the patch. This paper studies all three major c ategories of flow estimation methods-gradient-based, energy-based, and corr elation methods, and it analyzes different ways of compounding one-dimensio nal motion estimates (image gradients, spatiotemporal frequency triplets, l ocal correlation estimates) into two-dimensional velocity estimates, includ ing linear and nonlinear methods. Correcting for the bias would require knowledge of the noise parameters. In many situations, however, these are difficult to estimate accurately, as t hey change with the dynamic imagery in unpredictable and complex ways. Thus , the bias really is a problem inherent to optical flow estimation. We argu e that the bias is also integral to the human visual system. It is the caus e of the illusory perception of motion in the Ouchi pattern and also explai ns various psychophysical studies of the perception of moving plaids. (C) 2 001 Academic Press.