Computing optical flow with physical models of brightness variation

Although most optical flow techniques presume brightness constancy, it is w ell-known that this constraint is often violated, producing poor estimates of image motion. This paper describes a generalized formulation of optical flow estimation based on models of brightness variations that are caused by time-dependent physical processes. These include changing surface orientat ion with respect to a directional illuminant, motion of the illuminant, and physical models of heat transport in infrared images. With these models, w e simultaneously estimate the 2D image motion and the relevant physical par ameters of the brightness change model. The estimation problem is formulate d using total least squares (TLS), with confidence bounds on the parameters . Experiments in four domains, with both synthetic and natural inputs, show how this formulation produces superior estimates of the 2D image motion.