3-D STRUCTURE FROM VISUAL-MOTION - MODELING, REPRESENTATION AND OBSERVABILITY

The problem of 'structure from motion' concerns the reconstruction of the three-dimensional structure of a scene from its projection onto a moving two-dimensional surface. Such a problem is solved effectively b y the human visual system, judging from the ease with which we perform delicate control tasks involving vision as a sensor such as reaching for objects in the environment or driving a car. In this paper we stud y 'structure from motion' from the point of view of dynamical systems: we first formalize the problem of 3-D structure and motion reconstruc tion as the estimation of the state of certain nonlinear dynamical mod els. Then we study the feasibility of 'structure from motion' by analy zing the observability of such models. The models that define the visu al motion estimation problem for feature points in the Euclidean 3-D s pace are not locally observable; however, the non-observable manifold can be easily isolated by imposing metric constraints on the state spa ce. One of the peculiarities of vision as a sensor is its richness, wh ich can be a disadvantage when we are interested only in few of the un known parameters. For instance, if we want to control the direction of heading of our car by measuring brightness values on our retina, we h ave to overcome the effects that the shape of the environment, its ref lectance properties, illumination and other quantities have on our mea surements. Invariance to undesired parameters can be achieved by appro priate modeling or by choice of representation of the parameter space. We propose and analyze models for 3-D structure that are independent of 3-D motion and vice versa. Estimating unknown parameters from such models amounts to the identification of nonlinear and implicit systems with parameters on differentiable manifolds. such as a sphere or the so-called essential manifold. (C) 1997 Elsevier Science Ltd.