The problem of degeneracy in structure and motion recovery from uncalibrated image sequences

The aim of this work is the recovery of 3D structure and camera projection matrices for each frame of an uncalibrated image sequence. In order to achi eve this, correspondences are required throughout the sequence. A significa nt and successful mechanism for automatically establishing these correspond ences is by the use of geometric constraints arising from scene rigidity. H owever, problems arise with such geometry guided matching if general viewpo int and general structure are assumed whilst frames in the sequence and/or scene structure do not conform to these assumptions. Such cases are termed degenerate. In this paper we describe two important cases of degeneracy and their effec ts on geometry guided matching. The cases are a motion degeneracy where the camera does not translate between frames, and a structure degeneracy where the viewed scene structure is planar. The effects include the loss of corr espondences due to under or over fitting of geometric models estimated from image data, leading to the failure of the tracking method. These degenerac ies are not a theoretical curiosity, but commonly occur in real sequences w here models are statistically estimated from image points with measurement error. We investigate two strategies for tackling such degeneracies: the first use s a statistical model selection test to identify when degeneracies occur: t he second uses multiple motion models to overcome the degeneracies. The str ategies are evaluated on real sequences varying in motion, scene type, and length from 13 to 120 frames.