Euclidean reconstruction and reprojection up to subgroups

The necessary and sufficient conditions for being able to estimate scene st ructure, motion and camera calibration from a sequence of images are very r arely satisfied in practice. What exactly can be estimated in sequences of practical importance, when such conditions are not satisfied ? In this pape r we give a complete answer to this question. For every camera motion that fails to meet the conditions, we give explicit formulas for the ambiguities in the reconstructed scene, motion and calibration. Such a characterizatio n is crucial both for designing robust estimation algorithms (that do not t ry to recover parameters that cannot be recovered), and for generating nove l views of the scene by controlling the vantage point. To this end, we char acterize explicitly all the vantage points that give rise to a valid Euclid ean reprojection regardless of the ambiguity in the reconstruction. We also characterize vantage points that generate views that are altogether invari ant to the ambiguity. All the results are presented using simple notation t hat involves no tensors nor complex projective geometry, and should be acce ssible with basic background in linear algebra.