Minimal projective reconstruction including missing data

The minimal data necessary for projective reconstruction from image points is well-known when each object point is visible in all images. In this pape r, we formulate and propose solutions to a new family of reconstruction pro blems for multiple images from minimal data, where there are missing points in some of the images. The ability to handle the minimal cases with missin g data is of great theoretical and practical importance. It is unavoidable to use them to bootstrap robust estimation such as RANSAC and LMS algorithm s and optimal estimation such as bundle adjustment. First, we develop a fra mework to parameterize the multiple view geometry needed to handle the miss ing data cases. Then, we present a solution to the minimal case of eight po ints in three images, where one different point is missing in each of the t hree images. We prove that there are, in general, as many as 11 solutions f or this minimal case. Furthermore, all minimal cases with missing data for three and four images are catalogued. Finally, we demonstrate the method on both simulated and real images and show that the algorithms presented in t his paper can be used for practical problems.