Three-dimensional reconstruction of points and lines with unknown correspondence across images

Three-dimensional reconstruction from a set of images is an important and d ifficult problem in computer vision. In this paper, we address the problem of determining image feature correspondences while simultaneously reconstru cting the corresponding 3D features, given the camera poses of disparate mo nocular views. First, two new affinity measures are presented that capture the degree to which candidate features from different images consistently r epresent the projection of the same 3D point or 3D line. An affinity measur e for point features in two different views is defined with respect to thei r distance from a hypothetical projected 3D pseudo-intersection point. Simi larly, an affinity measure for 2D image line segments across three views is defined with respect to a 3D pseudo-intersection line. These affinity meas ures provide a foundation for determining unknown correspondences using wei ghted bipartite graphs representing candidate point and line matches across different images. As a result of this graph representation, a standard gra ph-theoretic algorithm can provide an optimal, simultaneous matching and tr iangulation of points across two views, and lines across three views. Exper imental results on synthetic and real data demonstrate the effectiveness of the approach.