Systems and experiment paper: Construction of panoramic image mosaics withglobal and local alignment

This paper presents a complete system for constructing panoramic image mosa ics from sequences of images. Our mosaic representation associates a transf ormation matrix with each input image, rather than explicitly projecting al l of the images onto a common surface (e.g., a cylinder). In particular, to construct a full view panorama, we introduce a rotational mosaic represent ation that associates a rotation matrix (and optionally a focal length) wit h each input image. A patch-based alignment algorithm is developed to quick ly align two images given motion models. Techniques for estimating and refi ning camera focal lengths are also presented. In order to reduce accumulated registration errors, we apply global alignme nt (block adjustment) to the whole sequence of images, which results in an optimally registered image mosaic. To compensate for small amounts of motio n parallax introduced by translations of the camera and other unmodeled dis tortions, we use a local alignment (deghosting) technique which warps each image based on the results of pairwise local image registrations. By combin ing both global and local alignment, we significantly improve the quality o f our image mosaics, thereby enabling the creation of full view panoramic m osaics with hand-held cameras. We also present an inverse texture mapping algorithm for efficiently extrac ting environment maps from our panoramic image mosaics. By mapping the mosa ic onto an arbitrary texture-mapped polyhedron surrounding the origin, we c an explore the virtual environment using standard 3D graphics viewers and h ardware without requiring special-purpose players.