Image-based rendering using parameterized image varieties

This paper addresses the problem of characterizing the set of all images of a rigid set of m points and n lines observed by a weak perspective or para perspective camera. By taking explicitly into account the Euclidean constra ints associated with calibrated cameras, we show that the corresponding ima ge space can be represented by a six-dimensional variety embedded in R2(m+n ) and parameterized by the image positions of three reference points. The c oefficients defining this parameterized image variety (or PIV for short) ca n be estimated from a sample of images of a scene via linear and non-linear least squares. The PIV provides an integrated framework for using both poi nt and line features to synthesize new images from a set of pre-recorded pi ctures (image-based rendering). The proposed technique does not perform any explicit three-dimensional scene reconstruction but it supports hidden-sur face elimination, texture mapping and interactive image synthesis at frame rate on ordinary PCs. It has been implemented and extensively tested on rea l data sets.