Using photo-consistency to register 2D optical images of the human face toa 3D surface model

In this paper, we propose a novel method to register two or more optical im ages to a 3D surface model. The potential applications of such a registrati on method could be in medicine; for example, in image guided interventions, surveillance and identification, industrial inspection, computer assisted manufacture, computer assisted maintenance, or telemanipulation in remote o r hostile environments. Registration is performed by optimizing a similarit y measure with respect to the transformation parameters. We propose a novel similarity measure based on "photo-consistency." For each surface point, t he similarity measure computes how consistent the corresponding optical ima ge information in each view is with a lighting model. The relative pose of the optical images must be known. We validate the system using data from an optical-based surface reconstruction system and surfaces derived from magn etic resonance (MR) images of the human face. We test the accuracy and robu stness of the system with respect to the number of video images, video imag e noise, errors in surface location and area, and complexity of the matched surfaces. We demonstrate the algorithm working on 10 further optical-based reconstructions of the human head and skin surfaces derived from MR images of the heads of five volunteers. Matching four optical images to a surface model produced a 3D error of between 1.45 and 1.59 mm, at a success rate o f 100 percent, where the initial misregistration was up to 16 mm or degrees from the registration position.