CALIBRATION-FREE AUGMENTED REALITY

Camera calibration and the acquisition of Euclidean 3D measurements ha ve so far been considered necessary requirements for overlaying three- dimensional graphical objects with live video. In this article, we des cribe a new approach to video-based augmented reality that avoids both requirements: It does not use any metric information about the calibr ation parameters of the camera or the 3D locations and dimensions of t he environment's objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unk nown. Our approach is based on the following observation: Given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by 1) tracking r egions and color fiducial points at frame rate, and 2) representing vi rtual objects in a non-Euclidean, affine frame of reference that allow s their projection to be computed as a linear combination of the proje ction of the fiducial points. Experimental results on two augmented re ality systems, one monitor-based and one head-mounted, demonstrate tha t the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically.