Straight lines have to be straight - Automatic calibration and removal of distortion from scenes of structured environments

Most algorithms in 3D computer vision rely on the pinhole camera model beca use of its simplicity, whereas video optics, especially low-cost wide-angle or fish-eye lenses, generate a lot of non-linear distortion which can be c ritical. To find the distortion parameters of a camera, we use the followin g fundamental property: a camera follows the pinhole model if and only if t he projection of every line in space onto the camera is a line. Consequentl y, if we find the transformation on the video image so that every line in s pace is viewed in the transformed image as a line, then we know how to remo ve the distortion from the image. The algorithrn consists of first doing ed ge extraction on a possibly distorted video sequence, then doing polygonal approximation with a large tolerance on these edges to extract possible lin es from the sequence, and then finding the parameters of our distortion mod el that best transform these edges to segments. Results are presented on re al video images, compared with distortion calibration obtained by a full ca mera calibration method which uses a calibration grid.