EXTRACTION OF HIGH-RESOLUTION FRAMES FROM VIDEO SEQUENCES

The human visual system appears to be capable of temporally integratin g information in a video sequence in such a way that the perceived spa tial resolution of a sequence appears much higher than the spatial res olution of an individual frame. While the mechanisms in the human visu al system that do this are unknown, the effect is not too surprising g iven that temporally adjacent frames in a video sequence contain sligh tly different, but unique, information. This paper addresses how to us e both the spatial and temporal information present in a short image s equence to create a single high-resolution video frame. A novel observ ation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesia n restoration with a discontinuity-preserving prior image model is use d to extract a high-resolution,ideo still given a short low-resolution sequence. Estimates computed from a low-resolution image sequence con taining a subpixel camera pan show dramatic visual and quantitative im provements over bilinear, cubic B-spline, and Bayesian single frame in terpolations. Visual and quantitative improvements are also shown for an image sequence containing objects moving with independent trajector ies. Finally, the video frame extraction algorithm is used for the mot ion-compensated scan conversion of interlaced video data, with a visua l comparison to the resolution enhancement obtained from progressively scanned frames.