Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields

We consider the problems of motion-compensated frame interpolation (MCFI) a nd bidirectional prediction in a video coding environment. These applicatio ns generally require good motion estimates at the decoder, In this paper, m e use a multiscale optical-flow-based motion estimator that provides smooth , natural motion fields under bit-rate constraints. These motion estimates scale well with change in temporal resolution and provide considerable flex ibility in the design and operation of coders and decoders. In the MCFI app lication, this estimator provides excellent interpolated frames that are su perior to those of conventional motion estimators, both visually and in ter ms of peak signal-to-noise ratio (PSNR), We also consider the effect of occ lusions in the bidirectional prediction application and introduce a dense l abel field that complements our motion estimator. This label field enables us to adaptively weight the forward and backward predictions and gives us s ubstantial visual and PSNR improvements in the covered/uncovered regions of the sequence.