Foveated video compression with optimal rate control

Reccntly, foveated video compression algorithms have been proposed which, i n certain applications, deliver high-quality video at reduced bit rates by seeking to match the nonuniform sampling of the human retina. We describe s uch a framework here where foveated video is created by a nonuniform filter ing scheme that increases the compressibility of the video stream. We maxim ize a new foveal visual quality metric, the foveal signal-to-noise ratio (F SNR) to determine the best compression and rate control parameters for a gi ven target bit rate. Specifically, we establish a new optimal rate control algorithm for maximizing the FSNR using a Lagrange multiplier method define d on a curvilinear coordinate system. For optimal rate control, we also dev elop a piecewise R-D (rate-distortion)/R-Q (rate-quantization) model, A fas t algorithm for searching for an optimal Lagrange multiplier lambda* is sub sequently presented. For the new models, we show how the reconstructed vide o quality is affected, where the FPSNR is maximized, and demonstrate the co ding performance for H.263,+,++/MPEG-4 video coding. For N.263/MPEG video c oding, a suboptimal rate control algorithm is developed for fast, high-perf ormance! applications. In the simulations, we compare the reconstructed pic tures obtained using optimal rate control methods for foveated and normal v ideo. We show that foveated video coding using the suboptimal rate control algorithm delivers excellent performance under 64 kb/s.