An investigation on requantization for transcoding of video signals is carr
ied out. Specifically, MPEG-2 compatible discrete cosine transform (DCT) in
traframe coding is addressed. The aim of this work is twofold: first, to pr
ovide a theoretical analysis of the transcoding problem, and second, to der
ive quantization methods for efficient transcoding based on the results of
the analysis. The mean squared error (MSE) cost function is proposed for de
signing a quantizer with minimum distortion resulting in up to 1.3 dB gain
compared with the quantizer used in the MPEG-2 reference coder TM5, However
, the MSE quantizer leads in general to a larger hit rate and may therefore
only be applied locally to blocks of sensitive image content. A better rat
e-distortion performance can be provided by the maximum a posteriori (MAP)
cost function, In critical cases, the MAP quantizer gives a 0.4 dB larger s
ignal-to-noise-ratio (SNR) at the same bit rate compared with the TM5 quant
izer, The results are not limited to MPEG-2 and can be adapted to other cod
ing schemes such as H.263 or JPEG.