Video summarization using descriptors of motion activity: A motion activity based approach to key-frame extraction from video shots

We describe a video summarization technique that uses motion descriptors co mputed in the compressed domain, It can either speed up conventional color- based video summarization techniques, or rapidly generate a key-frame based summary by itself. The basic hypothesis of the work is that the intensity of motion activity of a video segment is a direct indication of its "summar izability," which we experimentally verify using the MPEG-7 [MPEG-7 Visual Committee Draft URL: http.,//www.cselt.it/mpeg/working documents.htm offici al MPEG site] motion activity descriptor and the fidelity measure proposed in H. S. Chang, S. Sull, and S. U. Lee, "Efficient video indexing scheme fo r content-based retrieval, " IEEE Trans, Circuits Syst. Video Technol. 9(8) , (1999). Note that the compressed domain extraction of motion activity int ensity is much simpler than the color-based calculations. We are thus able to quickly identify easy to summarize segments of a video sequence since th ey have a low intensity of motion activity. We are able to easily summarize these segments by simply choosing their first frames. We can then apply co nventional color-based summarization techniques to the remaining segments. We thus speed up color-based summarization by reducing the number of segmen ts processed. Our results also motivate a simple and novel key-frame extrac tion technique that relies on a motion activity based nonuniform sampling o f the frames. Our results indicate that it can either be used by itself or to speed up color-based techniques as explained earlier. (C) 2001 SPIE and IS&T.