INDEXING AND RETRIEVAL OF THE MPEG COMPRESSED VIDEO

To keep pace with the increased popularity of digital video as an arch ival medium, the development of techniques for fast and efficient anal ysis of video streams is essential. In particular, solutions to the pr oblems of storing, indexing, browsing, and retrieving video data from large multimedia databases are necessary to allow access to these coll ections. Given that video is often stored efficiently in a compressed format, the costly overhead of decompression can be reduced by analyzi ng the compressed representation directly. In earlier work, we present ed compressed domain parsing techniques which identified shots, subsho ts, and scenes. In this article, we present efficient key frame select ion, feature extraction, indexing, and retrieval techniques that are d irectly applicable to MPEG compressed video. We develop a frame type i ndependent representation which normalizes spatial and temporal featur es including frame type, frame size, macroblock encoding, and motion c ompensation vectors. Features for indexing are derived directly from t his representation and mapped to a low-dimensional space where they ca n be accessed using standard database techniques. Spatial information is used as primary index into the database and temporal information is used to rank retrieved clips and enhance the robustness of the system . The techniques presented enable efficient indexing, querying, and re trieval of compressed video as demonstrated by our system which typica lly takes a fraction of a second to retrieve similar video scenes from a database, with over 95% recall (C) 1998 SPIE and IS&T. [S1017-9909( 98)00302-X].