NETRA-V - TOWARD AN OBJECT-BASED VIDEO REPRESENTATION

We present here a prototype video analysis and retrieval system, calle d NeTra-V, that is being developed to build an object-based video repr esentation for functionalities such as search and retrieval of video o bjects. A region-based content description scheme using low-level visu al descriptors is proposed. In order to obtain regions for local featu re extraction, a new spatio-temporal segmentation and region-tracking scheme is employed. The segmentation algorithm uses all three visual f eatures: color, texture, and motion in the video data. A group process ing scheme similar to the one in the MPEG-2 standard is used to ensure the robustness of the segmentation. The proposed approach can handle complex scenes with large motion. After segmentation, regions are trac ked through the video sequence using extracted local features. The res ults of tracking are sequences of coherent regions, called ''subobject s.'' Subobjects are the fundamental elements in our low-level content description scheme, which can be used to obtain meaningful physical ob jects in a high-level content description scheme. Experimental results illustrating segmentation and retrieval are provided.