SEMCOG: A hybrid object-based image and video database system and its modeling, language, and query processing

Multimedia data types, such as image and video, are structurally more compl ex than traditional data types. We view an image as a compound object conta ining many sub-objects. Each sub-object corresponds to image regions that a re visually and semantically meaningful (e.g., car, man, etc.). In this pap er, we first present a framework for modeling images and videos using objec t-based hierarchical structures. The proposed structure supports image retr ieval at both whole-image and object levels. We show how to extend the obje ct-based image modeling framework to video retrieval at object, frame, acti on, and composite action levels. The advantages of our approach are as foll ows: (1) hierarchical model allows translation of video retrieval queries t o image retrieval queries with additional temporal relationships; (2) conce pt definitions release users from learning the system and implementation de tails and the language syntax; and (3) IFQ, a visual query interface, speci fically designed for object-based image retrieval is extended for video ret rieval query interfaces. (C) 1999 John Wiley & Sons, Inc.