Spatio-temporal composition of video objects: Representation and querying in video database systems

A key characteristic of video data is the associated spatial and temporal s emantics. It is important that a video model models the characteristics of objects and their relationships in time and space. Allen's 13 temporal rela tionships [2] are often used in formulating queries that contain the tempor al relationships among video frames. For the spatial relationships, most of the approaches are based on projecting objects on a two or three-dimension al coordinate system. However, very few attempts have been made formally to represent the spatio-temporal relationships of objects contained in the vi deo data and to formulate queries with spatio-temporal constraints. The pur pose of our work is to design a model representation for the specification of the spatio-temporal relationships among objects in video sequences. The model describes the spatial relationships among objects for each frame in a given video scene and the temporal relationships (for this frame) of the t emporal intervals measuring the duration of these spatial relationships. It also models the temporal composition of an object, which reflects the evol ution of object's spatial relationships over the subsequent frames in the v ideo scene and in the entire video sequence. Our model representation also provides an effective and expressive way for the complete and precise speci fication of distances among objects in digital video. This model is a basis for the annotation of raw video.