A knowledge-based approach to retrieve medical images by feature and conten
t with spatial and temporal constructs is developed. Selected objects of in
terest in a medical image (e.g., x-ray, MR image) are segmented, and contou
rs are generated from these objects. Features (e.g., shape, size, texture)
and content (e.g., spatial relationships among objects) are extracted and s
tored in a feature and content database. Knowledge about image features can
be expressed as a hierarchical structure called a Type Abstraction Hierarc
hy (TAH). The high-level nodes in the TAH represent more general concepts t
han low-level nodes. Thus, traversing along TAH nodes allows approximate ma
tching by feature and content if an exact match is not available. TAHs can
be generated automatically by clustering algorithms based on feature values
in the databases and hence are scalable to large collections of image feat
ures. Further, since TAHs are generated based on user classes and applicati
ons, they are context- and user-sensitive. A knowledge-based semantic image
model is proposed that consists of four layers (raw data layer, feature an
d content layer, schema layer, and knowledge layer) to represent the variou
s aspects of an image objects' characteristics. The model provides a mechan
ism for accessing and processing spatial, evolutionary and temporal queries
. A knowledge-based spatial temporal query language (KSTL) has developed th
at extends ODMG's OQL and supports approximate matching of feature and cont
ent, conceptual terms, and temporal logic predicates. Further, a visual que
ry language has been developed that accepts point click-and-drag visual ico
nic input on the screen that is then translated into KSTL. User models are
introduced to provide default parameter values for specifying query conditi
ons. We have implemented a Knowledge-Based Medical Database System (KMeD) a
t UCLA, and it is currently under evaluation by the medical staff. The resu
lts from this research should be applicable to other multimedia information
systems as well.