Textures and microstructures on the surfaces of cultural objects, such as b
uildings and parking lots, provide important information for many military
and civilian applications. The extraction of this information from aerial i
magery is difficult due to problems arising from perspective distortion, da
ta deficiency, and corruption caused by shadows and occlusions. In this pap
er we present a generic surface texture and microstructure extraction syste
m to deal with these problems, under the assumption that an initial 3-D geo
metric site model is given and sufficient camera and light source informati
on is known. The infrastructure of the system is an orthographic facet imag
e library that systematically collects surface intensities from multiple ae
rial images into a database, eliminates the effects of shadows and occlusio
ns, and forms a complete and consistent surface intensity representation. A
robust, local intensity based image analysis module is designed to symboli
cally extract surface microstructures with rectilinear and repetitive patte
rns. The paper shows successful applications of the system to extractions o
f both 2-D and 3-D microstructures. The system is typically useful for urba
n site model refinement and visualization. (C) 2001 Academic Press.