We describe a generic approach to image interpretation, based on combi
ning a general method of building flexible template models with geneti
c algorithm (GA) search. The method can be applied to a given image in
terpretation problem simply by training a statistical shape model, usi
ng a set of examples of the image structure to be located. A local opt
imization technique has been incorporated into the GA search and shown
to improve the speed of convergence and optimality of solution. We pr
esent results from three medical applications, demonstrating that the
new method offers significant improvements when compared with previous
ly reported approaches to flexible template matching, particularly the
ability to deal with different domains of application using a standar
d method and the possibility of employing complex multipart models. We
also describe how the method can be simply extended to track structur
es in image sequences and segment three dimensional objects in volume
images.