Two of the most difficult problems in Artificial Intelligence are proc
essing visual scenes and processing natural languages. There has been
a large amount of research in each of these fields but little on their
integration. This is surprising given the potential importance of int
egrated systems, not only for understanding human cognition but also f
or the range of practical applications that will be enabled. We review
previous work and provide an overview of our own work. We focus upon
the medical application of reconstructing complicated cerebral blood v
essel structures and associated pathologies from images and medical re
ports. This gives our work a clear and significant practical aim. We s
how how the ostensibly disparate technologies can be married using a s
ingle knowledge representation. Previous attempts at reconstruction ha
ve used images alone and no satisfactory solution exists. We believe t
hat the synergy provided by integrating vision and natural language pr
ocessing provides an information-rich environment that will enable pro
gress toward an efficient and robust solution. Such an integration wil
l have not only have important practical uses but also implications fo
r Artificial Intelligence, Cognitive science, Philosophy, and Psycholo
gy.