Augmented reality in echocardiography - A new method of computer supportedtraining and image processing using virtual and real three-dimensional data sets

Augmented reality (AR) applications link real with virtual image data, in o rder to increase their information content. In medicine they are especially useful for education and for supporting the interpretation of three-dimens ional (3D) image data. Simulators are used to train risky or expensive proc edures, In the AR application EchoCom2 a 3D surface model of the human hear t is linked with echocardiographic volume data sets. The 3D echocardiograph ic data sets are registered with the heart model to synchronize it's tempor al and spatial orientation. The heart model together with an animated ultra sound sector represents a reference scenario, which displays the currently selected cutting plane within the echocardiographic volume data set. Modifi cations of the cutting plane within the echocardiographic data are transfer red simultaneously and in real time to the reference scenario. The AR appli cation is used as a simulator to train two-dimensional echocardiographic ex aminations and as an orientation and navigation aid for the exploration of 3D echocardiographic data sets. Beginners in echocardiography have only a r udimentary conception of the spatial relationship between the actual ultras ound image and the 3D anatomy of the heart. They are unable to translate mu ltiple two-dimensional slices into a coherent 3D mental image of the heart. In EchoCom2 the trainee can interactively explore the 3D heart model and t he registered 3D echocardiographic data sets by the animated ultrasound sec tor, whose position is controlled by an electromagnetic orientation and pos ition system (EPOS). The data from the EPOS are used to calculate the echoc ardiographic images that are analogue to the position of the animated ultra sound sector. EchoCom2 is also used to support the interpretation of 3D ech ocardiographic data sets. The analysis of 3D echocardiographic data has to be done during a post processing. Defining the exact position of a cutting plane within the volume is difficult due to the lack of a standardized repr esentation, the independence of the cutting plane of any transducer positio n and the possibility to calculate an indefinite number of views. The simul taneous representation of the current cutting plane both in the volume data , and in the heart model enables the examiner ad hoc to recognize it's posi tion and the visualized structures.