A longitudinal evaluation of an educational software program: A case studyof Urinalysis-Tutor (TM)

Purpose. To examine students' learning before and after revising an educati onal software program and to explore students' patterns of use of an intera ctive feature that compares images. Method. Study participants were 466 University of Washington School of Medi cine students. Two cohorts of students (one in 1996 and one in 1997) used t he original version of the software. Following analysis of the students' le arning, the software program was modified based on instructional design pri nciples pertaining to visual learning and concept acquisition. A 1998 cohor t of students used the revised program and their performance was compared w ith that of the 1996 cohort. Analyses were based on pre- and post-test scor es, data collected from the observation of students, and navigational pathw ays tracked by the program. Results. There was very little difference in the overall performances of th e students who used the original program and those who used the revised pro gram. Error analysis focusing on 11 conceptual areas showed that reductions in errors occurred for six of 11 concepts, with statistically significant reductions of errors for two concepts. Additional navigational data collect ed in 1998 showed that students used an interactive feature for comparing i mages in different patterns. The data showed a positive association between performance and the anchored viewing mode of image display. Conclusions. While this study cannot point to specific design components th at facilitated or hindered learning, it demonstrated a potential benefit of linking usage-pattern data and performance. Future studies should evaluate design factors that affect usage patterns and performances based on naviga tional data collected while students interact with software programs.