INTRODUCING PROGRESS TESTING IN MCMASTER UNIVERSITY PROBLEM-BASED MEDICAL CURRICULUM - PSYCHOMETRIC PROPERTIES AND EFFECT ON LEARNING

Background. The progress test (or Quarterly Profile Examination), inve nted concurrently by the University of Missouri-Kansas City School of Medicine and the University of Limburg, is used to provide useful summ ative and formative judgments about students' knowledge without distor ting learning. All students in all classes sit the same examination at -regular intervals through the year, and their individual progress is noted. This paper reports on four years' experience with a progress te st, the Personal Progress Index (PPI), at the McMaster University Facu lty of Health Sciences. Method. The PPI, a 180-item multiple-choice te st with items drawn from all disciplines of medicine, is administered to medical students in all three classes three times per year. Individ ual feedback is provided, and accumulated student performance is deter mined for summative purposes. This paper examines extensive evidence o f reliability validity, and effect on student learning, using samples from the entering classes of 1992-1995 (a total of 400 students). Resu lts. Reliabilities across test administrations (within classes) ranged from .46 to .63. The PPI demonstrated strong construct validity, with highly significant statistical tests of differences between classes a nd changes within classes on successive administrations. The predictiv e validity of the PPI, i.e., whether it could predict performance on t he licensing examination of the Medical Council of Canada, increased m onotonically from a correlation of .12 for the first test administrati on one month into medical school to a high of about .60 for the cumula tive score across all administrations three months prior to the examin ation. Conclusion. The PPI seems to be performing as intended, with st udents showing growth in performance across the three years of the MD program. Additional benefits are that many more students now self-refe r for remediation (based on low PPI scores) and that the consistent re lative performances of individual students across test administrations allow for the identification of students who have severe and persiste nt problems.