DEVELOPING AND EVALUATING THE STUDENT ASSESSMENT SYSTEM IN THE PRECLINICAL PROBLEM-BASED CURRICULUM AT SHERBROOKE

Background. Students' learning was used as an outcome measure in the f irst phases of the major curriculum reform started in 1987 by the Univ ersity de Sherbrooke Faculty of Medicine, which shifted from a traditi onal to a student-centered, problem-based learning (PBL) and community -oriented program. The system for evaluating preclinical students' lea rning is intended to reinforce the integration of basic and clinical s ciences. Method. To discover whether the evaluation system was fulfill ing its intended goals, the authors used data from the classes of 1991 -1993 to assess the reliability and validity of three evaluation instr uments. The three instruments were (1) written examinations composed o f multiple-choice questions (MCQs), short-answer questions (SAQs), and problem-analysis questions (PAQs); (2) PBL tutor rating forms that ev aluate students' reasoning skills, communication and group-interaction skills, and autonomy and humanism; and (3) clinical skills evaluation s, including objective structured clinical examinations (OSCEs). The w eights allocated to the instruments reflected how the faculty valued e ach evaluation dimension in each of the three phases of the preclinica l curriculum. Results. Reliability indexes improved throughout the sys tem implementation. The written examinations proved to have content va lidity according to PBL learning objectives. As evaluated by students, the PAQs were found to be at a taxonomic level that assessed ability to analyze information a third of the time in the first year of implem entation of the PBL curriculum and 17% in the second year. Variations and correlations of students' mean performances across instructional u nits and between the evaluation instruments led to the development of a student longitudinal performance profile to be used before yearly pr omotion decisions are proposed, The profile was introduced in the fift h year of PBL implementation. Conclusion. The system allows students t o learn higher-taxonomic-level content and fulfills the institution's social responsibility of judging program outcomes and promoting qualif ied students, although evaluation by PBL tutors is still psychometrica lly questionable and the measurement of students' reasoning and abilit y to analyze problems is still an unfinished evaluation task.