Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: an experimental study on the script concordance test

Background The script concordance (SC) test was conceived to measure knowle dge organization, the presence of links between items of knowledge which al low for interpretation of data in clinical decision making situations. Earl ier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This stu dy explores the predictive validity of the test. Objectives To verify whether scores obtained by students at the end of cler kship predict their clinical reasoning performance at the end of residency. Design Comparison of scores obtained on a SC test taken at the end of clerk ship with those obtained 2 years later at the end of residency on two clini cal reasoning assessments of known validity, called the short-answer manage ment problems (SAMPs) and the simulated office orals (SOOs), and an objecti ve structured clinical examination (OSCE) aimed at assessing hands-on skill s and clinical reasoning. Data were treated by Pearson correlation analysis . Subjects and setting A cohort of 24 students from a medical school in Quebe c was followed up to the end of their residency in family medicine, complet ed in several schools across Quebec. Results The observed Pearson correlation coefficients of the SC test were s tatistically significant (0.451, P = 0.013; 0.447; P = 0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically sig nificant (0.340, P = 0.052) when compared with the OSCE. Conclusion The authors assumed that thr richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on t he measures of clinical reasoning (SAMP and SOO), but would predict less we ll performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study an coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also ind icates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in trainin g, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if th e later measures bear on a wider clinical domain.