The effect on reliability of adding a separate written assessment component to an objective structured clinical examination

Purpose To determine the effect on test reliability when a separate written assessment component is added to an objective structured clinical examinat ion (OSCE). Method Volunteers (n=38) from Maastricht Medical School were recruited to t ake a skills-related knowledge test in addition to their regular end-of-yea r OSCE. The OSCE scores of these volunteers did not differ from those of th e other students of their class. Multivariate generalizability theory was u sed to investigate the combined reliability of the two test formats as well as their respective contributions to overall reliability. Results Combining the two formats has an added value. The loss of reliabili ty due to the use of fewer stations in the OSCE can be fully compensated by lengthening the written test component. Conclusion From the perspective of test reliability, it is possible to econ omize on the resources needed for performance-based assessment by adding a separate written test component.