Stulberg classification system for evaluation of Legg-Calve-Perthes Disease: Intra-rater and inter-rater reliability

Background Researchers and clinicians commonly use the classification syste m of Stulberg et al, as a basis for treatment decisions during the active p hase of Legg-Calve-Perthes disease because of its putative utility as a pre dictor of long-term outcome. It is generally assumed that this system has a n:acceptable degree of reliability. This assumption, however, is not convin cingly supported by the literature. Methods: The purpose of the present study was to assess the inter-rater and intra-rater reliability of the classification system of Stulberg et al, wi th use of a pre-test, post-test design. During the pre-test phase, nine rat ers independently used the system to evaluate the radiographs of skeletally mature patients who had been managed for Legg-Calve-Perthes disease. The i ntervention between the pre-test and post-test phases consisted of a consen sus-building session during which all raters jointly arrived at standardize d definitions of the various joint structures that are assessed with use of the classification system. The effect of these definitions on reliability then was assessed by reevaluating the radiographs during the post-test phas e. Results: The pre-test intra-rater reliability coefficients ranged from 0.70 9 to 0.915, and the post-test coefficients ranged from 0.568 to 0.874. The pre-test inter-rater reliability coefficients ranged from 0.603 to 0.732, a nd the post-test coefficients ranged from 0.648 to 0.744, Contributing to t he variance was a lack of agreement concerning the assessment of joint stru ctures and the way in which the raters translated these evaluations into a classification according to the system of Stulberg et al, Conclusions: Although intra-rater reliability was marginally acceptable, th e degree of variability between the classifications assigned by different r aters even after the intervention - calls into question the reliability of the system of Stulberg et al.; consequently, the validity of any treatment decisions, outcome evaluations, or epidemiological studies based on this sy stem is also in question.