INTEROBSERVER AGREEMENT FOR 10-PERCENT CATEGORIES OF ANGIOGRAPHIC CAROTID STENOSIS

Background and Purpose Although the reliability of the assessment of s evere 70% to 99% carotid stenosis by carotid angiography has been prov en excellent, this may not necessarily be the case for a more detailed classification of carotid stenoses by 10% categories. Methods Angiogr ams of the carotid arteries were assessed pairwise by three independen t, experienced observers. The measurements of the degree of stenosis o f both the carotid bifurcation and the internal carotid artery were ma de according to the European Carotid Surgery Trial method. Kappa stati stics were used to assess the agreement beyond chance for severe (70% to 99%) carotid stenosis (kappa(1)) and for 10% categories of carotid stenosis (kappa(2)). The penalty scores were adjusted by weights for t he relative difference in risk (RDR) of stroke in the ipsilateral caro tid distribution between the 10% categories (kappa(3)). An adjustment of the RDR method was made by assuming that only patients with a sever e carotid stenosis would undergo surgery, and the penalty would be 0 i f no disagreement would exist about the indication for surgery (kappa( 4)). An even further adjustment (kappa(5)) was made by assuming that a ssessment of the rate of carotid stenosis by one or both observers wou ld lead to different treatment recommendations in 50% of the cases, an d accordingly the penalty for disagreement (RDR) was halved. Results O ne hundred twenty-one carotid bifurcations in 65 patients with a trans ient ischemic attack or nondisabling stroke were assessed. The intracl ass correlation between the exact estimates of carotid stenosis was .9 0 (95% confidence interval, .85 to .92). The mean difference in stenos is between the two raters was 0.8% (95% confidence interval, -2.1% to 3.7%). kappa(1) to kappa(5) equaled 0.80, 0.40, 0.79, 0.91, and 0.92 r espectively. Conclusions Interobserver agreement for distinct 10% cate gories of angiographic carotid stenosis is moderate, but when realisti c risk-and decision-based weights are used, agreement between experien ced observers can be almost perfect.