Critical appraisal of the design and reporting of studies of imaging and measurement of carotid stenosis

Background and Purpose-Several hundred studies have been published over the last few years on imaging and measurement of carotid stenosis. Despite all this research, there is still no consensus about how best to image and mea sure stenosis. One possible explanation for this is that many of the studie s have not been large enough or methodologically sound enough to allow usef ul conclusions to be drawn. We aimed to assess the design and methods of a random sample of published studies of imaging and measurement of carotid st enosis using 9 simple criteria. Methods-A formal literature search was performed for studies of imaging and measurement of carotid stenosis. Two subsets were randomly selected for de tailed assessment: 20 studies published before 1991 and 20 published betwee n 1993 and 1997 (some years after the initial publication of the ECST and N ASCET trials). The criteria used to assess the selected studies were as fol lows: prospective rather than retrospective study design; patient selection based on a consecutive series or a random sample; adequate detail of study population; adequate detail of imaging techniques; inclusion of all invest igations, ie, patients with poor-quality imaging were not excluded; blinded assessment of images; adequate detail of derivation of measurement of sten osis from images or data; adequate data on the reproducibility of measureme nts of stenosis; and study powered according to a sample-size calculation. Results-There were many basic methodological deficiencies in both subsets o f studies, with relatively little evidence of improvement with time. For ex ample, only 33% of studies were prospective, only 45% studied a consecutive or random selection of patients, and only 38% reported any data on the rep roducibility of measurements. More than half of the studies satisfied less than or equal to 4 of the 9 quality criteria. However, there was considerab le variation between studies, with 7 studies satisfying greater than or equ al to 7 criteria and 10 studies satisfying less than or equal to 2. No stud y was based on a sample-size calculation. The number of patients studied wa s often small, particularly in the more recent studies: median sample size was 100 in the 1970-1990 studies and 58 in the 1993-1997 studies (P<0.0001) . Conclusions-The design and reporting of published studies of imaging and me asurement of carotid stenosis are poor and have not improved much in recent years. The majority of published studies are not of a sufficient standard to enable the results to be used to inform clinical practice. The utility o f future studies could be improved considerably by better adherence to 9 si mple methodological guidelines.