THE QUALITY OF ALCOHOL TREATMENT RESEARCH - AN EXAMINATION OF INFLUENTIAL CONTROLLED TRIALS AND DEVELOPMENT OF A QUALITY RATING SYSTEM

Background. The importance of evidence-based practice has stimulated i nterest in the methodology of clinical trials. Various weaknesses of e valuation research in the alcohol field have been indicated previously . This study set out to develop a comprehensive system fur the assessm ent of the methodological quality of outcome research for treatment of alcohol misuse and to apply the system to well-known trials in the ar ea. Methodology. A sample of the most highly cited controlled trials o f interventions for alcohol misuse was selected using the Science Cita tion Index. Thirty methodological criteria were formulated and a scori ng system devised. Two raters applied this system to the sample of tri als. Reliability testing teas performed and used to refine the criteri a. Results. Inter-rater reliability of the overall quality score was i nitially 0.85 and 0.92 after review and re-rating. Internal consistenc y was also high (0.87), Quality score correlated with year of publicat ion. Certain areas of methodology were poorly addressed in the sample including specification of main outcomes, documentation of recruitment and selection procedures, testing of blinding, analysis of withdrawal s and reporting of results. Discussion. The methodology of this sample of trials was frequently deficient in ways which might bias results o r compromise generalizability. It is recommended that the system of qu ality assessment described here is used to evaluate existing research and to inform the design of future studies.