VALIDITY OF THE APPROPRIATENESS EVALUATIO N PROTOCOL

BACKGROUND: The study was carried out in order to assess the inter-obs erver reliability and validity in respect of clinical appraisal given by the Appropriateness Evaluation Protocol (AEP), in the context of th e Spanish Public Hospital System. MATERIAL ANO METHOD: In order to ass ess the reliability a total of 614 hospital stays chosen at random fro m 56 hospital admissions were independently analysed by three reviewer s (two doctors and one nurse). In order to assess the validity, the fi ndings obtained by the nurse were compared with the majority opinion g iven by the 7 hospital specialists in respect of each of hospital stay s under evaluation. As part of the analytical procedure, indices for o bserved agreement, and specific agreement were calculated, as well as the Kappa statistic, all forming of various random samples of 614 hosp ital stays. in order to assess the predictive validity of the AEP, its sensitivity, specificity and predictive values were all measured agai nst the majority clinical judgement. RESULTS: The study exhibited a hi gh degree of inter-observer reliability (specific agreement > 64%, kap pa > 0.75) and a reasonable validity in comparison with the consensus of opinions formed by a least 4 or 5 of its 7 clinical reviewers (spec ific agreement > 61%, kappa > 0.64), these values decreasing notably w hen the consensus of 6 or 7 of the reviewers was required. The AEP rev ealed a high degree of sensitivity and a low degree of specificity in comparison with the majority clinical assessment, thus minimising the occurrence of false results when the stay was regarded as appropriate, and producing false negatives (appropriate hospital stays regarded as inappropriate) varying in degree from moderate to very high. CONCLUSI ONS: The results showing high reliability and moderate validity regard ing clinical assessment shows the AEP to be a useful instrument in the sifting-out of inappropriate use of hospitalisation, although they do not allow a definitive judgement to be made concerning the efficiency of hospital services nor judgements regarding individual cases.