A validated, reliable method of scoring the severity of medication errors

A method of scoring the severity of medication errors that does not require knowledge of patient outcomes was developed and tested. Thirty health care professionals from four U.K. hospitals scored 50 medicat ion errors in terms of potential patient outcomes on a scale of 0 to 10, wh ere 0 represented a case with no potential effect and 10 a case that would result in death. Sixteen error cases reported in the literature with actual patient outcomes were included among the cases to assess the validity of t he scores. Ten of the errors were scored twice. The severity of the error c ases, the occasion on which they were scored, the judge, each judge's profe ssion, and the interactions between these were considered as potential sour ces of variability in scoring. The data were analyzed by applying generaliz ability theory to two models: one based on the 10 cases that were scored tw ice and ignoring the effect of differences in profession and one based on a ll 50 cases and ignoring the effect of the occasion of scoring. Generalizab ility coefficients for different numbers of judges and scoring occasions we re calculated. A generalizability coefficient of 0.8 or more was considered to represent acceptable reliability. Most of the variance was attributable to differences in the cases. The anal ysis showed that to achieve a generalizability coefficient of more than 0.8 , at least four judges would have to score each case, each on one occasion, with the mean score used as a severity indicator. A reliable, valid method of scoring the severity of medication errors that did not require knowledge of patient outcomes was developed; at least four judges were required in order to achieve reliable scores, and reliability w as not affected by the professions of the judges or the number of occasions on which the errors were scored.