A method of scoring the severity of medication errors that does not require
knowledge of patient outcomes was developed and tested.
Thirty health care professionals from four U.K. hospitals scored 50 medicat
ion errors in terms of potential patient outcomes on a scale of 0 to 10, wh
ere 0 represented a case with no potential effect and 10 a case that would
result in death. Sixteen error cases reported in the literature with actual
patient outcomes were included among the cases to assess the validity of t
he scores. Ten of the errors were scored twice. The severity of the error c
ases, the occasion on which they were scored, the judge, each judge's profe
ssion, and the interactions between these were considered as potential sour
ces of variability in scoring. The data were analyzed by applying generaliz
ability theory to two models: one based on the 10 cases that were scored tw
ice and ignoring the effect of differences in profession and one based on a
ll 50 cases and ignoring the effect of the occasion of scoring. Generalizab
ility coefficients for different numbers of judges and scoring occasions we
re calculated. A generalizability coefficient of 0.8 or more was considered
to represent acceptable reliability.
Most of the variance was attributable to differences in the cases. The anal
ysis showed that to achieve a generalizability coefficient of more than 0.8
, at least four judges would have to score each case, each on one occasion,
with the mean score used as a severity indicator.
A reliable, valid method of scoring the severity of medication errors that
did not require knowledge of patient outcomes was developed; at least four
judges were required in order to achieve reliable scores, and reliability w
as not affected by the professions of the judges or the number of occasions
on which the errors were scored.