In order to compare the quality of CPR performance after various train
ing methods, training outcome assessment must provide meaningful data
and do it in a way that is reliable. Few studies have provided details
of their assessment procedures, and even fewer report on whether the
measures to evaluate performance are reliable (yielding information co
nsistently over multiple trials), or valid (measuring the outcome inte
nded). Few studies have attempted to replicate assessment methods used
by other authors. Conventional skill sheets have not been shown to as
sess compressions and ventilations reliably and validly, When using an
instrumented manikin, skill checklists can be simplified by eliminati
ng qualitative assessment of compressions and ventilations. Using a sa
mple of 171 CPR trainees rated by trained evaluators, we provide detai
ls of agreement between two evaluators and use an established statisti
c (Cronbach's cr) to assess the reliability of a 14-item simplified CP
R checklist. The level of agreement between two raters was high (Pears
on product-moment correlation = 0.87) as was the reliability estimate
obtained by Cronbach's or (0.89). As criterion-related evidence of the
validity of the CPR checklist to assess CPR performance, a correlatio
n with a five-point subjective overall rating of CPR was estimated (Sp
earman correlation = 0.92). We urge standardized reporting of CPR trai
ning outcomes in order to achieve comparability across studies.