The authors extend the usual approach to the assessment of test or rat
er reliability to situations that have previously not been appropriate
for the application of this standard (Spearman-Brown) approach. Speci
fically, the authors (a) provide an accurate overall estimate of the r
eliability of a test (or a panel of raters) comprising 2 or more diffe
rent kinds of items (or raters), a quite common situation, and (b) pro
vide a simple procedure for constructing the overall instrument when i
t comprises 2 or more kinds of items, judges, or raters, each with its
own costs and its own reliabilities.