2-UNIT RELIABILITY-ANALYSIS OF QUESTIONNAIRES USED IN A REGULATORY SYSTEM

An interjudge reliability test was conducted to evaluate the questionn aires used in the surveillance of residential care institutions. Becau se the reliability test war carried out as part of the routine surveil lance program and not as part of a controlled experiment, it was subje ct to deviations from the optimal reliability test model. However this nonpure design provided an opportunity to not only examine the reliab ility of the items in the surveillance tool, but also to gain a better understanding of the use of a reliability test in an ''imperfect'' fi eld setting. Two different surveyor reams administered the 257 questio ns on the questionnaires to a representative sample of 32 institutions on two separate occasions. In order to explain the variance rn the re liability scores, a multivariate analysis was conducted for two units of analysis: the surveillance questions and the institutions. Based on the results of the reliability test, changes were introduced to impro ve the questionnaires and their administration.