AGREEMENT BETWEEN COMMITTEE-BASED AND FIELD-BASED JOB ANALYSES - A STUDY IN THE CONTEXT OF LICENSURE TESTING

The authors examined agreement between importance ratings used in job analysis. The ratings were obtained from small committees of content e xperts and from field-survey respondents. Three measures of agreement were used: a relative index (product-moment correlation), an absolute index (intraclass correlation), and a dichotomous index(cutpoint). Dat a were obtained from 2 job analysis studies conducted for purposes of developing teacher-licensure tests in Spanish and in chemistry. In bot h studies there was a high level of agreement as measured by the relat ive and dichotomous indexes and a moderately high level of agreement a s measured by the absolute index. The implications of these findings f or job analysts and test developers are discussed.