A MULTISTAGE DOMINANT PROFILE METHOD FOR SETTING STANDARDS ON COMPLEXPERFORMANCE ASSESSMENTS

During pilot testing of standard-setting procedures for use with compl ex performance assessments, panelists were asked to construct standard s-related policies schematically and/or textually. Schematic construct ions consisted of profiles characterizing their judgments of minimally acceptable performance on a multi-exercise performance assessment, in the sense that such performance barely warranted certification as a h ighly accomplished teacher. Some panelists also wrote descriptions of minimally acceptable performance policies. Written and constructed pol icies were compared to panelists' judgments when using the judgmental policy capturing standard-setting procedure for consistency between th e two methods, essentially a comparison of panelists' explicit and imp licit policies. The multi-stage dominant profile method shows promise for training panelists in standard setting, demonstrating implications of panelists' policies for certification decisions, and determining t he policy most preferred by panelists.