STANDARD-SETTING - THE NEXT-GENERATION (WHERE FEW PSYCHOMETRICIANS HAVE GONE BEFORE)

Two major testing practices that emerged in the decade of the 1990s--t he use of polytomous item formats and multiple cut-scores--have stimul ated a new generation of standard-setting methods. These methods are r eviewed in the context of national and state testing programs. The bes t strategies from the decade of the 1980s with a proven track record a re then combined with the most promising new techniques into a Generic Eclectic Method (GEM) for standard setting. This GEM provides a struc ture and IO-step iterative, behavioral-anchoring judgmental process th at can be applied to almost any educational, licensure, or certificati on test. Decisions on six issues are required to use GEM in a specific cut-score situation: (a) examinee target population (students, teache rs, administrators, etc.), (b) unit of judgment (item, item cluster, o r work sample), (c) item scoring format (dichotomous, polytomous, or a combination of both), (d) test-centered (unscored unit) or examinee-c entered (previously scored unit or item response theory scale) approac h, (e) number of achievement levels (cut-scores), and (f) optional wei ghting of objectives for decision policy analysis. Various types of re liability and validity evidence of the effectiveness of the judgmental process are described, and directions for future research are suggest ed.