The dependability and interchangeability of assessment methods in science

In this study, we investigated the importance of occasion as a source of er ror variance in (a) estimates of the dependability (generalizability) of sc ience assessment scores and (b) the interchangeability of science test form ats. Two science tests were developed to measure 8th-grade students' knowle dge of concepts related to electricity and electric circuits: A hands-on as sessment that provided students with equipment to manipulate and an analogo us pencil-and-paper version. Students were administered both tests on 2 occ asions, approximately 1 month apart. Results of the univariate generalizabi lity results showed that explicitly recognizing occasion as a facet of erro r variance altered the interpretation about the substantial sources of erro r in the measurement and gave lower estimates of the dependability of scien ce scores. Including occasion as an explicit source of variance in the mult ivariate generalizability analyses influenced the interpretation of the obs erved correlation between hands-on and pencil-and-paper scores but had litt le influence on the estimated disattenuated correlation between assessment methods.