Ca. Berg et P. Smith, ASSESSING STUDENTS ABILITIES TO CONSTRUCT AND INTERPRET LINE GRAPHS -DISPARITIES BETWEEN MULTIPLE-CHOICE AND FREE-RESPONSE INSTRUMENTS, Science education, 78(6), 1994, pp. 527-554
The author is concerned about the methodology and instrumentation used
to assess both graphing abilities and the impact of microcomputer-bas
ed laboratories (MBL) on students' graphing abilities for four reasons
: (1) the ability to construct and interpret graphs is critical for de
veloping key ideas in science; (2) science educators need to have vali
d information for making teaching decisions; (3) educators and researc
hers are heralding the arrival of MBL as a tool for developing graphin
g abilities; and (4) some of the research which supports using MBL app
ears to have significant validity problems. In this article, the autho
r will describe the research which challenges the validity of using mu
ltiple-choice instruments to assess graphing abilities. The evidence f
rom this research will identify numerous disparities between the resul
ts of multiple-choice and free-response instruments. In the first stud
y, 72 subjects in the seventh, ninth, and eleventh grades were adminis
tered individual clinical interviews to assess their ability to constr
uct and interpret graphs. A wide variety of graphs and situations were
assessed. In three instances during the interview, students drew a gr
aph that would best represent a situation and then explained their dra
wings. The results of these clinical graphing interviews were very dif
ferent from similar questions assessed through multiple-choice formats
in other research studies. In addition, insights into students' think
ing about graphing reveal that some multiple-choice graphing questions
from prior research studies and standardized tests do not discriminat
e between right answers/right reasons, right answers/wrong reasons, an
d answers scored ''wrong'' but correct for valid reasons. These result
s indicate that in some instances multiple-choice questions are not a
valid measure of graphing abilities. In a second study, the researcher
s continued to pursue the questions raised about the validity of multi
ple-choice tests to assess graphing, researching the following questio
ns: What can be learned about subjects' graphing abilities when studen
ts draw their own graphs compared to assessing by means of a multiple-
choice instrument? Does the methodology used to assess graphing abilit
ies: (1) affect the percentage of subjects who answer correctly; (2) a
lter the percentage of subjects affected by the ''picture of the event
'' phenomenon? Instruments were constructed consisting of three graphi
ng questions that asked students: (a) multiple-choice-choose a graph t
hat best represents the situation; (b) free-response-draw a graph that
best represents the situation. The sample of 1416 subjects from an ur
ban/suburban area included 50% boys/50% girls from grades 8 through 12
; subjects from high, medium, and low ability groups; and subjects fro
m both public and private schools. The subjects completed either the m
ultiple-choice or the free draw instrument. The free draw instrument w
as scored by comparing the subject's response to categories of possibl
e answers that had been identified from the first study. The results s
how as much as 19% difference in correct responses, three times as man
y ''picture of the events'' from multiple-choice inst uments, and sign
ificant differences in how multiple-choice and free-response affect va
rious ability levels and grade levels. As such, some of the research s
tudies that used multiple-choice instruments to examine graphing and t
he impact of MBL on students' graphing abilities may be invalid. (C) 1
994 John Wiley & Sons, Inc.