Lm. Baker et K. Dunbar, Experimental design heuristics for scientific discovery: the use of "baseline" and "known standard" controls, INT J HUM-C, 53(3), 2000, pp. 335-349
What type of heuristics do scientists use when they design experiments? In
this paper, we analysed the ways biological scientists designed complex exp
eriments at their weekly laboratory meetings. In two studies, we found that
one of the key components of experimental design is that specific types of
control conditions are used in the service of different goals that are imp
ortant in scientific discovery. "Baseline" control conditions are identical
to the experimental manipulation, except that a key feature, such as a rea
gent, is absent from the control condition and present in the experimental
condition. "Known standard" control conditions involve performing the exper
imental technique on materials where the expected result is already well kn
own; if the expected result is obtained, the scientist can have confidence
that the procedure is working. In Study 1, which analysed transcripts of re
al-world biology laboratory meetings, we found that scientists used baselin
e controls when testing hypotheses and known standard controls when focusin
g on possible error. In Study 2, undergraduate science students were asked
to address the goals of hypothesis testing and dealing with potential error
as they designed experiments. Like the real-world scientists, science majo
rs proposed baseline controls to test hypotheses and known standard control
s to deal with potential error. We argue that baseline control conditions p
lay an important role in hypothesis testing: unexpected results obtained on
baseline control conditions can alert scientists that their hypotheses are
incorrect, and hence should encourage the scientists to reformulate their
hypotheses. We further argue that use of known standard controls is a heuri
stic that enables scientists to solve an important problem in real-world sc
ience: when to trust their data. Both of these heuristics can be incorporat
ed into experimental design programs, thus making it more likely that scien
tific discoveries will be made. (C) 2000 Academic Press.