Experimental design heuristics for scientific discovery: the use of "baseline" and "known standard" controls

What type of heuristics do scientists use when they design experiments? In this paper, we analysed the ways biological scientists designed complex exp eriments at their weekly laboratory meetings. In two studies, we found that one of the key components of experimental design is that specific types of control conditions are used in the service of different goals that are imp ortant in scientific discovery. "Baseline" control conditions are identical to the experimental manipulation, except that a key feature, such as a rea gent, is absent from the control condition and present in the experimental condition. "Known standard" control conditions involve performing the exper imental technique on materials where the expected result is already well kn own; if the expected result is obtained, the scientist can have confidence that the procedure is working. In Study 1, which analysed transcripts of re al-world biology laboratory meetings, we found that scientists used baselin e controls when testing hypotheses and known standard controls when focusin g on possible error. In Study 2, undergraduate science students were asked to address the goals of hypothesis testing and dealing with potential error as they designed experiments. Like the real-world scientists, science majo rs proposed baseline controls to test hypotheses and known standard control s to deal with potential error. We argue that baseline control conditions p lay an important role in hypothesis testing: unexpected results obtained on baseline control conditions can alert scientists that their hypotheses are incorrect, and hence should encourage the scientists to reformulate their hypotheses. We further argue that use of known standard controls is a heuri stic that enables scientists to solve an important problem in real-world sc ience: when to trust their data. Both of these heuristics can be incorporat ed into experimental design programs, thus making it more likely that scien tific discoveries will be made. (C) 2000 Academic Press.