ANALYSES OF FACTORS RELATED TO POSITIVE TEST BIAS IN SOFTWARE TESTING

In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon i n which hypothesis testers tend to test a hypothesis with data which c onfirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with da ta which are designed to disconfirm the hypothesis. The first study co nsidered how positive test bias is influenced by the expertise level o f the subjects, the completeness of the software specifications and wh ether or not the programs contained errors. The results demonstrated s trong evidence of positive test bias regardless of condition. The effe cts appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some ca ses, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses tes ters generate during testing. The results further emphasize that subje cts test their programs in a biased way and support the notion that th e program specification drives testers' hypotheses. We conclude that p ositive test bias is a critical concern in software testing and may ha ve a seriously detrimental effect on the quality of testing. The resul ts further emphasize the importance of complete and thorough program s pecifications in order to enhance effective testing.