EXPERIMENTAL-MODELS FOR VALIDATING TECHNOLOGY

Experimentation helps determine the effectiveness of proposed theories and methods. But computer science has not developed a concise taxonom y of methods that could be used to demonstrate the validity of new sof tware development techniques. Partly as a result of this, the state of experimentation in software engineering is poor. In this article, the authors describe the: four general approaches to experimentation and the important aspects of data collection. They propose a taxonomy for software engineering experimentation that describes 12 different exper imental approaches. To underscore the need for experimentation, the au thors also analyze 612 papers in software engineering published in 198 5, 1990, and 1995. They classified each paper according to the data co llection method used to validate the claims in the paper. They found t hat about a third of the papers had no experimental validation, and ma ny authors used sloppy terminology to describe their validation method . The software engineering community, the authors conclude, can do a b etter job reporting its results. This, in turn, will help industry sel ect the right new technology to implement.