COMPARING DETECTION METHODS FOR SOFTWARE REQUIREMENTS INSPECTIONS - AREPLICATED EXPERIMENT

Software requirements specifications (SRS) are often validated manuall y. One such process is inspection, in which several reviewers independ ently analyze all or part of the specification and search for faults. These faults are then collected at a meeting of the reviewers and auth or(s). Usually, reviewers use Ad Hoc or Checklist methods to uncover f aults. These methods force all reviewers to rely on nonsystematic tech niques to search for a wide variety of faults. We hypothesize that a S cenario-based method, in which each reviewer uses different, systemati c techniques to search for different, specific classes of faults, will have a significantly higher success rate. We evaluated this hypothesi s using a 3x2(4) partial factorial, randomized experimental design. Fo rty eight graduate students in computer science participated in the ex periment. They were assembled into sixteen, three-person teams. Each t eam inspected two SRS using some combination of Ad Hoc, Checklist or S cenario methods. For each inspection we performed four measurements: 1 ) individual fault detection rate, 2) team fault detection rate, 3) pe rcentage of faults first identified at the collection meeting (meeting gain rate), and 4) percentage of faults first identified by an indivi dual, but never reported at the collection meeting (meeting loss rate) . The experimental results are that 1) the Scenario method had a highe r fault detection rate than either Ad Hoc or Checklist methods, 2) Sce nario reviewers were more effective at detecting the faults their scen arios are designed to uncover, and were no less effective at detecting other faults than both Ad Hoc or Checklist reviewers, 3) Checklist re viewers were no more effective than Ad Hoc reviewers, and 4) Collectio n meetings produced no net improvement in the fault detection rate-mee ting gains were offset by meeting losses.