An experimental comparison of reading techniques for defect detection in UML design documents

The basic motivation for software inspections is to detect and remove defec ts before they propagate to subsequent development phases where their detec tion and removal become more expensive. To maximize this potential, the exa mination of the artefact under inspection must be as thorough and detailed as possible. This implies the need for systematic reading techniques that t ell inspection participants what to look for and, more importantly, how to scrutinize a software document. Recent research efforts have investigated t he benefits of scenario-based reading techniques for defect detection in fu nctional requirements and functional code documents. A major finding has be en that these techniques help inspection teams find more defects than exist ing state-of-the-art approaches, such as, ad hoc or checklist-based reading (CBR). In this paper we describe and experimentally compare one scenario-b ased reading technique, namely perspective-based reading (PBR), for defect detection in object-oriented design documents using the notation of the uni fied modeling language (UML) to the more traditional CSR approach. The comp arison was performed in a controlled experiment with 18 practitioners as su bjects. Our results indicate that PER teams discovered, on average, 58% of the defects and had an average cost per defect ratio of 56 min per defect, In this way, PER is more effective than CBR (i.e., it resulted in inspectio n teams detecting, on average, 41% more unique defects than CBR). Moreover, the cost of defect detection using PER is significantly lower than CBR (i. e., PER exhibits, on average, a 58% cost per defect improvement over CBR). This study therefore provides evidence demonstrating the efficacy of PER sc enarios for defect detection in UML design documents. In addition, it demon strates that a PER inspection is a promising approach for improving the qua lity of models developed using the UML notation. (C) 2000 Elsevier Science Inc. All rights reserved.