A decision-analytic stopping rule for validation of commercial software systems

The decision about when to release a software product commercially is not a question of when the software has attained some objectively justifiable de gree of correctness. It is, rather, a question of whether the software achi eves a reasonable balance among engineering objectives, market demand, cust omer requirements, and marketing directives of the software organization. I n this paper, we present a rigorous framework for addressing this important decision. Conjugate distributions from statistical decision theory provide an attractive means of modeling the cost and rate of bugs given informatio n acquired during software testing, as well as prior information provided b y software engineers about the fidelity of the software before testing begi ns. In contrast to methods such as [1] and [15], the stopping analysis pres ented here yields a computationally simple rule for deciding when to releas e a commercial software product based on information revealed to engineers during software testing-complicated numerical procedures are not needed. Ou r method has the added benefits that it is sequential: It measures explicit ly the costs of customer dissatisfaction associated with bugs as well as th e costs of declining market position while the testing process continues; a nd it incorporates a practical framework for cost-criticality assessment th at makes sense to professional software developers. A probabilistic model o f catastrophic bugs provides another useful way of characterizing and measu ring the software's expected performance after commercial release. Taken to gether, these tools provide a software organization with a clearer basis fo r making decisions about when to release a commercial software product.